Patent Application
20220137396
Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section. In particular, disparate technologies are discussed that it would not be obvious to discuss together absent the teachings of the present invention.
Modern vehicles are often equipped with sensors designed to detect objects and landscape features around the vehicle in real-time to enable technologies such as lane change assistance, collision avoidance, and autonomous driving. Some commonly used sensors include image sensors (e.g., infrared or visible light cameras), acoustic sensors (e.g., ultrasonic parking sensors), radio detection and ranging (RADAR) sensors, magnetometers (e.g., passive sensing of large ferrous objects, such as trucks, cars, or rail cars), and light detection and ranging (LiDAR) sensors.
A LiDAR system typically uses a light source and a light detection system to estimate distances to environmental features (e.g., pedestrians, vehicles, structures, plants, etc.). For example, a LiDAR system may transmit a light beam (e.g., a pulsed laser beam) to illuminate a target and then measure the time it takes for the transmitted light beam to arrive at the target and then return to a receiver near the transmitter or at a known location. In some LiDAR systems, the light beam emitted by the light source may be steered across a two-dimensional or three-dimensional region of interest according to a scanning pattern, to generate a “point cloud” that includes a collection of data points corresponding to target points in the region of interest. The data points in the point cloud may be dynamically and continuously updated, and may be used to estimate, for example, a distance, dimension, location, and speed of an object relative to the LiDAR system.
Light steering typically involves the projection of light in a pre-determined direction to facilitate, for example, the detection and ranging of an object, the illumination and scanning of an object, or the like. Light steering can be used in many different fields of applications including, for example, autonomous vehicles, medical diagnostic devices, etc., and can be configured to perform both transmission and reception of light. For example, a light steering transmitter may include a micro-mirror to control the projection direction of light to detect/image an object. Moreover, a light steering receiver may also include a micro-mirror to select a direction of incident light to be detected by the receiver, to avoid detecting other unwanted signals. A micro-mirror assembly typically includes a micro-mirror and an actuator. In a micro-mirror assembly, a micro-mirror can be connected to a substrate via a connection structure (e.g., a torsion bar, a spring, etc.) to form a pivot point. One such type of micro-mirror assembly can be a micro-electro-mechanical system (MEMS)-type structure that may be used for a light detection and ranging (LiDAR) system in an autonomous vehicle, which can be configured for detecting objects and determining their corresponding distances from the vehicle. LiDAR systems typically work by illuminating a target with an optical pulse and measuring the characteristics of the reflected return signal. The return signal is typically captured as a point cloud. The width of the optical-pulse often ranges from a few nanoseconds to several microseconds.
Micro-mirror devices used in a LIDAR system can be designed to operate (scan) at a resonant frequency of the MEMS mirror structure for larger scanning angles. The resonant frequency can be controlled by the design of the MEMS mirror structure and the supporting torsion springs that support them. By operating at the resonant frequency, the mirror can more easily be rotated, with less power, since it tends to resonate or oscillate at that frequency. This allows the achievement of a large scanning angle with a low operating voltage. When the surrounding temperature changes, stress develops at the interface between the device and its package because of a mismatch in CTE (coefficient of thermal expansion) of the two materials. The tension within the torsion springs coupled to the suspended micro-mirror changes. This results in a shift of the micro-mirror's resonant frequency, and related system components need to adapt to the new frequency. In addition to frequency change, the tension between the micro-mirror and the package may also result in a bowed micro-mirror (ideally the micro-mirror or mirror array should be perfectly flat sitting on a silicon die substrate) and thus cause un-wanted light divergence. Accommodating for such temperature sensitivity can greatly increase the complexity of overall system.
Stress develops in the interface between the chip (die) and the package because of a mismatch in CTE (coefficient of thermal expansion) of the two materials. For example, a die could be mainly made of silicon and an enclosure could be a ceramic package which is made of alumina. The CTE of these two materials are different and they expand and contract at different rates with temperature. Alumina expands and contracts more than silicon, and thus stress develops at the interface of the two materials. This stress is transmitted to the devices in the substrate and can especially be noticeable in MEMS devices.
Embodiments provide a method and mechanism for reducing changes both in the resonant frequency and in flatness of a MEMS mirror structure with temperature due to a mismatch between the CTE of the MEMS die and the package substrate. A die attach layer with a low Young's modulus, such as less than 25,000 or less than 15,000 psi, is used to allow absorption of some of the stress due to the mismatch in the CTE of the MEMS die and the package substrate. Young's modulus (sometimes referred to as the Young modulus) is a mechanical property that measures a material's stiffness.
In addition, in embodiments a thicker die attach layer than normal is used to absorb some of the stress, increasing the height of the die attach layer from the normal range around 25 μm to between 50-150 μm thick. In addition, a thicker die substrate can absorb more stress, and the combination of a thicker die substrate and thicker die attach layer provide added benefits.
In further embodiments a pattern of open cavities is etched in the bottom of the die substrate. The die substrate may be made thicker to provide room for the cavities, as well as being thicker to absorb stress. The cavities form a pattern that further reduces stress by providing open space to absorb some of the transmitted stress due to differences in thermal expansion and contraction. A variety of patterns could be used, such as a perimeter cavity, a simple cross pattern, a cross-hatched pattern, or a circular pattern.
Embodiments provide a micro-electro-mechanical system (MEMS) apparatus for beam steering in a Light Detection and Ranging (LiDAR) system of an autonomous vehicle. A mirror mass has a reflective surface and at least first and second respective sides. First and second supporting torsion springs, wherein the first and second supporting torsion springs have first ends, respectively, connect to the first and second respective sides of the mirror mass, on opposite sides, to support the mirror mass. First and second common terminals connect to the first and second supporting torsion springs, respectively, on second ends of the first and second supporting torsion springs. A plurality of first fingers extend from the mirror mass on first and second sides orthogonal to the first and second supporting torsion springs. First and second bias terminals are opposite the first and second sides of the mirror mass. A plurality of second fingers extend from the first and second bias terminals. The plurality of second fingers are interleaved with the plurality of first fingers and partially overlap the plurality of first fingers. An oxide layer is below the first and second common terminals and the first and second bias terminals. A die substrate is below the oxide layer. A chip package has a chip package substrate. A die attach layer is between the die substrate and the chip package substrate, and is adhesively bonded to both the die substrate and the chip package substrate. The die attach layer has a Young's modulus less than 25,000 or less than 15,000 psi. The die substrate is patterned with a plurality of cavities open to the die attach layer.
Other embodiments provide a method for forming a micro-electromechanical system (MEMS) mirror device. This includes the steps of providing a die substrate, etching a plurality of cavities in the bottom of the die substrate to form a pattern of open cavities, and forming an oxide layer over the die substrate. A MEMS mirror structure is then formed over the oxide layer. The MEMS mirror structure includes:
The terms and expressions that have been employed are used as terms of description and not of limitation, and there is no intention in the use of such terms and expressions of excluding any equivalents of the features shown and described or portions thereof. It is recognized, however, that various modifications are possible within the scope of the systems and methods claimed. Thus, it should be understood that, although the present system and methods have been specifically disclosed by examples and optional features, modification and variation of the concepts herein disclosed should be recognized by those skilled in the art, and that such modifications and variations are considered to be within the scope of the systems and methods as defined by the appended claims.
This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used in isolation to determine the scope of the claimed subject matter. The subject matter should be understood by reference to appropriate portions of the entire specification of this disclosure, any or all drawings, and each claim.
The foregoing, together with other features and examples, will be described in more detail below in the following specification, claims, and accompanying drawings.
The features of the various embodiments described above, as well as other features and advantages of certain embodiments of the present invention will, be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:
Aspects of the present disclosure relate generally to a LiDAR system, and more particularly to scanning an environment with a laser and MEMS-based mirrors, and in particular to minimizing the effect of temperature changes on the MEMS mirror resonant frequency and flatness.
In the following description, various examples of MEMS-based micro mirror structures are described. For purposes of explanation, specific configurations and details are set forth in order to provide a thorough understanding of the embodiments. However, it will be apparent to one skilled in the art that certain embodiments may be practiced or implemented without every detail disclosed. Furthermore, well-known features may be omitted or simplified in order to prevent any obfuscation of the novel features described herein.
The following high level summary is intended to provide a basic understanding of some of the novel innovations depicted in the figures and presented in the corresponding descriptions provided below. Techniques disclosed herein relate generally to microelectromechanical (MEMS) mirrors that can be used in, for example, light detection and ranging (LiDAR) systems or other light beam steering systems. More specifically, and without limitation, disclosed herein are embodiments that provide a micro-electromechanical system (MEMS) die attachment method and mechanism that minimizes the effect of temperature changes on the resonant frequency.
In particular, a die attach layer is between the die substrate and the chip package substrate, and is adhesively bonded to both the die substrate and the chip package substrate. The die attach layer has a Young's modulus less than 25,000 or less than 15,000 psi, meaning that it is relatively soft, not stiff, and thus absorbs more stress. The stress is thus reduced by putting a layer with a low Young's modulus between the semiconductor die and the chip package, such as a silicone adhesive. Young's modulus defines the relationship between stress (force per unit area) and strain (proportional deformation). Young's modulus may be different in different directions in the material. Most metals and ceramics, along with a number of other materials, are isotropic, which means that their mechanical properties are the same in all orientations. Anisotropic materials have a Young's modulus value that depends on the direction of the force vector. For example, composites can be anisotropic. Metals and ceramics can be treated with certain impurities to make their grain structures directional.
A low Young's modulus thus allows stress to be absorbed by the silicone adhesive, rather than being mostly transmitted to the semiconductor die. For MEMS mirror dies, the orientation of the mirror is critical. Thus care must be used when applying pressure to attach the die to the silicon adhesive. The attachment should avoid the silicone adhesive being compressed (squished) such that the die, and thus the mirror, is slightly tilted. Because the angle of the mirror is important down to fractions of a degree of the wavelength of laser light, this causes alignment problems. One solution is to apply pressure on the four corners of the chip, avoiding contact and damage to the sensitive mirror structure. In addition, a partial vacuum may be used to assist the placement. Another solution is using a die attach film, which is less likely to cause tilting because it maintains the same thickness.
Other embodiments reduce stress by etching cavities in the bottom of the die substrate to form a pattern of open cavities. The die substrate is patterned with a plurality of cavities open to the die attach layer. The die substrate may be made thicker to provide room for the cavities. The cavities form a pattern that further reduces stress by providing space to absorb some of the transmitted stress due to differences in thermal expansion and contraction. A variety of patterns could be used, such as a perimeter cavity, a simple cross pattern, a cross-hatched pattern, or a circular pattern.
The detailed discussion below, and accompanying figures, will first describe a general Lidar system incorporating embodiments. Next, the mirror structure that operates at a resonant frequency is described. That is followed by detailed descriptions of the novel die attach layer and manufacturing process of embodiments. A dual axis mirror structure is also described. Next are descriptions of the control systems that can react to the change in resonant frequency with temperature, and the computer systems for controlling the systems.
Generally, aspects of the invention are directed to implementations of light steering, which can be used in a number of different applications. For example, a Light Detection and Ranging (LiDAR) module of an autonomous vehicle may incorporate a light steering system. The light steering system can include a transmitter and receiver system to steer emitted incident light in different directions around a vehicle, and to receive reflected light off of objects around the vehicle using a sequential scanning process, which can be used to determine distances between the objects and the vehicle to facilitate autonomous navigation.
Light steering can be implemented by way of micro-mirror assemblies as part of an array, with each micro-mirror assembly having a movable micro-mirror and an actuator (or multiple actuators). The micro-mirrors and actuators can be formed as microelectromechanical systems (MEMS) on a semiconductor substrate, which allows for the integration of the MEMS with other circuitries (e.g., controller, interface circuits, etc.) on the semiconductor substrate, which can allow for simpler, easier, more robust, and cost-effective manufacturing processes.
In a micro-mirror assembly, a micro-mirror can be mechanically connected (e.g., “anchored”) to the semiconductor substrate via a connection structure (e.g., torsion bar, torsion spring, torsion beam, etc.) to form a pivot point and an axis of rotation. As described herein, “mechanically connected,” or “connected,” can include a direct connection or an indirect connection. For example, the micro-mirror can be indirectly connected to the substrate via a connection structure (e.g., torsion bar or torsion spring) to form a pivot/connection point. The micro-mirror can be rotated around the pivot/connection point (“referred to herein as a pivot point”) on the axis of rotation by an actuator. An electrostatic actuator is typically used; however, any suitable type of actuator may be implemented (e.g., piezoelectric, thermal mechanical, etc.), and one of ordinary skill in the art with the benefit of this disclosure would appreciate the many modifications, combinations, variations, and alternative embodiments thereof.
In some embodiments, each micro-mirror can be configured to be rotated by a rotation angle or moved vertically to reflect (and steer) light towards a target direction. For rotation, the connection structure can be deformed to accommodate the rotation, but the connection structure also has a degree of spring stiffness, which varies with the rotation angle and counters the rotation of the micro-mirror to set a target rotation angle. To rotate a micro-mirror by a target rotation angle, an actuator can apply a torque to the micro-mirror based on the rotational moment of inertia of the mirror, as well as the degree of spring stiffness for a given target rotation angle. Different torques can be applied to rotate (e.g., oscillate) the micro-mirror at or near a resonant frequency to achieve different target rotation angles. The actuator can then remove the torque, and the connection structure can return the micro-mirror back to its default orientation for the next rotation. A vertical actuator, such as an electrostatic force actuator, or a thermal actuator with a piston, can be used in embodiments. The rotation or vertical displacement of the micro-mirror can be repeated in the form of an oscillation, typically at or near a resonant frequency of the micro-mirror based on the mass of the micro-mirror and the spring constant of the connection structure.
In certain embodiments, each micro-mirror can be rotated around two orthogonal axes to provide a first range of angles of projection along a vertical dimension and to provide a second range of angles of projection along a horizontal dimension. The first range and the second range of angles of projection can define a two-dimensional field of view (FOV) in which light is projected to detect/scan an object. The FOV can also define a two-dimensional range of directions of incident lights that can be reflected by the object and detected by the receiver. Less commonly, LiDAR systems may also operate over a single axis (e.g., along a horizontal direction). One of ordinary skill in the art with the benefit of this disclosure would appreciate the many implementations and alternative embodiments thereof.
As shown, this structure allows rotation around the axis of the springs/torsion beams 302. In another embodiment not shown in order to not complicate the diagram, additional springs can be provided to give a second, orthogonal axis of rotation of the mirror mass 304 (See
Substrate 406 corresponds to silicon substrate 322 in
As described in the background, stresses can be transferred to the mirror structures due to a mismatch between the coefficient of thermal expansion (CTE) of the package substrate 402 and the chip substrate 406. One method for addressing this is to provide a die attach layer 404 that compensates for the stress. One method for reducing the effect of CTE mismatch due to die attachment is to use a die attach material with a relatively low Young's modulus. The low Young's modulus of a die attach material results in the die attach material itself being deformed more than the chip/device substrate and the package substrate, accommodating and relieving some of the stress due to the different degrees of expansion during a temperature change. In effect, the die attach layer can act as a sort of thermal shock absorber. A thicker die attach layer will improve this stress absorbing characteristic. The die attach layer can be a glue or epoxy in some embodiments, or a film with adhesive in other embodiments. As noted above, however, with standard die attach glues, the chip can tilt during attachment as a result of the die being squished different amounts at different parts of the die. This can result in a misalignment of the mirror structures. Accordingly, some embodiments of the present invention replace such a chip attach glue with a die attach film, which is more rigid and will not squish to the extent of a die attached glue. In order to provide the stress relieving characteristics desired, the die attach film is made thicker than normal.
Additionally, the inventors determined that the stress relief properties of a die attach layer could be further improved by cutting a pattern into the bottom of the chip die. This is another method for reducing the effect of CTE mismatch—providing a pattern of gaps to absorb stress in the bottom surface of a device. Patterns cut into the bottom surface of the device die reduce the lateral stiffness on the bottom, making the bottom easier to deform and take away some of the stress due to the mismatch in thermal expansion of the device and its enclosure. A variety of different patterns of cuts or cavities can be used. A pattern that simply makes the interior hollow is an alternative, but may not be as effective as other patterns. In one embodiment, the depth of the cut is 25-300 microns or 50-150 microns. Alternately, a deeper cut can be used but would require more time or deep etch equipment. The etch depth is limited by the thickness of the Si substrate layer thickness and it typically is 300-800 μm. In one embodiment with a patterned (with cavities) substrate bottom, a die attach material 25-50 μm thick is used for die tilt stability. Etched-in areas would have a thicker die attach material if an epoxy die attach material is used, since the epoxy would at least partially fill the etched cavities.
Relieving the stress limits the effects of temperature change on the resonant frequency and flatness of the MEMS mirrors. The resonant frequency is dependent on the tension of the torsion beam that the mirror rotates around. Ideally, the tension would be relatively constant with temperature. However, the silicon CTE is low compared to typical package materials, such as ceramics. Thus, the materials expand or contract different amounts with temperature change, affecting the torsion beam tension and thus the resonant frequency. A thicker die attach material helps. The die attach layer can deform when there is a mismatch between the CTE of the silicon substrate of the die, and the CTE of the package material. The attached material expands more than silicon if the temperature goes up, and due to material's softness silicon does not expand as much, resulting in less pressure on the torsion beams. In addition, making the silicon substrate itself thicker can help by providing more space for stress to dissipate, and can also provide room for etching cavities.
A die attach film with a low Young's modulus is chosen to provide the desired stress release. In particular, a film with a Young's modulus that is less than 15,000 psi is used. Alternately, a Young's modulus between 1,000 psi and 15,000 psi is chosen.
Die attach layer 404, in one embodiment, is between 25-150 μm thick (1 mil [thousandth of an inch]-6 mil). Alternately, it could be 15-175 μm thick. More particularly, it is between 50-125 μm thick. In one embodiment, it is 100 μm thick+−10%.
The patterns comprise cavities etched in the bottom of the die substrate. In one embodiment, the cavities are 100 μm wide and 100 μm deep. Alternately, the cavities are 50-150 μm wide and 25-300 μm deep. The cavity width could vary by a lot. For instance, the width of cavities in
DOE 0 is a standard die and die attach layer/film without anything more added.
DOE 1 simply adds another 100 μm in thickness to the 500 m all Si die substrate layer 406 of
DOE 2 simply uses a thicker die attach layer, with a 100 μm thick die attach material. This layer, shown as die attach layer 404 in
DOE 3 corresponds to pattern 505 shown in
DOE 4 provides a 100 μm thick die attach layer, leaving a perimeter gap 502 as shown in
The
The
The
The simulation thus shows that the resonant frequency change due to temperature variation can be reduced by (1) a softer die attach material (lower Young's modulus) or (2) a thicker die attach material (DOE 2), or a cross pattern in the substrate (DOE 3). Also, with certain patterned Si layers (DOE 3), the resonant frequency change due to temperature variation can be reduced to a similar degree as using a thicker die attach layer. The use of a patterned silicon layer provides (1) better optical tilt control during assembly than with a thicker die attach layer, and (2) better optical tilt stability by using a thinner die attach layer between the die bottom surface and the package surface.
In one embodiment, the die attach layer is a film composed of an adhesive layer for fixing a semiconductor chip in a dicing step, and an adhesive layer attached to a back surface of the chip in a die bonding step and attached to a wiring board such as a lead frame. The packaging process using such a die-attach film includes a dicing step of cutting the wafer into chips or dies. A die bonding step attaches the separated chips to a mounting plate of a circuit film or lead frame. A wire bonding step connects a circuit pattern such as a chip pad and a lead frame provided on the semiconductor chip with an electrical connection means such as a wire. In order to protect the internal circuit and other components of the semiconductor chip, a molding process for enclosing the outside with an encapsulate is included.
In one example, the thickness ratio (adhesive/film) may be in the range of 0.15 to 0.5. Examples of plastic films include an olefin-based film such as polypropylene film or polyethylene film; Polyester-based films such as polyethylene terephthalate film; Polycarbonate film; Polyvinyl chloride film; Polytetrafluoroethylene film; Polybutene film; Polybutadiene film; Vinyl chloride copolymer film; Ethylene-vinyl acetate copolymer films; Ethylene-propylene copolymer film; or a mixture of one or more kinds of ethylene-ethyl acrylate copolymer films and the like, but is not limited thereto. The heterogeneous mixing of the base films in the above means a film which the base film is composed of two or more laminated films of the above-described films or is prepared from the copolymer of the above-mentioned resins. Such base films may also be subjected to conventional physical or chemical treatments, such as matt treatment, corona discharge treatment, primer treatment or crosslinking treatment, as necessary.
In addition, when the ultraviolet curable pressure-sensitive adhesive is used in the adhesive portion, the base film is preferably excellent in the ultraviolet transmittance, for example, may have a UV transmittance of 70% or more, preferably 90% or more. Adhesive examples include an ultraviolet curable adhesive or a thermosetting adhesive. When using an ultraviolet curable adhesive, adhesive force is reduced by irradiating an ultraviolet-ray from the base film side. In the case of a thermosetting adhesive, appropriate heat is applied and an adhesive force is reduced.
Step 706 is providing a chip package having a chip package substrate. Step 708 is attaching a die attach layer between the die substrate and the chip package substrate. Step 710 is adhesively bonding the die attach layer to both the die substrate and the chip package substrate. A die attach layer with a Young's modulus less than 15,000 psi is chosen. The die attach layer may be a film that is attached before the chip is diced into individual die.
In sum, embodiments of the present invention provide a micro-electromechanical system (MEMS) apparatus for beam steering in a Light Detection and Ranging (LiDAR) system 102 of an autonomous vehicle 100. A mirror mass 304 has a reflective surface and at least first and second respective sides. First and second supporting torsion springs 302 are formed, wherein the first and second supporting torsion springs have first ends, respectively, connected to the first and second respective sides of the mirror mass, on opposite sides, to support the mirror mass. First and second common terminals 310 are connected to the first and second supporting torsion springs, respectively, on second ends of the first and second supporting torsion springs. A plurality of first fingers 306 extend from the mirror mass on first and second sides orthogonal to the first and second supporting torsion springs. First and second bias terminals 314 are opposite the first and second sides of the mirror mass. A plurality of second fingers 312 extend from the first and second bias terminals. The plurality of second fingers are interleaved with the plurality of first fingers and partially overlap the plurality of first fingers. An oxide layer is below the first and second common terminals and the first and second bias terminals. A die substrate 322 is below the oxide layer. A chip package has a chip package substrate 402. A die attach layer 404 is between the die substrate and the chip package substrate, and is adhesively bonded to both the die substrate and the chip package substrate. The die attach layer has a Young's modulus less than 15,000 psi. The die substrate is patterned with a plurality of cavities (502, 504, 508, 512, 516) open to the die attach layer.
The die attach system and method described herein compensates for changes in resonant frequency with temperature, which allows systems with a narrow range of control to be used. Those control systems, into which the present invention is integrated, will now be described.
Transmit block 1010, as described above, can incorporate a number of systems that facilitate that generation and emission of a light signal, including dispersion patterns (e.g., 360 degree planar detection), pulse shaping and frequency control, Time-Of-Flight (TOF) measurements, and any other control systems to enable the LiDAR system to emit pulses in the manner described above. In the simplified representation of
In certain embodiments, processor(s) 1020 may include one or more microprocessors (μCs) and can be configured to control the operation of system 1000. Alternatively or additionally, processor 1020 may include one or more microcontrollers (MCUs), digital signal processors (DSPs), or the like, with supporting hardware, firmware (e.g., memory, programmable I/Os, etc.), and/or software, as would be appreciated by one of ordinary skill in the art. Alternatively, MCUs, μCs, DSPs, ASIC, programmable logic device, and the like, may be configured in other system blocks of system 1000. For example, control system block 1040 may include a local processor to certain control parameters (e.g., operation of the emitter). Processor(s) 1020 may control some or all aspects of transmit block 1010 (e.g., optics/emitter 1032, control system 1040, dual sided mirror 220 position as shown in
Light signal generator 1030 may include circuitry (e.g., a laser diode) configured to generate a light signal, which can be used as the LiDAR send signal, according to certain embodiments. In some cases, light signal generator 1030 may generate a laser that is used to generate a continuous or pulsed laser beam at any suitable electromagnetic wavelengths spanning the visible light spectrum and non-visible light spectrum (e.g., ultraviolet and infra-red). In some embodiments, lasers are commonly in the range of 600-1200 nm, although other wavelengths are possible, as would be appreciated by one of ordinary skill in the art.
Optics/Emitter block 1032 (also referred to as transmitter 1032) may include one or more arrays of mirrors (including but not limited to dual sided mirror 220 as described above in
Power block 1015 can be configured to generate power for transmit block 1010, receive block 1050, as well as manage power distribution, charging, power efficiency, and the like. In some embodiments, power management block 1015 can include a battery (not shown), and a power grid within system 1000 to provide power to each subsystem (e.g., control system 1040, etc.). The functions provided by power management block 1015 may be subsumed by other elements within transmit block 1010, or may provide power to any system in LiDAR system 1000. Alternatively, some embodiments may not include a dedicated power block and power may be supplied by a number of individual sources that may be independent of one another.
Control system 1040 may control aspects of light signal generation (e.g., pulse shaping), optics/emitter control, TOF timing, or any other function described herein. In some cases, aspects of control system 1040 may be subsumed by processor(s) 1020, light signal generator 1030, or any block within transmit block 1010, or LiDAR system 1000 in general.
Receive block 1050 may include circuitry configured to detect a process a return light pulse to determine a distance of an object, and in some cases determine the dimensions of the object, the velocity and/or acceleration of the object, and the like. Processor(s) 1065 may be configured to perform operations such as processing received return pulses from detectors(s) 1060, controlling the operation of TOF module 1034, controlling threshold control module 1080, or any other aspect of the functions of receive block 1050 or LiDAR system 1000 in general.
TOF module 1034 may include a counter for measuring the time-of-flight of a round trip for a send and return signal. In some cases, TOF module 1034 may be subsumed by other modules in LiDAR system 1000, such as control system 1040, optics/emitter 1032, or other entity. TOF modules 1034 may implement return “windows” that limit a time that LiDAR system 1000 looks for a particular pulse to be returned. For example, a return window may be limited to a maximum amount of time it would take a pulse to return from a maximum range location (e.g., 250 m). Some embodiments may incorporate a buffer time (e.g., maximum time plus 10%). TOF module 1034 may operate independently or may be controlled by other system block, such as processor(s) 1020, as described above. In some embodiments, transmit block may also include a TOF detection module. One of ordinary skill in the art with the benefit of this disclosure would appreciate the many modification, variations, and alternative ways of implementing the TOF detection block in system 1000.
Detector(s) 1060 may detect incoming return signals that have reflected off of one or more objects. In some cases, LiDAR system 1000 may employ spectral filtering based on wavelength, polarization, and/or range to help reduce interference, filter unwanted frequencies, or other deleterious signals that may be detected. Typically, detector(s) 1060 can detect an intensity of light and records data about the return signal (e.g., via coherent detection, photon counting, analog signal detection, or the like). Detector (s) 1060 can use any suitable photodetector technology including solid state photodetectors (e.g., silicon avalanche photodiodes, complimentary metal-oxide semiconductors (CMOS), charge-coupled devices (CCD), hybrid CMOS/CCD devices) or photomultipliers. In some cases, a single receiver may be used or multiple receivers may be configured to operate in parallel.
Gain sensitivity model 1070 may include systems and/or algorithms for determining a gain sensitivity profile that can be adapted to a particular object detection threshold. The gain sensitivity profile can be modified based on a distance (range value) of a detected object (e.g., based on TOF measurements). In some cases, the gain profile may cause an object detection threshold to change at a rate that is inversely proportional with respect to a magnitude of the object range value. A gain sensitivity profile may be generated by hardware/software/firmware, or gain sensor model 1070 may employ one or more look up tables (e.g., stored in a local or remote database) that can associate a gain value with a particular detected distance or associate an appropriate mathematical relationship there between (e.g., apply a particular gain at a detected object distance that is 10% of a maximum range of the LiDAR system, apply a different gain at 15% of the maximum range, etc.). In some cases, a Lambertian model may be used to apply a gain sensitivity profile to an object detection threshold. The Lambertian model typically represents perfectly diffuse (matte) surfaces by a constant bidirectional reflectance distribution function (BRDF), which provides reliable results in LiDAR system as described herein. However, any suitable gain sensitivity profile can be used including, but not limited to, Oren-Nayar model, Nanrahan-Krueger, Cook-Torrence, Diffuse BRDF, Limmel-Seeliger, Blinn-Phong, Ward model, HTSG model, Fitted Lafortune Model, or the like. One of ordinary skill in the art with the benefit of this disclosure would understand the many alternatives, modifications, and applications thereof.
Threshold control block 1080 may set an object detection threshold for LiDAR system 1000. For example, threshold control block 1080 may set an object detection threshold over a certain a full range of detection for LiDAR system 1000. The object detection threshold may be determined based on a number of factors including, but not limited to, noise data (e.g., detected by one or more microphones) corresponding to an ambient noise level, and false positive data (typically a constant value) corresponding to a rate of false positive object detection occurrences for the LiDAR system. In some embodiments, the object detection threshold may be applied to the maximum range (furthest detectable distance) with the object detection threshold for distances ranging from the minimum detection range up to the maximum range being modified by a gain sensitivity model (e.g., Lambertian model).
Although certain systems may not expressly discussed, they should be considered as part of system 1000, as would be understood by one of ordinary skill in the art. For example, system 1000 may include a bus system (e.g., CAMBUS) to transfer power and/or data to and from the different systems therein. In some embodiments, system 1000 may include a storage subsystem (not shown). A storage subsystem can store one or more software programs to be executed by processors (e.g., in processor(s) 1020). It should be understood that “software” can refer to sequences of instructions that, when executed by processing unit(s) (e.g., processors, processing devices, etc.), cause system 1000 to perform certain operations of software programs. The instructions can be stored as firmware residing in read only memory (ROM) and/or applications stored in media storage that can be read into memory for processing by processing devices. Software can be implemented as a single program or a collection of separate programs and can be stored in non-volatile storage and copied in whole or in-part to volatile working memory during program execution. From a storage subsystem, processing devices can retrieve program instructions to execute in order to execute various operations (e.g., software-controlled spring auto-adjustment, etc.) as described herein. Some software controlled aspects of LiDAR system 1000 may include aspects of gain sensitivity model 1070, threshold control 1080, control system 1040, TOF module 1034, or any other aspect of LiDAR system 1000.
It should be appreciated that system 1000 is meant to be illustrative and that many variations and modifications are possible, as would be appreciated by one of ordinary skill in the art. System 1000 can include other functions or capabilities that are not specifically described here. For example, LiDAR system 1000 may include a communications block (not shown) configured to enable communication between LiDAR system 1000 and other systems of the vehicle or remote resource (e.g., remote servers), etc., according to certain embodiments. In such cases, the communications block can be configured to provide wireless connectivity in any suitable communication protocol (e.g., radio-frequency (RF), Bluetooth, BLE, infra-red (IR), ZigBee, Z-Wave, Wi-Fi, or a combination thereof).
While system 1000 is described with reference to particular blocks (e.g., threshold control block 1080), it is to be understood that these blocks are defined for understanding certain embodiments of the invention and is not intended to imply that embodiments are limited to a particular physical arrangement of component parts. The individual blocks need not correspond to physically distinct components. Blocks can be configured to perform various operations, e.g., by programming a processor or providing appropriate processes, and various blocks may or may not be reconfigurable depending on how the initial configuration is obtained. Certain embodiments can be realized in a variety of apparatuses including electronic devices implemented using any combination of circuitry and software. Furthermore, aspects and/or portions of system 1000 may be combined with or operated by other sub-systems as informed by design. For example, power management block 1015 and/or threshold control block 1080 may be integrated with processor(s) 1020 instead of functioning as separate entities.
In some examples, internal bus subsystem 1104 (e.g., CAMBUS) can provide a mechanism for letting the various components and subsystems of computer system 1100 communicate with each other as intended. Although internal bus subsystem 1104 is shown schematically as a single bus, alternative embodiments of the bus subsystem can utilize multiple buses. Additionally, network interface subsystem 1112 can serve as an interface for communicating data between computer system 1100 and other computer systems or networks. Embodiments of network interface subsystem 1112 can include wired interfaces (e.g., Ethernet, CAN, RS232, RS485, etc.) or wireless interfaces (e.g., ZigBee, Wi-Fi, cellular, etc.).
In some cases, user interface input devices 1114 can include a keyboard, pointing devices (e.g., mouse, trackball, touchpad, etc.), a barcode scanner, a touch-screen incorporated into a display, audio input devices (e.g., voice recognition systems, microphones, etc.), Human Machine Interfaces (HMI) and other types of input devices. In general, use of the term “input device” is intended to include all possible types of devices and mechanisms for inputting information into computer system 1100. Additionally, user interface output devices 1116 can include a display subsystem, a printer, or non-visual displays such as audio output devices, etc. The display subsystem can be any known type of display device. In general, use of the term “output device” is intended to include all possible types of devices and mechanisms for outputting information from computer system 1100.
Storage subsystem 1106 can include memory subsystem 1108 and file/disk storage subsystem 1110. Subsystems 1108 and 1110 represent non-transitory computer-readable storage media that can store program code and/or data that provide the functionality of embodiments of the present disclosure. In some embodiments, memory subsystem 1108 can include a number of memories including main random access memory (RAM) 1118 for storage of instructions and data during program execution and read-only memory (ROM) 1120 in which fixed instructions may be stored. File storage subsystem 1110 can provide persistent (i.e., non-volatile) storage for program and data files, and can include a magnetic or solid-state hard disk drive, an optical drive along with associated removable media (e.g., CD-ROM, DVD, Blu-Ray, etc.), a removable flash memory-based drive or card, and/or other types of storage media known in the art.
It should be appreciated that computer system 1100 is illustrative and not intended to limit embodiments of the present disclosure. Many other configurations having more or fewer components than system 1100 are possible.
The various embodiments further can be implemented in a wide variety of operating environments, which in some cases can include one or more user computers, computing devices or processing devices, which can be used to operate any of a number of applications. User or client devices can include any of a number of general purpose personal computers, such as desktop or laptop computers running a standard operating system, as well as cellular, wireless and handheld devices running mobile software and capable of supporting a number of networking and messaging protocols. Such a system also can include a number of workstations running any of a variety of commercially available operating systems and other known applications for purposes such as development and database management. These devices also can include other electronic devices, such as dummy terminals, thin-clients, gaming systems and other devices capable of communicating via a network.
Most embodiments utilize at least one network that would be familiar to those skilled in the art for supporting communications using any of a variety of commercially available protocols, such as TCP/IP, UDP, OSI, FTP, UPnP, NFS, CIFS, and the like. The network can be, for example, a local-area network, a wide-area network, a virtual private network, the Internet, an intranet, an extranet, a public switched telephone network, an infrared network, a wireless network, and any combination thereof.
In embodiments utilizing a network server, the network server can run any of a variety of server or mid-tier applications, including HTTP servers, FTP servers, CGI servers, data servers, Java servers, and business application servers. The server(s) also may be capable of executing programs or scripts in response to requests from user devices, such as by executing one or more applications that may be implemented as one or more scripts or programs written in any programming language, including but not limited to Java®, C, C # or C++, or any scripting language, such as Perl, Python or TCL, as well as combinations thereof. The server(s) may also include database servers, including without limitation those commercially available from Oracle®, Microsoft®, Sybase®, and IBM®.
The environment can include a variety of data stores and other memory and storage media as discussed above. These can reside in a variety of locations, such as on a storage medium local to (and/or resident in) one or more of the computers or remote from any or all of the computers across the network. In a particular set of embodiments, the information may reside in a storage-area network (SAN) familiar to those skilled in the art. Similarly, any necessary files for performing the functions attributed to the computers, servers or other network devices may be stored locally and/or remotely, as appropriate. Where a system includes computerized devices, each such device can include hardware elements that may be electrically coupled via a bus, the elements including, for example, at least one central processing unit (CPU), at least one input device (e.g., a mouse, keyboard, controller, touch screen or keypad), and at least one output device (e.g., a display device, printer or speaker). Such a system may also include one or more storage devices, such as disk drives, optical storage devices, and solid-state storage devices such as RAM or ROM, as well as removable media devices, memory cards, flash cards, etc.
Such devices also can include a computer-readable storage media reader, a communications device (e.g., a modem, a network card (wireless or wired), an infrared communication device, etc.), and working memory as described above. The computer-readable storage media reader can be connected with, or configured to receive, a non-transitory computer readable storage medium, representing remote, local, fixed, and/or removable storage devices as well as storage media for temporarily and/or more permanently containing, storing, transmitting, and retrieving computer-readable information. The system and various devices also typically will include a number of software applications, modules, services or other elements located within at least one working memory device, including an operating system and application programs, such as a client application or browser. It should be appreciated that alternate embodiments may have numerous variations from that described above. For example, customized hardware might also be used and/or particular elements might be implemented in hardware, software (including portable software, such as applets) or both. Further, connection to other computing devices such as network input/output devices may be employed.
Non-transitory storage media and computer-readable storage media for containing code, or portions of code, can include any appropriate media known or used in the art such as, but not limited to, volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data, including RAM, ROM, Electrically Erasable Programmable Read-Only Memory (EEPROM), flash memory or other memory technology, CD-ROM, DVD or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other medium which can be used to store the desired information and which can be accessed by a system device. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will appreciate other ways and/or methods to implement the various embodiments. However, computer-readable storage media does not include transitory media such as carrier waves or the like.
Other variations are within the spirit of the present disclosure. Thus, while the disclosed techniques are susceptible to various modifications and alternative constructions, certain illustrated examples thereof are shown in the drawings and have been described above in detail. It should be understood, however, that there is no intention to limit the disclosure to the specific form or forms disclosed, but on the contrary, the intention is to cover all modifications, alternative constructions and equivalents falling within the spirit and scope of the disclosure, as defined in the appended claims. For instance, any of the examples, alternative examples, etc., and the concepts thereof may be applied to any other examples described and/or within the spirit and scope of the disclosure.
The use of the terms “a” and “an” and “the” and similar referents in the context of describing the disclosed examples (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms “comprising,” “having,” “including,” and “containing” are to be construed as open-ended terms (i.e., meaning “including, but not limited to,”) unless otherwise noted. The term “connected” is to be construed as partly or wholly contained within, attached to, or joined together, even if there is something intervening. The phrase “based on” should be understood to be open-ended, and not limiting in any way, and is intended to be interpreted or otherwise read as “based at least in part on,” where appropriate. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. All methods described herein can be performed in any suitable order unless otherwise indicated herein or otherwise clearly contradicted by context. The use of any and all examples, or exemplary language (e.g., “such as”) provided herein, is intended merely to better illuminate examples of the disclosure and does not pose a limitation on the scope of the disclosure unless otherwise claimed. No language in the specification should be construed as indicating any non-claimed element as essential to the practice of the disclosure.
