Patent Grant
11841464
Unless otherwise indicated herein, the materials described in this section are not prior art to the claims in this application and are not admitted to be prior art by inclusion in this section.
Vehicles can be configured to operate in an autonomous mode in which the vehicle navigates through an environment with some, little, or no input from a driver. Such autonomous or semi-autonomous vehicles can include one or more sensors that are configured to detect information about the environment in which the vehicle operates.
One such sensor is a light detection and ranging (LIDAR) device. A LIDAR can estimate distance to environmental features while scanning through a scene to assemble a “point cloud” indicative of reflective surfaces in the environment. Individual points in the point cloud can be determined by transmitting a laser pulse and detecting a returning pulse, if any, reflected from an object in the environment, and determining the distance to the object according to the time delay between the transmitted pulse and the reception of the reflected pulse. A laser, or set of lasers, can be rapidly and repeatedly scanned across a scene to provide continuous real-time information on distances to reflective objects in the scene. Combining the measured distances and the orientation of the laser(s) while measuring each distance allows for associating a three-dimensional position with each returning pulse. In this way, a three-dimensional map of points indicative of locations of reflective features in the environment can be generated for the entire scanning zone.
The present disclosure generally relates to light detection and ranging (LIDAR) systems, which may be configured to obtain information about an environment. Such LIDAR devices may be implemented in vehicles, such as autonomous and semi-autonomous automobiles, trucks, motorcycles, and other types of vehicles that can move within their respective environments.
In a first aspect, a system is provided. The system includes a plurality of light-emitter devices. The plurality of light-emitter devices is operable to emit light along a plurality of emission vectors such that the emitted light interacts with an external environment of the system. The system also includes a receiver subsystem configured to provide information indicative of interactions between the emitted light and the external environment. The system additionally includes a controller operable to carry out operations. The operations include determining, for at least one light-emitter device of the plurality of light-emitter devices, a light pulse schedule. The light pulse schedule is based on a respective emission vector of the at least one light-emitter device and a three-dimensional map of the external environment. The light pulse schedule includes at least one light pulse parameter and a listening window duration. The system also includes causing the at least one light-emitter device of the plurality of light-emitter devices to emit a light pulse according to the light pulse schedule.
In a second aspect, a method is provided. The method includes determining, for at least one light-emitter device of a plurality of light-emitter devices, a light pulse schedule. The plurality of light-emitter devices is operable to emit light along a plurality of emission vectors. The light pulse schedule is based on a respective emission vector of the at least one light-emitter device and a three-dimensional map of an external environment. The light pulse schedule includes at least one light pulse parameter and a listening window duration. The method also includes causing the at least one light-emitter device of the plurality of light-emitter devices to emit a light pulse according to the light pulse schedule. The light pulse interacts with an external environment.
In a third aspect, a system is provided. The system includes a plurality of light-emitter devices. The plurality of light-emitter devices is operable to emit light along a plurality of emission vectors such that the emitted light interacts with an external environment of the system. The system also includes a receiver subsystem configured to provide information indicative of interactions between the emitted light and the external environment. The system further includes a controller operable to carry out operations. The operations include determining, for at least one light-emitter device of the plurality of light-emitter devices, a light pulse schedule. The light pulse schedule is based on a respective emission vector of the at least one light-emitter device and a three-dimensional map of the external environment. The determined light pulse schedule includes at least one light pulse parameter and a listening window duration. The operations also include causing the at least one light-emitter device of the plurality of light-emitter devices to emit a first light pulse according to the determined light pulse schedule and a second light pulse according to a default light pulse schedule.
Other aspects, embodiments, and implementations will become apparent to those of ordinary skill in the art by reading the following detailed description, with reference where appropriate to the accompanying drawings.
Example methods, devices, and systems are described herein. It should be understood that the words “example” and “exemplary” are used herein to mean “serving as an example, instance, or illustration.” Any embodiment or feature described herein as being an “example” or “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments or features. Other embodiments can be utilized, and other changes can be made, without departing from the scope of the subject matter presented herein.
Thus, the example embodiments described herein are not meant to be limiting. Aspects of the present disclosure, as generally described herein, and illustrated in the figures, can be arranged, substituted, combined, separated, and designed in a wide variety of different configurations, all of which are contemplated herein.
Further, unless context suggests otherwise, the features illustrated in each of the figures may be used in combination with one another. Thus, the figures should be generally viewed as component aspects of one or more overall embodiments, with the understanding that not all illustrated features are necessary for each embodiment.
A LIDAR system may include a transmit assembly that includes a plurality of light emitters. In some embodiments, the plurality of light emitters may be arranged along a substrate having a principal plane. Furthermore, each light emitter may be arranged as having a different emission angle such that the plurality of light emitters may be configured to emit light at various emission angles (e.g., within an angular range between +2 degrees above the horizon and −18 degrees below the horizon assuming a 2 meter LIDAR height and a flat earth and vehicle pose condition) along the principal plane. The LIDAR system may be configured to rotate about a yaw axis such that the transmit assembly and the plurality of light emitters illuminate the environment. In some embodiments, the LIDAR system may rotate at 10 Hz or 20 Hz. Other rotational rates are possible and contemplated herein. In some embodiments, the LIDAR system may provide point cloud information for an autonomous or semi-autonomous vehicle (e.g., a self-driving car, truck, boat, or aerial vehicle).
In some embodiments, the respective light emitters of the plurality of light emitters may emit light pulses in sequential order (e.g., starting by firing a light emitter with a highest emission angle and ending by firing a light emitter with a lowest emission angle or vice versa). After firing each light pulse, there may be a pause (e.g., a “listening” period) during which time the light pulse may travel from the light emitter, interact with an object in the environment (e.g., scatter or reflect), and a portion of the light pulse may return to the LIDAR device and be received by photodetectors of a receive assembly. In some cases, the listening period or listening window for a light pulse to travel 150 meters and back (a total distance of 300 meters) may be approximately 1 microsecond.
To improve resolution in the yaw angle, the listening window may be reduced for light pulses that are emitted by downward-pointing light emitters. That is, if a light pulse from a downward-pointing light emitter is likely to interact with a ground surface within, for example, 20 meters, the listening window may be reduced to approximately 130 nanoseconds, based on a round trip time of the emitted light pulse. In other embodiments, the range of listening windows may be adjustable between 100 nanoseconds and 2 microseconds; however, other listening windows are possible. The listening window corresponding to each light pulse may be adjusted based on the likely range to the ground surface in the environment. Accordingly, the overall time to fire each of the plurality of light emitters is reduced and the LIDAR device may be able to start a new vertical scan while rotating over a smaller yaw angle. In other words, by reducing the overall cycle time, at least some light emitters may be configured to fire more frequently and, in some embodiments, finer yaw resolution may be provided by the LIDAR system.
Systems and methods described herein include dynamic adjustment of the listening windows based on the emission angle of each light pulse and a maximum predicted distance for each yaw angle. That is, as the LIDAR device spins about the yaw axis, listening windows may be adjusted based on how far the light pulses are anticipated to travel before they interact with the environment (e.g., the ground). The maximum predicted distance may be based on the pose of the LIDAR system and/or the vehicle associated with the LIDAR system (e.g., the pose of the vehicle) and/or may be based on elevation data, which may be obtained by mapping data or sampling data. In an example embodiment, the sampling data may include obtaining the elevation data by performing a 360-degree scan of the LIDAR system while pulsing the emitter devices with a predefined listening window (e.g., 2 microseconds). In such a scenario, the LIDAR system may obtain information that includes the distance to the ground as a function of yaw angle. The LIDAR system may then use the information to adjust listening windows in subsequent scans.
In some embodiments, a 360-degree scan of the LIDAR system may include a mixture of long listening windows with normal/reduced time listening windows on a fine time scale. That is, the LIDAR system may be configured to interleave, alternate, or otherwise intermingle long listening times with shortened listening windows. Other ways to vary listening windows associated with emitted light pulses from a LIDAR, specifically taking into account predicted distances to objects in the environment, are contemplated herein.
Additionally or alternatively, the power of each light pulse may be adjusted based on the maximum predicted distance for each combination of yaw angle and beam elevation within a given field of view of the LIDAR. For example, if the sampling data indicate a maximum predicted distance for a given light pulse is 10 meters, the amount of power provided for the light pulse may be decreased by 80% or more compared to light pulses predicted to travel 200 meters. That is, the power of a given light pulse may relate to a maximum reliable detection range. In an effort to reduce power usage by the LIDAR system while reliably detecting objects in the environment, the light pulse power may be adjusted based on the maximum predicted distance that the light pulse may travel. As such, the LIDAR system may more efficiently transmit light pulses into its environment and reduce issues associated with retro-reflection and blooming, which may be caused by receiving too many photons from close-range targets/objects.
The emitted light beams 52 and focused light 58 may traverse the shared space 40 also included in the housing 10. In some embodiments, the emitted light beams 52 propagate along a transmit path through the shared space 40 and the focused light 58 propagates along a receive path through the shared space 40.
The sensing system 10 can determine an aspect of the one or more objects (e.g., location, shape, etc.) in the environment of the sensing system 10 by processing the focused light 58 received by the receive block 30. For example, the sensing system 10 can compare a time when pulses included in the emitted light beams 52 were emitted by the transmit block 20 with a time when corresponding pulses included in the focused light 58 were received by the receive block 30 and determine the distance between the one or more objects and the sensing system 10 based on the comparison.
The housing 12 included in the sensing system 10 can provide a platform for mounting the various components included in the sensing system 10. The housing 12 can be formed from any material capable of supporting the various components of the sensing system 10 included in an interior space of the housing 12. For example, the housing 12 may be formed from a structural material such as plastic or metal.
In some examples, the housing 12 may include optical shielding configured to reduce ambient light and/or unintentional transmission of the emitted light beams 52 from the transmit block 20 to the receive block 30. The optical shielding can be provided by forming and/or coating the outer surface of the housing 12 with a material that blocks the ambient light from the environment. Additionally, inner surfaces of the housing 12 can include and/or be coated with the material described above to optically isolate the transmit block 20 from the receive block 30 to prevent the receive block 30 from receiving the emitted light beams 52 before the emitted light beams 52 reach the lens 50.
In some examples, the housing 12 can be configured for electromagnetic shielding to reduce electromagnetic noise (e.g., Radio Frequency (RF) Noise, etc.) from ambient environment of the sensor system 10 and/or electromagnetic noise between the transmit block 20 and the receive block 30. Electromagnetic shielding can improve quality of the emitted light beams 52 emitted by the transmit block 20 and reduce noise in signals received and/or provided by the receive block 30. Electromagnetic shielding can be achieved by forming and/or coating the housing 12 with one or more materials such as a metal, metallic ink, metallic foam, carbon foam, or any other material configured to appropriately absorb or reflect electromagnetic radiation. Metals that can be used for the electromagnetic shielding can include for example, copper or nickel.
In some examples, the housing 12 can be configured to have a substantially cylindrical shape and to rotate about an axis of the sensing system 10. For example, the housing 12 can have the substantially cylindrical shape with a diameter of approximately 10 centimeters. In some examples, the axis is substantially vertical. By rotating the housing 12 that includes the various components, in some examples, a three-dimensional map of a 360 degree view of the environment of the sensing system 10 can be determined without frequent recalibration of the arrangement of the various components of the sensing system 10. Additionally or alternatively, the sensing system 10 can be configured to tilt the axis of rotation of the housing 12 to control the field of view of the sensing system 10.
Although not illustrated in
In some examples, the various components of the sensing system 10 such as the transmit block 20, receive block 30, and the lens 50 can be removably mounted to the housing 12 in predetermined positions to reduce burden of calibrating the arrangement of each component and/or subcomponents included in each component. Thus, the housing 12 acts as the platform for the various components of the sensing system 10 to provide ease of assembly, maintenance, calibration, and manufacture of the sensing system 10.
The transmit block 20 includes a plurality of light sources 22 that can be configured to emit the plurality of emitted light beams 52 via an exit aperture 26. In some examples, each of the plurality of emitted light beams 52 corresponds to one of the plurality of light sources 22. The transmit block 20 can optionally include a mirror 24 along the transmit path of the emitted light beams 52 between the light sources 22 and the exit aperture 26.
The light sources 22 can include laser diodes, light emitting diodes (LED), vertical cavity surface emitting lasers (VCSEL), organic light emitting diodes (OLED), polymer light emitting diodes (PLED), light emitting polymers (LEP), liquid crystal displays (LCD), microelectromechanical systems (MEMS), or any other device configured to selectively transmit, reflect, and/or emit light to provide the plurality of emitted light beams 52. In some examples, the light sources 22 can be configured to emit the emitted light beams 52 in a wavelength range that can be detected by detectors 32 included in the receive block 30. The wavelength range could, for example, be in the ultraviolet, visible, and/or infrared portions of the electromagnetic spectrum. In some examples, the wavelength range can be a narrow wavelength range, such as provided by lasers. In one example, the wavelength range includes wavelengths that are approximately 905 nm. Additionally, the light sources 22 can be configured to emit the emitted light beams 52 in the form of pulses. In some examples, the plurality of light sources 22 can be disposed on one or more substrates (e.g., printed circuit boards (PCB), flexible PCBs, etc.) and arranged to emit the plurality of light beams 52 towards the exit aperture 26.
In some examples, the plurality of light sources 22 can be configured to emit uncollimated light beams included in the emitted light beams 52. For example, the emitted light beams 52 can diverge in one or more directions along the transmit path due to the uncollimated light beams emitted by the plurality of light sources 22. In some examples, vertical and horizontal extents of the emitted light beams 52 at any position along the transmit path can be based on an extent of the divergence of the uncollimated light beams emitted by the plurality of light sources 22.
The exit aperture 26 arranged along the transmit path of the emitted light beams 52 can be configured to accommodate the vertical and horizontal extents of the plurality of light beams 52 emitted by the plurality of light sources 22 at the exit aperture 26. It is noted that the block diagram shown in
In some examples of the sensing system 10, it may be desirable to minimize size of the exit aperture 26 while accommodating the vertical and horizontal extents of the plurality of light beams 52. For example, minimizing the size of the exit aperture 26 can improve the optical shielding of the light sources 22 described above in the functions of the housing 12. Additionally or alternatively, the wall separating the transmit block 20 and the shared space 40 can be arranged along the receive path of the focused light 58, and thus, the exit aperture 26 can be minimized to allow a larger portion of the focused light 58 to reach the wall. For example, the wall can be coated with a reflective material (e.g., reflective surface 42 in shared space 40) and the receive path can include reflecting the focused light 58 by the reflective material towards the receive block 30. In this case, minimizing the size of the exit aperture 26 can allow a larger portion of the focused light 58 to reflect off the reflective material with which the wall is coated.
To minimize the size of the exit aperture 26, in some examples, the divergence of the emitted light beams 52 can be reduced by partially collimating the uncollimated light beams emitted by the light sources 22 to minimize the vertical and horizontal extents of the emitted light beams 52 and thus minimize the size of the exit aperture 26. For example, each light source of the plurality of light sources 22 can include a cylindrical lens arranged adjacent to the light source. The light source may emit a corresponding uncollimated light beam that diverges more in a first direction than in a second direction. The cylindrical lens may pre-collimate the uncollimated light beam in the first direction to provide a partially collimated light beam, thereby reducing the divergence in the first direction. In some examples, the partially collimated light beam diverges less in the first direction than in the second direction. Similarly, uncollimated light beams from other light sources of the plurality of light sources 22 can have a reduced beam width in the first direction and thus the emitted light beams 52 can have a smaller divergence due to the partially collimated light beams. In this example, at least one of the vertical and horizontal extents of the exit aperture 26 can be reduced due to partially collimating the light beams 52.
Additionally or alternatively, to minimize the size of the exit aperture 26, in some examples, the light sources 22 can be arranged along a shaped surface defined by the transmit block 20. In some examples, the shaped surface may be faceted and/or substantially curved. The faceted and/or curved surface can be configured such that the emitted light beams 52 converge towards the exit aperture 26, and thus the vertical and horizontal extents of the emitted light beams 52 at the exit aperture 26 can be reduced due to the arrangement of the light sources 22 along the faceted and/or curved surface of the transmit block 20.
In some examples, a curved surface of the transmit block 20 can include a curvature along the first direction of divergence of the emitted light beams 52 and a curvature along the second direction of divergence of the emitted light beams 52, such that the plurality of light beams 52 converge towards a central area in front of the plurality of light sources 22 along the transmit path.
To facilitate such curved arrangement of the light sources 22, in some examples, the light sources 22 can be disposed on a flexible substrate (e.g., flexible PCB) having a curvature along one or more directions. For example, the curved flexible substrate can be curved along the first direction of divergence of the emitted light beams 52 and the second direction of divergence of the emitted light beams 52. Additionally or alternatively, to facilitate such curved arrangement of the light sources 22, in some examples, the light sources 22 can be disposed on a curved edge of one or more vertically-oriented printed circuit boards (PCBs), such that the curved edge of the PCB substantially matches the curvature of the first direction (e.g., the vertical plane of the PCB). In this example, the one or more PCBs can be mounted in the transmit block 20 along a horizontal curvature that substantially matches the curvature of the second direction (e.g., the horizontal plane of the one or more PCBs). For example, the transmit block 20 can include four PCBs, with each PCB mounting sixteen light sources, so as to provide 64 light sources along the curved surface of the transmit block 20. In this example, the 64 light sources are arranged in a pattern such that the emitted light beams 52 converge towards the exit aperture 26 of the transmit block 20.
The transmit block 20 can optionally include the mirror 24 along the transmit path of the emitted light beams 52 between the light sources 22 and the exit aperture 26. By including the mirror 24 in the transmit block 20, the transmit path of the emitted light beams 52 can be folded to provide a smaller size of the transmit block 20 and the housing 12 of the sensing system 10 than a size of another transmit block where the transmit path that is not folded.
The receive block 30 includes a plurality of detectors 32 that can be configured to receive the focused light 58 via an entrance aperture 36. In some examples, each of the plurality of detectors 32 is configured and arranged to receive a portion of the focused light 58 corresponding to a light beam emitted by a corresponding light source of the plurality of light sources 22 and reflected of the one or more objects in the environment of the sensing system 10. The receive block 30 can optionally include the detectors 32 in a sealed environment having an inert gas 34.
The detectors 32 may comprise photodiodes, avalanche photodiodes, single-photon avalanche diodes (SPADs), phototransistors, silicon photomultipliers (SiPMs), cameras, active pixel sensors (APS), charge coupled devices (CCD), cryogenic detectors, or any other sensor of light configured to receive focused light 58 having wavelengths in the wavelength range of the emitted light beams 52.
To facilitate receiving, by each of the detectors 32, the portion of the focused light 58 from the corresponding light source of the plurality of light sources 22, the detectors 32 can be disposed on one or more substrates and arranged accordingly. For example, the light sources 22 can be arranged along a curved surface of the transmit block 20. Detectors 32 can be arranged along a curved surface of the receive block 30. In some embodiments, the curved surface of the receive block 30 may include a similar or identical curved surface as that of transmit block 20. Thus, each of the detectors 32 may be configured to receive light that was originally emitted by a corresponding light source of the plurality of light sources 22.
To provide the curved surface of the receive block 30, the detectors 32 can be disposed on the one or more substrates similarly to the light sources 22 disposed in the transmit block 20. For example, the detectors 32 can be disposed on a flexible substrate (e.g., flexible PCB) and arranged along the curved surface of the flexible substrate to each receive focused light originating from a corresponding light source of the light sources 22. In this example, the flexible substrate may be held between two clamping pieces that have surfaces corresponding to the shape of the curved surface of the receive block 30. Thus, in this example, assembly of the receive block 30 can be simplified by sliding the flexible substrate onto the receive block 30 and using the two clamping pieces to hold it at the correct curvature.
The focused light 58 traversing along the receive path can be received by the detectors 32 via the entrance aperture 36. In some examples, the entrance aperture 36 can include a filtering window that passes light having wavelengths within the wavelength range emitted by the plurality of light sources 22 and attenuates light having other wavelengths. In this example, the detectors 32 receive the focused light 58 substantially comprising light having the wavelengths within the wavelength range.
In some examples, the plurality of detectors 32 included in the receive block 30 can include, for example, avalanche photodiodes in a sealed environment that is filled with the inert gas 34. The inert gas 34 may comprise, for example, nitrogen.
The shared space 40 includes the transmit path for the emitted light beams 52 from the transmit block 20 to the lens 50, and includes the receive path for the focused light 58 from the lens 50 to the receive block 30. In some examples, the transmit path at least partially overlaps with the receive path in the shared space 40. By including the transmit path and the receive path in the shared space 40, advantages with respect to size, cost, and/or complexity of assembly, manufacture, and/or maintenance of the sensing system 10 can be provided.
While the exit aperture 26 and the entrance aperture 36 are illustrated as being part of the transmit block 20 and the receive block 30, respectively, it is understood that such apertures may be arranged or placed at other locations. In some embodiments, the function and structure of the exit aperture 26 and the entrance aperture 36 may be combined. For example, the shared space 40 may include a shared entrance/exit aperture. It will be understood that other ways to arrange the optical components of system 10 within housing 12 are possible and contemplated.
In some examples, the shared space 40 can include a reflective surface 42. The reflective surface 42 can be arranged along the receive path and configured to reflect the focused light 58 towards the entrance aperture 36 and onto the detectors 32. The reflective surface 42 may comprise a prism, mirror or any other optical element configured to reflect the focused light 58 towards the entrance aperture 36 in the receive block 30. In some examples, a wall may separate the shared space 40 from the transmit block 20. In these examples, the wall may comprise a transparent substrate (e.g., glass) and the reflective surface 42 may comprise a reflective coating on the wall with an uncoated portion for the exit aperture 26.
In embodiments including the reflective surface 42, the reflective surface 42 can reduce size of the shared space 40 by folding the receive path similarly to the mirror 24 in the transmit block 20. Additionally or alternatively, in some examples, the reflective surface 42 can direct the focused light 58 to the receive block 30 further providing flexibility to the placement of the receive block 30 in the housing 12. For example, varying the tilt of the reflective surface 42 can cause the focused light 58 to be reflected to various portions of the interior space of the housing 12, and thus the receive block 30 can be placed in a corresponding position in the housing 12. Additionally or alternatively, in this example, the sensing system 10 can be calibrated by varying the tilt of the reflective surface 42.
The lens 50 mounted to the housing 12 can have an optical power to both collimate the emitted light beams 52 from the light sources 22 in the transmit block 20, and focus the reflected light 56 from the one or more objects in the environment of the sensing system 10 onto the detectors 32 in the receive block 30. In one example, the lens 50 has a focal length of approximately 120 mm. By using the same lens 50 to perform both of these functions, instead of a transmit lens for collimating and a receive lens for focusing, advantages with respect to size, cost, and/or complexity can be provided. In some examples, collimating the emitted light beams 52 to provide the collimated light beams 54 allows determining the distance travelled by the collimated light beams 54 to the one or more objects in the environment of the sensing system 10.
While, as described herein, lens 50 is utilized as a transmit lens and a receive lens, it will be understood that separate lens and/or other optical elements are contemplated within the scope of the present disclosure. For example, lens 50 could represent distinct lenses or lens sets along discrete optical transmit and receive paths.
In an example scenario, the emitted light beams 52 from the light sources 22 traversing along the transmit path can be collimated by the lens 50 to provide the collimated light beams 54 to the environment of the sensing system 10. The collimated light beams 54 may then reflect off the one or more objects in the environment of the sensing system 10 and return to the lens 50 as the reflected light 56. The lens 50 may then collect and focus the reflected light 56 as the focused light 58 onto the detectors 32 included in the receive block 30. In some examples, aspects of the one or more objects in the environment of the sensing system 10 can be determined by comparing the emitted light beams 52 with the focused light beams 58. The aspects can include, for example, distance, shape, color, and/or material of the one or more objects. Additionally, in some examples, by rotating the housing 12, a three-dimensional map of the surroundings of the sensing system 10 can be determined.
In some examples where the plurality of light sources 22 are arranged along a curved surface of the transmit block 20, the lens 50 can be configured to have a focal surface corresponding to the curved surface of the transmit block 20. For example, the lens 50 can include an aspheric surface outside the housing 12 and a toroidal surface inside the housing 12 facing the shared space 40. In this example, the shape of the lens 50 allows the lens 50 to both collimate the emitted light beams 52 and focus the reflected light 56. Additionally, in this example, the shape of the lens 50 allows the lens 50 to have the focal surface corresponding to the curved surface of the transmit block 20. In some examples, the focal surface provided by the lens 50 substantially matches the curved shape of the transmit block 20. Additionally, in some examples, the detectors 32 can be arranged similarly in the curved shape of the receive block 30 to receive the focused light 58 along the curved focal surface provided by the lens 50. Thus, in some examples, the curved surface of the receive block 30 may also substantially match the curved focal surface provided by the lens 50.
System 100 includes a plurality of light-emitter devices 110, a receiver subsystem 120, and a controller 150. System 100 also includes a light pulse schedule 160.
In some embodiments, the plurality of light-emitter devices 110 can include laser diodes, light-emitting diodes, or other types of light-emitting devices. In an example embodiment, the plurality of light-emitter devices 110 include InGaAs/GaAs laser diodes configured to emit light at a wavelength around 903 nanometers. In some embodiments, the plurality of light-emitting devices 110 comprises at least one of: a laser diode, a laser bar, or a laser stack. Additionally or alternatively, the plurality of light-emitter devices 110 may include one or more master oscillator power amplifier (MOPA) fiber lasers. Such fiber lasers may be configured to provide light pulses at or around 1550 nanometers and may include a seed laser and a length of active optical fiber configured to amplify the seed laser light to higher power levels. However, other types of light-emitting devices, materials, and emission wavelengths are possible and contemplated.
In some embodiments, the plurality of light-emitter devices 110 is configured to emit light into an environment along a plurality of emission vectors toward respective target locations so as to provide a desired resolution. In such scenarios, the plurality of light-emitter devices 110 is operable to emit light along the plurality of emission vectors such that the emitted light interacts with an external environment of the system 100.
In some embodiments, the system 100 may include at least one substrate having a plurality of angled facets along a front edge. In such scenarios, each angled facet may include a respective die attach location. As an example, each light-emitter device could be coupled to a respective die attach location so as to be operable to emit light along its respective emission vector.
In such embodiments, the at least one substrate may be disposed along one or more a vertical planes. In such a scenario, the plurality of emission vectors may be defined with respect to a horizontal plane. Furthermore, as an example, the at least one substrate may be oriented vertically within a housing configured to rotate about a rotational axis, which may itself be substantially vertical. In other words, the plurality of light-emitter devices 110 and the receiver subsystem 120 may be coupled to the housing. In such scenarios, the housing is configured to rotate about a rotational axis.
For example, each light-emitter device could be oriented along the common substrate so as to emit light toward a respective target location along a respective emission vector. It will be understood that many different physical and optical techniques may be used to direct light toward a given target location. All such physical and optical techniques are contemplated herein.
In some embodiments, the desired resolution could include a target resolution at a given distance away from the system 100. For example, the desired resolution may include a vertical resolution of 7.5 centimeters at 25 meters from the system 100 and/or between adjacent target locations along a horizontal ground plane, whichever is closer. Other desired resolutions, both along a two-dimensional surface and within three-dimensional space, are possible and contemplated herein.
In some embodiments, the at least one substrate could be disposed along a vertical plane. In such a scenario, at least two emission vectors of the plurality of emission vectors may vary with respect to a horizontal plane.
In embodiments where the plurality of light-emitter devices 110 are distributed over a plurality of substrates, each portion of the plurality of light-emitter devices 110 could configured to illuminate the environment at a respective pointing angle with respect to the vertical plane. As an example, the plurality of light-emitter devices 110 may include at least 64 light-emitter devices. However, a greater or fewer number of light-emitter devices 110 could be used.
In some embodiments, the plurality of light-emitter devices 110 could be configured to provide light pulses between approximately 1-10 nanoseconds in duration. Other light pulse durations are possible.
In some embodiments, system 100 may include optical elements (not illustrated), which could include respective lenses optically coupled to a respective output facet of the respective light-emitter devices. The respective lenses may include, but are not limited to, fast-axis collimating lenses.
In some embodiments, the receiver subsystem 120 may be similar or identical to receiver block 30, as illustrated and described in reference to
The receiver subsystem 120 may include a plurality of light-detector devices. In such scenarios, the plurality of light-detector devices could be configured to detect light having a wavelength of at least one of: 1550 nm or 780 nm. Other wavelengths are possible and contemplated herein. In some embodiments, the light-detector devices could include at least one of: an avalanche photodiode, a single photon avalanche detector (SPAD), or a silicon photomultiplier (SiPM). In further embodiments, the light-detector devices may include a plurality of InGaAs photodetectors. Other types of photodetectors are possible and contemplated.
The controller 150 may include an on-board vehicle computer, an external computer, or a mobile computing platform, such as a smartphone, tablet device, personal computer, wearable device, etc. Additionally or alternatively, the controller 150 may include, or be connected to, a remotely-located computer system, such as a cloud server network. In an example embodiment, the controller 150 may be configured to carry out some or all method blocks or steps described herein.
The controller 150 may include one or more processors 152 and at least one memory 154. The processor 152 may include, for instance, an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA). Other types of processors, computers, or devices configured to carry out software instructions are contemplated herein. The memory 154 may include a non-transitory computer-readable medium, such as, but not limited to, read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), non-volatile random-access memory (e.g., flash memory), a solid state drive (SSD), a hard disk drive (HDD), a Compact Disc (CD), a Digital Video Disk (DVD), a digital tape, read/write (R/W) CDs, R/W DVDs, etc.
The one or more processors 152 of controller 150 may be configured to execute instructions stored in the memory 154 so as to carry out various operations described herein.
Additionally or alternatively, the controller 150 could include a circuit (e.g., a synchronous digital circuit) operable to carry out the various operations described herein. For example, the circuit may include a shot table. Other functions of the circuit (e.g., reading and sequencing) may be performed by a synchronous digital logic circuit. In some embodiments, the circuit and its operation may be specified in Verilog or another hardware description language. In such scenarios, the controller 150 need not include a processor.
The operations carried out by the controller 150 may include determining, for at least one light-emitter device of the plurality of light-emitter devices, a light pulse schedule. The light pulse schedule is based on a respective emission vector of the at least one light-emitter device and a three-dimensional map of the external environment. The light pulse schedule includes at least one light pulse parameter and a listening window duration. In some embodiments, the at least one light pulse parameter may include at least one of: a desired pulse onset time, a desired wavelength, a desired pulse power, or a desired pulse duration. Other types of light pulse parameters may be possible and contemplated herein.
In some embodiments, the operation of determining the light pulse schedule may include determining an object and a corresponding object distance. As a non-limiting example, the object may include at least one of: a ground surface, a vehicle, an obstacle, or an occluding element. In some embodiments, the object is located along the respective emission vector of the at least one light-emitter device in the external environment. Furthermore, the operations may include determining the listening window duration based on the corresponding object distance and a speed of the light pulse.
The operations include causing the at least one light-emitter device of the plurality of light-emitter devices to emit a light pulse according to the light pulse schedule.
In some embodiments, the operations may additionally or alternatively include, during the listening window duration, receiving information indicative of an interaction between the light pulse and the external environment. In such scenarios, the operations may include, based on the received information, adjusting the three-dimensional map of the external environment. Yet further, in some embodiments, the operations may also include, based on the received information, adjusting the light pulse schedule. In some embodiments, the listening window duration is adjustable within an inclusive range between 200 nanoseconds and 2 microseconds. However, other listening window durations are possible and contemplated.
Block 202 of timing sequence 200 includes causing a light-emitter device to transmit a light pulse into an environment (e.g., an environment of the system 100) at t0. The listening window may commence once a light pulse is emitted and may remain “open” until tlistening. That is, during the listening window between t0 and tlistening, the receiver subsystem (e.g., receiver subsystem 120) may be operable to receive a reflected light pulse that has interacted with an object in the environment.
As described in block 204, in some instances, no reflected light, or insufficient reflected light, may be received by the receiver subsystem during the listening window of duration tlistening. In such a scenario, the listening window may “close” or expire as illustrated in block 206. In such scenarios, the system 100 may determine that no object is present along the emission vector of the light pulse within a predetermined range. In some embodiments, the predetermined range may be based on the maximum roundtrip distance that the light pulse may be able to travel within tlistening (e.g., (tlistening/2)*speed of light).
Turning to timing sequence 210, block 202 may again include causing a light-emitter device to emit a light pulse into an environment at to. In block 212, the light pulse may interact with an object in the environment at t1. For instance, the light pulse may be reflected from the object and at least a portion of the light from the light pulse may be redirected back toward the receiver subsystem. In block 214, the reflected portion of light may be received by the receiver subsystem (e.g., receiver subsystem 120) at t2, which is prior to tlistening,1. According to timing sequence 210, block 206 may include the listening window expiring (at time tlistening,1). Upon expiration of the listening window, block 216 may repeat the timing sequence 210 with subsequent light pulses and listening windows.
Timing sequence 220 may illustrate some of the embodiments described herein. Namely, timing sequence 220 may provide a way to dynamically adjust subsequent listening window durations based on objects in an environment of the system. In such scenarios, block 202 includes causing a light-emitting device to emit a light pulse into the environment at to. Furthermore, the light pulse may interact with the object in the environment at t1. Yet further, block 214 may include that the reflected light (or at least a portion thereof) is received by the receiver subsystem at t2. In some embodiments, the listening window tlistening,1 may remain open for the predetermined time. However, in some cases, the listening window need not remain open for the predetermined time and the next light pulse could be emitted immediately after receiving the reflected light.
Additionally or alternatively, upon receiving the reflected light, the system (e.g., system 100) may carry out block 222. Block 222 may include adjusting a subsequent listening window tlistening,2. In some embodiments, the subsequent listening window may correspond to the immediate next light pulse and/or another future light pulse that is anticipated to interact with the object in the environment. In some embodiments, block 222 may be carried out at tlistening,1, or at a later time.
In example embodiments, tlistening,2 could be adjusted to t2. That is, the subsequent listening window could be shortened to match the roundtrip time for the prior light pulse to interact with the object and return to the receiver subsystem.
Additionally or alternatively, the subsequent listening period tlistening,2 could be adjusted to t2+tbuffer. In such embodiments, tbuffer may include a “buffer time” to ensure the subsequent listening window stays open long enough to detect reflected light from the object. In some embodiments, the buffer time may be based on a determined velocity vector of the object. For example, if the object is determined to be moving away from the light source, the buffer time tbuffer may greater than that of a scenario in which the light source to object distance is constant. Furthermore, tbuffer may be a predetermined amount of time that could correspond to a maximum velocity vector of the object. For example, the predetermined amount of buffer time could be based on a maximum distance the object could be at when a subsequent light pulse interacts with the object. It will be understood that tbuffer may be based on other considerations as well, all of which are contemplated herein.
The timing diagram 220 relates to setting a predetermined listening period based on an object determined in an environment of the system. It will be understood that such a timing diagram 220 may be repeated in serial and/or parallel fashion for each light-emitter device and/or each light pulse emitted by the system. As such, a light pulse schedule (e.g., light pulse schedule 160) may be built up and/or adjusted based on objects that are detected in the environment. Namely, listening periods may be reduced or minimized so as to reduce the time spent waiting for a listening period to expire even though a reflected light pulse had already been detected. By utilizing dynamic listening periods, more light pulses could be emitted (and reflected light pulses received), which could provide higher spatial and/or temporal sensing resolution and/or a larger field of view sensing within a given amount of time.
One or more of the sensor systems 302, 304, 306, 308, and 310 may be configured to rotate about an axis (e.g., the z-axis) perpendicular to the given plane so as to illuminate an environment around the vehicle 300 with light pulses. Based on detecting various aspects of reflected light pulses (e.g., the elapsed time of flight, polarization, etc.), information about the environment may be determined.
In an example embodiment, sensor systems 302, 304, 306, 308, and 310 may be configured to provide respective point cloud information that may relate to physical objects within the environment of the vehicle 300. While systems 10 and 100, vehicle 300 and sensor systems 302 and 304 are illustrated as including certain features, it will be understood that other types of systems are contemplated within the scope of the present disclosure.
As an example, an example embodiment may include a system having a plurality of light-emitter devices. The system may include a transmit block of a LIDAR device. For example, the system may be, or may be part of, a LIDAR device of a vehicle (e.g., a car, a truck, a motorcycle, a golf cart, an aerial vehicle, a boat, etc.). Each light-emitter device of the plurality of light-emitter devices is configured to emit light pulses along a respective beam elevation angle. The respective beam elevation angles could be based on a reference angle or reference plane, as described elsewhere herein. In some embodiments, the reference plane may be based on an axis of motion of the vehicle.
While certain description and illustrations herein describe systems with multiple light-emitter devices, LIDAR systems with fewer light-emitter devices (e.g., a single light-emitter device) are also contemplated herein. For example, light pulses emitted by a laser diode may be controllably directed about an environment of the system. The angle of emission of the light pulses may be adjusted by a scanning device such as, for instance, a mechanical scanning mirror and/or a rotational motor. For example, the scanning devices could rotate in a reciprocating motion about a given axis and/or rotate about a vertical axis. In another embodiment, the light-emitter device may emit light pulses towards a spinning prism mirror, which may cause the light pulses to be emitted into the environment based on an angle of the prism mirror angle when interacting with each light pulse. Additionally or alternatively, scanning optics and/or other types of electro-opto-mechanical devices are possible to scan the light pulses about the environment.
In some embodiments, a single light-emitter device may emit light pulses according to a variable shot schedule and/or with variable power per shot, as described herein. That is, emission power and/or timing of each laser pulse or shot may be based on a respective elevation angle of the shot. Furthermore, the variable shot schedule could be based on providing a desired vertical spacing at a given distance from the LIDAR system or from a surface (e.g., a front bumper) of a given vehicle supporting the LIDAR system. As an example, when the light pulses from the light-emitter device are directed downwards, the power-per-shot could be decreased due to a shorter anticipated maximum distance to target. Conversely, light pulses emitted by the light-emitter device at an elevation angle above a reference plane may have a relatively higher power-per-shot so as to provide sufficient signal-to-noise to adequately detect pulses that travel longer distances.
Furthermore, the shot schedule could be adjusted to reduce the wait time until a subsequent shot for a light pulse that is directed downwards. That is, due to a shorter distance traveled, the listening window may not be as long in duration as that for light pulses that travel farther within a given environment.
As a further example, sensor system 304 may be configured to emit light pulses into an environment of the vehicle 300 over an elevation angle range 420, which may be defined between a maximum elevation angle 422 and a minimum elevation angle 424. In some embodiments, a plurality of light-emitter devices of sensor system 304 may illuminate the environment about the vehicle 300 with a non-linear elevation angle distribution. That is, to achieve a desired vertical beam resolution, the plurality of light-emitter devices of sensor system 304 may be arranged over a set of beam elevation angles that include heterogeneous differences in elevation angle between adjacent beams.
As illustrated in
In scenarios where the first light pulse path 416 is determined to be unobstructed, or is anticipated to be unobstructed, a predetermined listening period associated with a given light pulse could be set to a maximum listening period duration (e.g., 2 microseconds). Other listening period durations are possible and contemplated. In some embodiments, setting the listening period to a maximum duration may provide sensing of objects when they become closer than a predetermined maximum sensing distance.
As illustrated in
In scenarios where the second light pulse path 426 is determined to be unobstructed, or is anticipated to be unobstructed, a predetermined listening period associated with a given light pulse emitted along the second light pulse path 426 could be set to a maximum listening period duration (e.g., 2 microseconds). Other listening period durations are possible and contemplated.
As illustrated in
Additionally, a second light pulse path 434 extending from sensor system 304 may also intersect with the object (e.g., at a rear bumper portion of the other vehicle 440).
In sensing scenario 430, at least some light of the emitted light pulses may be reflected by the surface 442 as reflected light. When the reflected light is received by sensor systems 302 or 304, a round trip time may be provided. Based at least on the determined round trip time, a relative distance between the sensor systems 302 and 304 and the other vehicle 440 could be estimated or otherwise determined.
As such, under methods and systems described herein, a light pulse emitted (by sensor system 302 along the first light pulse path 432 or by sensor system 304 along the second light pulse path 434) may have an associated listening period that could be based on the estimated or anticipated distance to the object. As an example, the listening period duration could be set to an amount of time equal to the anticipated round trip time (e.g., 200 nanoseconds) to the surface 442 and back. Other listening period durations are possible and contemplated.
As described elsewhere herein, the estimated or anticipated distance to the object could be based on a LIDAR point cloud, or a three-dimensional (e.g., a depth map) or two-dimensional map. Furthermore, the associated listening period set for light pulses emitted toward anticipated objects could be based on the anticipated distance to the target or the anticipated distance to the target plus a buffer time so as to account for possible relative movement between the sensor systems 302 and 304 and the vehicle.
Sensing scenario 450 may include respective light pulse schedules (e.g., light pulse schedule 160) for each of the sensor systems 302, 308 and 310. Furthermore, each of the light pulse schedules may include predetermined listening periods based on either a maximum sensing distance or an anticipated ground location.
Example embodiments may include adjusting various aspects of an emitted light pulse and/or a listening window duration based on a changing environment around the vehicle as it moves around the world. Specifically, aspects of the emitted light pulses and their associated listening window durations may be varied based on, without limitation, an undulating roadway (e.g., grade changes when driving uphill or downhill, driving around a curve, etc.), objects on or adjacent to the roadway (e.g., pedestrians, other vehicles, buildings, etc.), or other static or dynamically-varying environmental conditions or contexts.
Accordingly, in some embodiments and under some conditions, systems and methods described herein need not always scan the entire range of possible beam elevation angles (e.g., angles between an entire angle range between minimum beam elevation angle 492 and the maximum beam elevation angle 493). Instead, the beam-scanning range may be varied based on the dynamically-changing yaw-dependent contours of the roadway and/or other portions of the environment around the vehicle 300.
Referring to
In some embodiments, the listening window durations may be adjusted within a range of predetermined listening window durations. In such a scenario, the range of predetermined listening window durations could include a maximum listening window duration, which could correspond to a maximum detection range, and a minimum listening window duration, which could correspond to a minimum detection range. By way of example, the maximum detection range could be approximately 200 meters or more. In some embodiments, the minimum detection range could be approximately 5 meters or less. Correspondingly, to detect a round-trip light pulse, the maximum and minimum listening window durations could be 1.3 microseconds and 33 nanoseconds, respectively. Other maximum and minimum listening window durations are possible and contemplated.
Furthermore, for the light pulses that are emitted into the angles between the minimum beam elevation angle 492 and the “ground-skimming” beam elevation angle 494, the corresponding listening windows may be lengthened (or maximized) in an effort to increase the likelihood that objects on or close to the ground will be detected.
In some embodiments, systems and methods described herein may adjust listening windows and/or aspects of light pulse emission based on a contour line that extends around the vehicle (e.g., through 360 degrees or a plurality of yaw angles) and may be defined as a continuous line that is located at a predetermined distance away from the vehicle 300 (e.g., 60, 100, or 200 meters away) and/or at a predetermined height above a ground surface. Such a contour line may be dynamically adjusted as the vehicle 300 moves around its environment. The contour line could be determined based on a topographic map or current or prior point cloud information obtained by the vehicle 300 and/or other vehicles. In some embodiments, the contour line could pass through various points described in
In other words, consider a scenario where the contour line represents a predetermined height of one meter from the ground at 60 meters distance from the vehicle 300. When the vehicle 300 is on level terrain with no objects at one meter from the ground, the contour line could be represented by a two-dimensional circle with 60 meter radius that is centered on the vehicle. However, when the vehicle 300 encounters hilly terrain and/or objects at one meter from the ground, the contour line could include a three-dimensional circle, oval, or irregular shape based on topographical features and/or object data. In some embodiments, the listening window durations could be adjusted based on a shape of the contour line.
As described above, in some embodiments, light pulses need not be emitted into beam elevation angles above the “ground-skimming” beam elevation angle 498. Additionally or alternatively, listening window durations for light pulses emitted into such an elevation angle range may be reduced or eliminated altogether. Other distinctions between transmission and reception of light pulses into yaw-dependent beam angles ranges are possible based on a topographic map, point cloud information, or other knowledge about objects and/or ground surfaces within an environment of the vehicle 300. In some embodiments, the point cloud information may be gathered by a vehicle utilizing the LIDAR system (from a previous scan earlier in the drive and/or from a scan from a prior drive of the vehicle along the same route) or another vehicle utilizing a LIDAR system. The other vehicle could be part of a common fleet of vehicles or be associated with a different fleet.
The plurality of light-emitter devices 516a-516j could be attached to respective die attach locations. As such, each light-emitter device could be oriented so as to emit light pulses along a different elevation angle. Furthermore, each respective light-emitter device of the plurality of light-emitter devices 516a-516j could be attached to a respective plurality of pulser circuits 520a-520j. In some example embodiments, the respective pulser circuits could cause the light-emitter devices 516a-516j to emit light pulses as described herein. Furthermore, the plurality of pulser circuits 520a-520j could be controlled at least in part, based on a light pulse schedule (e.g., light pulse schedule 160) as described herein.
The light pulse schedule could be adjusted dynamically based on an object being sensed, or anticipated to be, along a given respective elevation angle (or emission vector). In some embodiments, if a given light pulse is anticipated or interact with an object in the environment, a corresponding listening window may be set or adjusted based, at least in part, on a known or anticipated distance to that object.
Furthermore, other optical elements may be included in system 500. For example, a respective plurality of lenses 518a-518j could be optically coupled to the respective plurality of light-emitter devices 516a-516j. Yet further, other elements, such as alignment features 524, communication interface 522, socket 521, and other electronic components 523a and 523b may be included in system 500.
Block 602 includes determining, for at least one light-emitter device of a plurality of light-emitter devices, a light pulse schedule. In such a scenario, the plurality of light-emitter devices is operable to emit light along a plurality of emission vectors. The light pulse schedule is based on a respective emission vector of the at least one light-emitter device and a three-dimensional map of an external environment. The light pulse schedule includes at least one light pulse parameter and a listening window duration.
In some embodiments, determining the light pulse schedule may include determining an object and a corresponding object distance. In such scenarios, the object is located along the respective emission vector of the at least one light-emitter device in the external environment. The method 600 may also include determining the listening window duration based on the corresponding object distance and a speed of the light pulse.
In some embodiments, the object could include at least one of: a ground surface, a vehicle, an obstacle, or an occluding element.
Furthermore, in some cases, determining the light pulse schedule can include adjusting the listening window duration within an inclusive range between about 200 nanoseconds and about 2 microseconds. Other listening window durations are possible and contemplated.
Block 604 includes causing the at least one light-emitter device of the plurality of light-emitter devices to emit a light pulse according to the light pulse schedule. The light pulse interacts with an external environment.
In some embodiments, method 600 includes causing a housing to rotate about a rotational axis. In such scenarios, the plurality of light-emitter devices is coupled to the housing. As described elsewhere herein, the light-emitter devices could be rotated about a rotational axis similar to that of sensor 302, as illustrated and described in relation to
Additionally, method 600 may include, during the listening window duration, receiving information indicative of an interaction between the light pulse and the external environment. In such scenarios, method 600 may also include, based on the received information, adjusting the three-dimensional map of the external environment. Additionally or alternatively, method 600 may include, based on the received information, adjusting the light pulse schedule.
The particular arrangements shown in the Figures should not be viewed as limiting. It should be understood that other embodiments may include more or less of each element shown in a given Figure. Further, some of the illustrated elements may be combined or omitted. Yet further, an illustrative embodiment may include elements that are not illustrated in the Figures.
A step or block that represents a processing of information can correspond to circuitry that can be configured to perform the specific logical functions of a herein-described method or technique. Alternatively or additionally, a step or block that represents a processing of information can correspond to a module, a segment, a physical computer (e.g., a field programmable gate array (FPGA) or application-specific integrated circuit (ASIC)), or a portion of program code (including related data). The program code can include one or more instructions executable by a processor for implementing specific logical functions or actions in the method or technique. The program code and/or related data can be stored on any type of computer readable medium such as a storage device including a disk, hard drive, or other storage medium.
The computer readable medium can also include non-transitory computer readable media such as computer-readable media that store data for short periods of time like register memory, processor cache, and random access memory (RAM). The computer readable media can also include non-transitory computer readable media that store program code and/or data for longer periods of time. Thus, the computer readable media may include secondary or persistent long term storage, like read only memory (ROM), optical or magnetic disks, compact-disc read only memory (CD-ROM), for example. The computer readable media can also be any other volatile or non-volatile storage systems. A computer readable medium can be considered a computer readable storage medium, for example, or a tangible storage device.
While various examples and embodiments have been disclosed, other examples and embodiments will be apparent to those skilled in the art. The various disclosed examples and embodiments are for purposes of illustration and are not intended to be limiting, with the true scope being indicated by the following claims.
The present application is a continuation application claiming priority to U.S. patent application Ser. No. 15/852,788, filed Dec. 22, 2017, the contents of which are hereby incorporated by reference.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6057909 | Yahav et al. | May 2000 | A |
| 7733464 | David et al. | Jun 2010 | B2 |
| 8436763 | Wintermantel | May 2013 | B2 |
| 9103669 | Giacotto et al. | Aug 2015 | B2 |
| 9383753 | Templeton et al. | Jul 2016 | B1 |
| 20080266171 | Weber et al. | Oct 2008 | A1 |
| 20100135007 | Seabrook | Jun 2010 | A1 |
| 20130088872 | Ball | Apr 2013 | A1 |
| 20130201288 | Billerbeck et al. | Aug 2013 | A1 |
| 20150226853 | Seo et al. | Aug 2015 | A1 |
| 20160245919 | Kalscheur | Aug 2016 | A1 |
| 20160274589 | Templeton et al. | Sep 2016 | A1 |
| 20170003392 | Bartlett | Jan 2017 | A1 |
| 20170219695 | Hall et al. | Aug 2017 | A1 |
| 20180284278 | Russell et al. | Oct 2018 | A1 |
| Number | Date | Country |
|---|---|---|
| 1502047 | Jun 2004 | CN |
| 102713667 | Oct 2012 | CN |
| 105556338 | May 2016 | CN |
| 3217190 | Sep 2017 | EP |
| 05-057688 | Jul 1993 | JP |
| 07-191146 | Jul 1995 | JP |
| 07-209421 | Aug 1995 | JP |
| 07-260937 | Oct 1995 | JP |
| 2014-145697 | Aug 2014 | JP |
| 2016-017785 | Feb 2016 | JP |
| 2016-534346 | Nov 2016 | JP |
| 10-2017-0132884 | Dec 2017 | KR |
| 2014032831 | Mar 2014 | WO |
| 2017110417 | Jun 2017 | WO |
| 2017132703 | Aug 2017 | WO |
| 2017192566 | Nov 2017 | WO |
| Entry |
|---|
| International Searching Authority, International Search Report and Written Opinion dated Mar. 4, 2019, issued in connection with International Patent Application No. PCT/US2018/058452, filed on Oct. 31, 2018, 12 pages. |
| European Patent Office, Extended European Search Report dated Jul. 8, 2021, issued in connection with European Patent Application No. 18892337.9, 12 pages. |
| Number | Date | Country | |
|---|---|---|---|
| 20220236377 A1 | Jul 2022 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 15852788 | Dec 2017 | US |
| Child | 17659674 | US |
