Patent Application
20180088211
This disclosure relates generally to LiDAR signal processing, and more specifically to a method for digitizing laser pulses using LiDAR signal processing.
Light Detection and Ranging (LiDAR) is a remote sensing technology that uses light pulses to measure ranges or distances of an object. It is a technology that has application in various fields including archeology, meteorology, bathymetry, etc. Further, the information extracted from these measurements can have many uses, including determining surface characteristics, creating elevation models, and even obtaining three-dimensional (3-D) images of an object at a distance. To determine the 3-D image of an object at a distance, differences (or time of flight estimations) are determined between the time transmission of laser light pulses and the reception of the sampled reflected signals. Generally, conventional components used in sampling and determining these time of flight estimations can be slow, inaccurate, or too large to provide the real time capabilities required in applications like space exploration and military surveillance.
The present disclosure is directed to an apparatus and methods for performing time of flight estimation. The apparatus includes a synthesizing card electrically coupled to an analog-to-digital converter, the synthesizing card transmitting a clock signal to the analog-to-digital converter for synchronizing the analog-to-digital converter. The apparatus can further include a receiver for receiving a plurality of radio frequency pulses, where each radio frequency pulse is digitized into an in-phase and quadrature sample at a clock edge of the clock signal at the analog-to-digital converter. The apparatus includes a buffer for storing each in-phase and quadrature sample digitized by the analog-to-digital converter. In addition, the apparatus includes at least one correlator electrically coupled to the buffer to determine a time of flight of each of the in-phase and quadrature samples.
The method can include transmitting a clock signal by a synchronizing card, to an analog-to-digital converter for synchronizing the analog-to-digital converter. The method can also include receiving, by a receiver, a plurality of radio frequency pulses, where each of the radio frequency pulse is digitized into an in-phase and quadrature sample at a clock edge of the clock signal at the analog-to-digital converter. The method can include storing, by a buffer, each in-phase and quadrature sample digitized by the analog-to-digital converter and retrieving and determining, by a correlator, a time of flight of each of the in-phase and quadrature samples.
The apparatus can also include a non-transitory machine readable medium having stored thereon machine-readable instructions executable to cause a machine to perform operations including transmitting a clock signal to an analog-to-digital converter for synchronizing the analog-to-digital converter. The medium can further execute instructions to perform receiving, by a receiver, a plurality of radio frequency pulses, where each radio frequency pulse is digitized into an in-phase and quadrature sample at a clock edge of the clock signal at the analog-to-digital converter. Also, the medium can include instructions causing the machine to execute instructions including storing, by a buffer, each in-phase and quadrature sample digitized by the analog-to-digital converter. The medium can also include a machine readable medium for retrieving and determining, by a correlator, a time of flight of each of the in-phase and quadrature samples.
The description will be more fully understood with reference to the following figures and charts, which are presented as various embodiments of the disclosure and should not be construed as a complete recitation of the scope of the disclosure, wherein:
Aspects of the present disclosure involve systems, methods, devices and the like for sampling received signals for performing time of flight estimation using LiDAR signal processing. In one aspect, a radio frequency analog-to-digital converter is used for real time waveform digitalization. The radio frequency analog-to-digital converter may be coupled to a mezzanine card and used to generate a clock for the converter. In some instances, the mezzanine card may use oscillators and a synthesizer to generate a clock signal that is used for sampling the received signals. The sampled signals may be stored in a buffer and correlated for time of flight estimation. The correlation may occur using numerous correlators running in parallel with the number of correlators varying based on the desired speed of the system.
The system bus 105 may he any of several types of bus structures, including a memory bus or memory controller, a peripheral bus, and a local bus using any of a variety of bus architectures. A basic input/output (BIOS) stored in ROM 120 or the like may provide a basic routine that helps to transfer information between elements within the computing device 100, such as during start-up. The computing device 100 further includes storage devices 130 or computer-readable storage media such as a hard disk drive, a magnetic disk drive, an optical disk drive, a tape drive, a solid-state drive, a RAM drive, a removable storage device, a redundant array of inexpensive disks (RAID), a hybrid storage device, or the like. The storage device 130 is connected to the system bus 105 by a drive interface. The drives and the associated computer-readable storage devices provide nonvolatile storage of computer-readable instructions, data structures, program modules, and other data for the computing device 100. In one aspect, a hardware module that performs a particular function includes the software component stored in a tangible computer-readable storage device in connection with the necessary hardware components, such as the processor 110, bus 105, display (or any output device) 135, and so forth, to carry out a particular function. In another aspect, the system can use a processor and a computer-readable storage device to store instructions which, when executed by the processor, cause the processor to perform operations, a method or other specific actions. The basic components and appropriate variations can be modified depending on the type of device, such as whether the device 100 is a small, handheld computing device, a desktop computer, or a computer server. When the processor 110 executes instructions to perform “operations,” the processor 110 can perform the operations directly and/or facilitate, direct, or cooperate with another device or component to perform the operations.
Although the exemplary embodiment(s) described herein employs the hard disk 130, other types of computer-readable storage devices which can store data that are accessible by a computer, such as magnetic cassettes, flash memory cards, digital versatile disks (DVDs), cartridges, random access memories (RAMS) 125, read only memory (ROM) 120, a cable containing a bit stream and the like, may also be used in the exemplary operating environment. According to this disclosure, tangible computer-readable storage media, computer-readable storage devices, computer-readable storage media, and computer-readable memory devices expressly exclude media such as transitory waves, energy, carrier signals, electromagnetic waves, and signals per se.
To enable user interaction with the computing device 100, an input device 145 can be any number of input mechanisms, such as a microphone for speech, a touch-sensitive screen for gesture or graphical input, a keyboard, a mouse, a motion input speech and so forth. An output device 135 can also be one or more of a number of output mechanisms known to those of skill in the art. In some instances, multimodal systems enable a user to provide multiple types of input to communicate with the computing device 100. The communications interface 140 generally governs and manages the user input and system output. There is no restriction on operating on any particular hardware arrangement, and therefore the basic hardware depicted may easily be substituted for improved hardware or firmware arrangements as they are developed.
For clarity of explanation, the illustrative system embodiment is presented as including individual functional blocks, including functional blocks labeled as a “processor” or processor 110. The functions of these blocks may be provided through the use of either shared or dedicated hardware, including, but not limited to, hardware capable of executing software and hardware, such as a processor 110, that is purpose-built to operate as an equivalent to software executing on a general purpose processor. For example, the functions of one or more processors presented in
The logical operations of the various embodiments are implemented as: (1) a sequence of computer implemented steps, operations, or procedures running on a programmable circuit within a general use computer; (2) a sequence of computer implemented steps, operations, or procedures running on a specific-use programmable circuit; and/or (3) interconnected machine modules or program engines within the programmable circuits. The system 100 shown in
LiDAR processing module 114 can use the computing device 100 of
One or more parts of the example computing device 100, up to and including the entire computing device 100, can be virtualized. For example, a virtual processor can be a software object that executes according to a particular instruction set, even when a physical processor of the same type as the virtual processor is unavailable. A virtualization layer or a virtual “host” can enable virtualized components of one or more different computing devices or device types by translating virtualized operations to actual operations. Ultimately however, virtualized hardware of every type is implemented or executed by some underlying physical hardware. Thus, a virtualization compute layer can operate on top of a physical compute layer. The virtualization compute layer can include one or more of a virtual machine, an overlay network, a hypervisor, virtual switching, and any other virtualization application.
The processor 110 can include all types of processors disclosed herein, including a virtual processor. However, when referring to a virtual processor, the processor 110 includes the software components associated with executing the virtual processor in a virtualization layer and the underlying hardware necessary to execute the virtualization layer. The system 100 can include a physical or virtual processor 110 that receives instructions stored in a computer-readable storage device, which can cause the processor 110 to perform certain operations. When referring to a virtual processor 110, the system also includes the underlying physical hardware executing the virtual processor 110.
In some embodiments, the core processing unit in the system can be the Goddard Space Flight Center (GSFC)-developed SpaceCube, a hybrid computing platform designed to provide command and data handling functions for earth-orbiting satellites. The SpaceCube includes five slices (cards): two Power Slices, two Processor Slices, and one Video Control Module (VCM) Slice. Other configurations are possible. Each processor slice contains two Xilinx Virtex Field Programmable Gate Arrays (FPGAs), and each FPGA contains two PPC405 processors running at 250 MHz. These eight processors host multiple instantiations of the system pose application FPose, along with command and telemetry handling software that allows a flight-like ground terminal to control the system remotely. The VCM provides sensor data compression and 16 Gb of flash memory to store raw sensor images for later playback. The Argon SpaceCube is an engineering development unit (EDU) version of the hardware own on the (Relative Navigation Sensor) RNS experiment and Materials International Space Station Experiment.
As indicated,
As conventionally understood, LiDAR includes the emission of laser pulses of light used to determine the range to a distant object. The distance to the object is determined by measuring the time delay between the emission of the laser pulse of light and the detection of the reflected signal using a correlator. In particular, the time delay can be determined by sampling the reflected signal and comparing against the transmitted pulse which oftentimes comes in the form of an ideal Gaussian pulse. System 200 illustrates some of the modules that can be used to perform both the digitizing and the comparison. As indicated, in one implementation, a LiDAR system may include PCIs 240,242 and wireless modules 222-226 which connect system 200 to external systems, devices, or components. For example, wireless modules 222-226 may be Airbone embedded dual band wireless device server and Ethernet solutions that .support Wi-Fi, Ethernet, and serial communications. Further, the wireless modules may be configured for UART, SPI, Ethernet, GPIO, and 802.11 interfaces.
The wireless modules 222-226 may then be connected to FPGAs 202,204 for processing and/or RF ADC 208 for digitizing the incoming waveforms from the wireless module 224. Generally, ADC modules have been placed external to system 200, however, as illustrated in
Coupled to the RF ADC 208 may be a mezzanine card 210 that generates the clock for use in sampling/digitizing the received signal by the RF ADC 208. Thus, the mezzanine card 210 may eliminate the need for an external clock as may be necessary by conventional systems. In the current embodiment, the clock is integrated into the system 200 through a connection established between the mezzanine card 210 and the on-board RF ADC 208. For example, the mezzanine card 210 may sit on top of the RF ADC 208 and interface via a connector. As another example, the mezzanine card 210 may reside on the breadboard of system 200. Still in another example, the mezzanine card 210 may reside on a separate system.
The mezzanine card 210 may include an evacuated controlled crystal oscillator (EMXO) 212, a synthesizer 214, a phase-locked loop (PLL) 216, and a voltage controlled oscillator (VCO) 218, which can generate the clock signal used by the RF ADC 208 for synchronizing the RF ADC 208 and sampling the received signal. As an example, a reference oscillator (e.g., EMXO 212) may generate a signal that is fed into a synthesizer 214. The synthesizer is a component that may be used to combine oscillators and filters to generate sounds, shapes, or signals that are desired. Thus, the synthesizer generates the desired signal and that is sent to a phase-locked loop (PLL) 216 which controls the voltage controlled oscillator (VCO) 218, such that the VCO 218 generates the clock that will be used by the RF ADC 208 for digitizing the received waveform in real time.
In particular, the analog-to-digital converter 208 may digitize the waveform into in-phase and quadrature components. The dual channel samples can then be stored in a buffer located in one of the FPGAs 202, 204. In one example, a digital bus may connect the RF ADC 208 to FPGA 204 which receives the digitized signal and stores it in a buffer located in the FPGA 204. In some instances, the digitized signal may be stored in an external buffer or memory module 230-238. Thus, the stored digitized signals may be retrieved and fed into correlators as necessary. Thus, the number of correlators instantiated to perform the time of flight estimations may vary based on the sample rate of the LiDAR system 200. For example, four, sixteen, or sixty correlators may run in parallel based in part on the sampling rate and application of the system 200. The time of flight estimates obtained from the correlators may be used to obtain the range of an object and/or image scan. Thus, system 200 may result in a capable 2 FPGA processor board (e.g., FPGA 0, 1) that can sample two differential RF channels at 1.54 GSPS within 12 bit resolution and process and/or interpret the signals corresponding to the object at a distance in real time.
Note that system 200 is used for exemplary purposes and other modules and configurations may be contemplated. Further, more or less modules may exist in the implementation of the RF ADC 208 and/or mezzanine card 210. For example, in addition to a digital bus coupling the RF ADC 208 to the FPGA 1204, a low voltage differential signaling bus 220, as well as additional memory components 230-238 may be present for LiDAR processing by the system 200. Additionally, other modules and/or further components can be included in system 200 that are not illustrated in
Further, RF ADC 208 along with the other components may be radiation hardened to protect against radiation. For example, in space interferometry, instruments are exposed to an ionizing dose of radiation which decrease instrument reliability and cause the electronics to degrade. In one embodiment, RF ADC 208 is radiation hardened to protect against such radiation. This protection provides the converter with a shield against high energy particles so that single event upsets are mitigated. For example, in this design, the RF ADC 208 may be hardened to 100 kilorads such that the converter reliability is increased and the electronics are protected, In other examples, the RF ADC 208 and other components in system 200 can be hardened to less than 100 kilorads (e.g., 15 kilorads) while others can be hardened to greater than 100 kilorads.
Once the clock signal is generated, it is sent, in operation 304, to the RF analog-to-digital converter. The RF ADC then uses the clock signal to sample reflected RF pulses received at a receiver and arriving at the RF ADC for digitizing in operation 306. Photodiodes in the receiver may sense the pulse and transmit them to a front panel of the digitizer (e.g., RF ADC). The received RF pulses may derive from an object at a distance whose range is detected and image scanned. In some embodiments. each RF pulse received may represent a pixel of the object at a distance. Once the RF pulse is received, it is digitized in operation 308. In particular, each pulse is sampled at the rising edge of the in-phase and quadrature (I and Q) channels of the clock. Thus, each pulse is digitized into in-phase and quadrature components and processed over two channels.
Once the received pulse waveform has been digitized, method 300 continues to operation 310, where the I and Q samples are transferred over a digital bus to a buffer for storage. In one embodiment, the I and Q samples are transferred over the digital bus for storage in a buffer within an FPGA. The samples remain stored until the time arrives to retrieve the samples from the buffer in operation 312. The stored samples are recovered for processing including correlating the samples to determine a time of flight estimate. In processing the samples, a specified number of correlators will be instantiated in order to achieve desired sampling rate of the system. That is to say, an appropriate number of correlators will work in parallel such that each correlator processes a pulse (representing a pixel) in order to achieve a desired system sampling rate. At the correlators, the transmitted pulse is correlated with the received reflected pulse in order to determine the time of flight of the signal and the corresponding range of the object.
Note that the LiDAR processing scheme presented in
In the present disclosure, the methods disclosed may be implemented as sets of instructions in hardware or software. It may be further understood that the specific order or hierarchy of steps in the methods disclosed are instances of example approaches. Based upon design preferences, it is understood that the specific order or hierarchy of steps in the method can be rearranged while remaining within the disclosed subject matter. The accompanying method claims present elements of the various steps in a sample order, and are not necessarily meant to be limited to the specific order or hierarchy presented.
While the present disclosure has been described with reference to various implementations, it will be understood that these implementations are illustrative and that the scope of the present disclosure is not limited to them. Many variations, modifications, additions, and improvements are possible. More generally, embodiments in accordance with the present disclosure have been described in the context of particular implementations. Functionality may be separated or combined in blocks differently in various embodiments of the disclosure or described with different terminology. These and other variations, modifications, additions, and improvements may fall within the scope of the disclosure as defined in the claims that follow.
