Patent Application
20220365210
A conventional vehicle (an automobile, a truck, etc.) includes ultrasonic sensors, where the ultrasonic sensors are configured to generate sensor signals that indicate whether or not an object is located proximate to the vehicle (e.g., within 3-5 meters of the vehicle) and within a “blind” spot of a driver of the vehicle. For example, when the driver of the vehicle moves the vehicle in reverse, a first ultrasonic sensor located on a bumper of the vehicle can emit an ultrasonic signal; when an object is proximate to the bumper of the vehicle, at least a portion of the ultrasonic signal reflects from the object and the same sensor or a second ultrasonic sensor detects the reflected ultrasonic signal. When a magnitude of the reflected signal is detected as being above a threshold, the vehicle is configured to output a warning to the driver that the object is proximate to the vehicle. The warning, however, typically does not inform the driver of how far away the object is from the vehicle, a location of the object relative to the vehicle, size of the object, whether the object is static or moving, etc.
Therefore, while ultrasonic sensors provide information that may be helpful to a driver of a vehicle, that information is somewhat low resolution. This at least partially due to design priorities for conventional vehicles; specifically, modern ultrasonic systems included in vehicles focus on minimizing cost to meet this limited scope, and as such are designed with the following goals prioritized: 1) limit bandwidth required between ultrasonic sensors of the vehicle and an electronic control unit (ECU) for the ultrasonic sensors; and 2) reduce an amount of inrush current to the ECU. These goals are driven by the use of particular protocols in the automotive industry for transmitting data between ultrasonic sensors and the ECU, where the protocols include the Controller Area Network (CAN) protocol, the Local Interconnect Network (LIN) protocol, and the Serial Digital Interface (SDI) 3 protocol.
These (and similar protocols) are associated with deficiencies that are not problematic for use in conventional vehicles (which are controlled by human operators). For instance, use of the protocols referenced above results in star topologies; this is not problematic for conventional vehicles, as such vehicles include a relatively small number of ultrasonic sensors due to the lack of requirement of 360-degree coverage. If, however, a vehicle was to include numerous ultrasonic sensors (e.g., dispersed around an entire periphery of the vehicle), the length of cabling between the ECU and the ultrasonic sensors may be several hundreds of meters, resulting in design complexity and weight added to the vehicle. Further, bit depth with these protocols is somewhat limited, and signals received by the ECU from ultrasonic sensors may have a relatively low signal-to-noise ratio (SNR). To address the relatively low SNR, analog gain is used in the ultrasonic sensors over a limited “listening now” window. Additionally bit depth is reduced by performing thermal compensation/geometric spreading compensation in an application specific integrated circuit (ASIC) of the ultrasonic sensors. The overall data rate is further reduced by configuring the ASICs of the ultrasonic sensors to perform cross-correlation between a detected signal and an expected signal.
Performing these two actions (thermal compensation/geometric spreading compensation and cross-correlation) in the ASIC of the ultrasonic sensors has the following negative impacts: 1) a sensor is prevented from being able to listen to multiple modulations at once; 2) the firing sequence of the sensor is slowed; and 3) data that may represent shape and speed of an object in proximity to the vehicle is discarded. While these negative impacts are acceptable in conventional vehicles, such negative impacts render ultrasonic sensors, in their current form, unsuitable for use in fully autonomous vehicles, as the output of a conventional ECU (which indicates presence of an object somewhere proximate to the vehicle) does not include sufficient information and update rate to allow for a fully autonomous vehicle to autonomously navigate around the object and/or avoid the object. Lastly, conventional automotive ultrasonics do not enable steering or adjustable directivity, so overall thresholds need to be reduced to be robust to different intensities of ground reflection (gravel vs, pavement).
The following is a brief summary of subject matter that is described in greater detail herein. This summary is not intended to be limiting as to scope of the claims.
Described herein are various technologies pertaining to use of ultrasonic sensors in fully autonomous vehicles (e.g., level 5 autonomous vehicles), where the ultrasonic sensors are configured to provide information that is indicative of locations and types of objects that may be within 0-15 meters of the autonomous vehicle. In an example, and in contrast to conventional architecture, numerous ultrasonic sensors are electrically coupled in a bus topology with one another and with an electronic control unit (ECU) for the ultrasonic sensors, such that the ultrasonic sensors and the ECU are included in a daisy chain. With more specificity, the ultrasonic sensors and the ECU are electrically coupled in series by way of a twisted pair wire, and the ECU transmits data to and receives data from the ultrasonic sensors over the twisted pair wire by way of a communications protocol that employs differential signaling over the twisted pair wire. In an example, the communications protocol employs low voltage differential signaling (LVDS). In a specific example, the protocol is the A2B protocol of Analog Devices, Inc. (ADI).
Such architecture enables various functionalities that are not possible with conventional ultrasonic sensor architectures. Specifically, the twisted pair wire and corresponding protocol addresses bandwidth limitations associated with protocols and cabling conventionally used in automobiles, and therefore the following functionalities are enabled: 1) high-resolution signals from numerous ultrasonic sensors can be transmitted to the ECU, and the ECU can perform cross-correlation on any combination of these high-resolution signals; 2) the ECU can cause ultrasonic sensors to emit ultrasonic signals that include information that identifies the ultrasonic sensors that emitted the ultrasonic signals; and 3) Doppler information is included in detected (reflected signals), and the ECU can compute relative velocity of an object in proximity to the autonomous vehicle based upon the Doppler information.
Moreover, configuring the ECU to perform cross-correlation computations (instead of on the ASIC of an ultrasonic sensor) means that a single ultrasonic sensor is able to constantly receive reflected signals, and is further able to receive and distinguish multiple reflected signals simultaneously, resulting in creation of more data points than is possible using conventional ultrasonic sensor architectures and reduced latency when compared to conventional ultrasonic sensor architectures, which (due to Kalman filtering or other post processing) results in improved detection and localization precision when compared to conventional ultrasonic sensor architectures.
The above summary presents a simplified summary in order to provide a basic understanding of some aspects of the systems and/or methods discussed. herein. This summary is not an extensive overview of the systems and/or methods discussed herein. It is not intended to identify key/critical elements or to delineate the scope of such systems and/or methods. Its sole purpose is to present some concepts in a simplified form as a prelude to the more detailed description that is presented later.
Various technologies pertaining to an autonomous vehicle that includes ultrasonic sensors and an electronic control unit (ECU) for the ultrasonic sensors are now described with reference to the drawings, wherein like reference numerals are used to refer to like elements throughout. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more aspects. It may be evident, however, that such aspect(s) may be practiced without these specific details. In other instances, well-known structures and devices are shown in block diagram form in order to facilitate describing one or more aspects. Further, it is to be understood that functionality that is described as being carried out by certain system components may be performed by multiple components. Similarly, for instance, a component may be configured to perform functionality that is described as being carried out by multiple components.
Moreover, the term “or” is intended to mean an inclusive “or” rather than an exclusive “or.” That is, unless specified otherwise, or clear from the context, the phrase “X employs A or B” is intended to mean any of the natural inclusive permutations. That is, the phrase “X employs A or B” is satisfied by any of the following instances: X employs A; X employs B; or X employs both A and B. In addition, the articles “a” and “an” as used in this application and the appended claims should generally be construed to mean “one or more” unless specified otherwise or clear from the context to be directed to a singular form.
Described herein is an autonomous vehicle that includes numerous ultrasonic sensors that are electrically coupled to one another in series and are further electrically coupled in series to an electronic control unit (ECU) for the ultrasonic sensors. Thus, the numerous ultrasonic sensors and the ECU are included in a daisy chain and are electrically coupled to one another by way of a twisted pair wire. The ECU is in bidirectional communication with the ultrasonic sensors over the twisted pair wire by way of a communications protocol that employs differential signaling in connection with transmitting communications between coupled elements. For example, the communications protocol employs low voltage differential signaling (LVDS). In another example, the communications protocol is the A2B protocol as set forth by Analog Devices, Inc. (ADI). In still yet another example, the communications protocol is Ethernet, although any suitable protocol that can be used in connection with communicating over twisted pair wire is contemplated. As will be described in greater detail herein, such architecture allows for ultrasonic sensors to be employed to provide near-field information (e.g., within 15 meters) for an autonomous vehicle (e.g., a level 5 fully autonomous vehicle).
Referring now to
The ECU 102 and the ultrasonic sensors 104-112 are electrically coupled in series by way of a first twisted pair wire 124, and the ECU 102 and the ultrasonic sensors 114-122 are electrically coupled in series by way of a second twisted pair wire 126. Further, the ECU 102 is in bidirectional communication with the ultrasonic sensors 104-122 over the twisted pair wires 124 and 126 by way of a suitable communications protocol that facilitates bidirectional communications between the ECU 102 and the ultrasonic sensors 104-122 by way of differential signaling. For example, the communications protocol employs LVDS. In a more specific example, the communications protocol is the A2B protocol, Ethernet, or other suitable protocol that employs differential signaling in connection with enabling bidirectional communication over twisted pair wire.
The architecture depicted in
Further, the communications protocol used in connection with bidirectional communications between the ECU 102 and the ultrasonic sensors 104-122 allow for more data to be transmitted over the twist pair wires 124 and 126 than what is possible in conventional ultrasonic sensor system architectures. Hence, rather than ASICs of the ultrasonic sensors 104-122 being configured to perform cross-correlation computations, the ECU 102 is configured to perform cross-correlation computations. Performance of cross-correlation computations by the ECU 102 is advantageous, as one or more of the ultrasonic sensors 104-122 can be continuously outputting signals to the ECU 102, and the ECU 102 can perform cross-correlation computations for multiple signals output by multiple ultrasonic sensors 104-122 substantially simultaneously, resulting in more data points than what is possible with the conventional architecture.
in addition, the ultrasonic sensors 104-122 can be configured to include, in ultrasonic signals emitted by the ultrasonic sensors 104-122, information that is indicative of identities of the ultrasonic sensors 104-122 that respectively emitted the ultrasonic signals. More specifically, the first ultrasonic sensor 104 is configured to emit a first ultrasonic signal into an environment proximate the autonomous vehicle 100, where the first ultrasonic signal has first information included therein that is indicative of an identity of the first ultrasonic sensor 104. Such information may be a phase shift, a code, etc. Similarly, the second ultrasonic sensor 106 is configured to emit a second ultrasonic signal into the environment proximate the autonomous vehicle 100, where the second ultrasonic signal has second information included therein that is indicative of an identity of the second ultrasonic sensor 106. As noted above, such information may be a phase shift, a code, etc.
The emitted ultrasonic signals may reflect off of an object 128, and the reflected signals retain the information that identifies the ultrasonic sensors that emitted the ultrasonic signals. One or more ultrasonic sensors detects the reflected signals, generates electrical signals that represent the reflected signals, and transmits the generated electrical signals to the ECU 102. The ECU 102 performs cross-correlation computations on the electrical signals, where each of the electrical signals received by the ECU 102 can be cross-correlated by the ECU 102 with several (expected) time-series signals that correspond to the ultrasonic sensors that emitted the ultrasonic signals. This process is advantageous, as the ECU 102 can precisely identify an amount of time between when an ultrasonic sensor emitted an ultrasonic signal and when the corresponding reflected signal was detected by the ultrasonic sensor (or another ultrasonic sensor). Thus, the ECU 102 can precisely identify a distance between the ultrasonic sensor that emitted the ultrasonic signal and the object 128 from which the ultrasonic signal reflected. It is noted that the ECU 102 can compute a more precise distance between the autonomous vehicle 100 and the object 128 than what is possible using conventional approaches, as the additional information in emitted ultrasonic signals causes a fairly narrow peak to be included in the cross-correlation output (much narrower than the peak included in the cross-correlation output when cross-correlating between pulses that lack the additional information referenced above).
Still further, including information that identifies an emitting ultrasonic sensor in the ultrasonic signal emitted thereby results in Doppler information being included in reflected signals, where such Doppler information either is not captured by ultrasonic sensors in conventional architectures or is discarded in conventional ultrasonic sensor system architectures. In the autonomous vehicle 100, the ECU 102 can process signals received from the ultrasonic sensors 104-122 and compute relative velocity between the autonomous vehicle 100 and an object in the environment of the autonomous vehicle 100 (e.g., the object 128).
An example of operation of the autonomous vehicle 100 is now set forth. The second ultrasonic sensor 106 is configured to emit a first ultrasonic signal with first information included therein that identifies the second ultrasonic sensor 106. Similarly, the fourth ultrasonic sensor 110 is configured to emit a second ultrasonic signal that has second information included therein that identifies the fourth ultrasonic sensor 110. For instance, the first ultrasonic signal and the second ultrasonic signal may have a similar frequency. The ECU 102 can cause the second ultrasonic sensor 106 to emit the first ultrasonic signal over a first window of time and can cause the fourth ultrasonic sensor 110 to emit the second ultrasonic signal over a second window of time (which may or may not overlap with the first window of time), The third ultrasonic sensor 108 is configured to continuously monitor the frequency at which the second ultrasonic sensor 106 and the fourth ultrasonic sensor 110 emit ultrasonic signals.
The first ultrasonic signal emitted by the second ultrasonic sensor 106 reflects from the object 128, and thus a first reflected signal is formed. The third ultrasonic sensor 108 detects the first reflected signal and generates a first electrical signal that is representative of the first reflected signal. The third ultrasonic sensor 108 transmits the first electrical signal to the ECU 102 over the first twisted pair wire 124, and the ECU 102 cross-correlates the first electrical signal with a first time-series signal that is representative of the first ultrasonic signal emitted by the second ultrasonic sensor 106. The ECU 102, based upon the performed cross-correlation, can ascertain that the first reflected signal corresponds to the second ultrasonic sensor 106 and can further ascertain a precise amount of time between when the ECU 102 instructed the second ultrasonic sensor 106 to emit the first ultrasonic signal and when the ECU 102 received the first electrical signal.
The second ultrasonic signal emitted by the fourth ultrasonic sensor 110 reflects from the object 128, resulting in a second reflected signal being detected by the third ultrasonic sensor 108. The third ultrasonic sensor 108, upon detecting the second reflected signal, generates a second electrical signal that represents the second reflected signal and transmits the second electrical signal to the ECU 102 over the first twisted pair wire 124. The ECU 102 performs a cross correlation between the second electrical signal and a second time series signal that is representative of the second ultrasonic signal emitted by the fourth ultrasonic sensor 110. Based upon such cross-correlation, the ECU 102 determines (precisely) an amount of time between when the ECU 102 instructed the fourth ultrasonic sensor 110 to emit the second ultrasonic signal and when the second electrical signal was received by the ECU 102. Therefore, the ECU 102 can relatively quickly compute distances between the object 128 and the ultrasonic sensors 106 and 110. Further, the ECU 102 can compute relative velocity between the object 128 and the autonomous vehicle 100 based upon the received electrical signals from the ultrasonic sensors 106-108.
While the example set forth above refer to the ultrasonic sensors 106 and 110 emitting ultrasonic signals having similar frequencies, it is to be understood that the ultrasonic sensors 106 and 110 may emit ultrasonic signals having different frequencies, and two different ultrasonic sensors can detect reflected signals that correspond to the ultrasonic signals emitted by the ultrasonic sensors 106 and 110 (with the two detecting ultrasonic sensors configured to detect reflected signals having the different frequencies). Further, while the autonomous vehicle 100 is illustrated as including ten ultrasonic sensors, it is to be understood that the autonomous vehicle 100 may include many more ultrasonic sensors to allow for 360-degree coverage in the near field of the autonomous vehicle 100. The near field, in an example, may be between zero and two meters from the autonomous vehicle 100. In another example, the near field may be between zero and five meters of the autonomous vehicle 100. In yet another example, the near field may be between zero and fifteen meters from the autonomous vehicle 100.
Referring now to
Similar to what has been discussed above, ultrasonic sensors in a dense array can each emit ultrasonic signals that have information included therein that identifies the ultrasonic sensors from which the ultrasonic signals were emitted, This is advantageous in that ultrasonic signals emitted by ultrasonic sensors in a dense array may reflect from multiple different points on a scattering object. Based upon cross-correlations performed by the ECU 102, the 102 can control the ultrasonic sensors in the dense array to direct beams of ultrasonic energy to particular positions in space, and/or listen in specific directions, such that location of edges of the scattering object can be identified relative to the autonomous vehicle 100. Again, this allows for more granular determination of position of a scattering object relative to the autonomous vehicle 100 as well as type of the scattering object when compared to what is possible using conventional ultrasonic sensor system architectures in human-operated vehicles.
Referring now to
Now referring to
The autonomous vehicle 100 further includes several mechanical systems. The mechanical systems include a steering system 410, a braking system 412, and a propulsion system 414. The steering system 410 is configured to control direction of motion of the autonomous vehicle 100. The braking system 412 is configured to decelerate the autonomous vehicle 100. The propulsion system 414 is configured to propel the autonomous vehicle 100 in a direction, where the propulsion system 414 may be or include an electric motor or a combustion engine.
The autonomous vehicle 100 also includes a computing system 416 that is operably coupled to the ECU 102, the lidar sensor system 406, the camera sensor system 408, and the mechanical systems 410-414. The computing system 416 includes a processor 418 and memory 420 that includes instructions that are executed by the processor 418. The processor 418 can include any suitable type of processing circuitry, including but not limited to a central processing unit (CPU), a graphics processing unit (GPU), a field programmable gate array (FPGA), a microcontroller, a microprocessor, etc. The memory 420 includes a perception system 422 and a control system 424 that are executed by the processor 418. The perception system 422 is configured to receive outputs from the ECU 102, the lidar sensor system 406, and the camera sensor system 408, and is further configured to identify and track objects in the environment of the autonomous vehicle 100 based upon such outputs. For instance, the perception system 422 may include any suitable algorithms and/or computer implemented models to identify and track objects in the environment of the autonomous vehicle 100. Accordingly, the perception system 422 may include a Bayesian Network, a Deep Neural Network (DNN), a Recurrent Neural Network (RNN), or other suitable model/algorithm.
The control system 424 receives output of the perception system 422 and controls at least one of the steering system 410, the braking system 412, or the propulsion system 414 based upon outputs of the perception system 422. In a non-limiting example, the perception system 422 can receive output of the ECU 102 that indicates that an object is proximate a forward bumper on a first side of the autonomous vehicle 100. The perception system 422 determines that the object is a static traffic cone and can output an indication to the control system 424 that a traffic cone is proximate the forward bumper of the autonomous vehicle 100 on a first side of the autonomous vehicle 100. The control system 424 controls the steering system 410 and the propulsion system 414 to cause the autonomous vehicle 100 to avoid the cone while continuing to navigate a roadway.
Moreover, some of the acts described herein may be computer-executable instructions that can be implemented by one or more processors and/or stored on a computer-readable medium or media. The computer-executable instructions can include a routine, a sub-routine, programs, a thread of execution, and/or the like. Still further, results of acts of the methodologies can be stored in a computer-readable medium, displayed on a display device, and/or the like.
Referring solely to
At 504, an ECU for the ultrasonic sensors is positioned in the autonomous vehicle. At 506, the ultrasonic sensors are electrically coupled to the ECU by way of a bus that comprises a twisted pair wire, where data is transmitted between the ECU and the ultrasonic sensors over the twisted pair wire by way of differential signaling. The methodology 500 completes at 510.
With reference now to
At 606, at the ECU, the electrical signal is cross-correlated with a time-series signal that is assigned to the ultrasonic sensor that emitted the ultrasonic signal, whereupon the ECU generates a cross-correlation output. As indicated previously, the reflected signal can include information that allows identification of which ultrasonic sensor emitted the ultrasonic signal reflected from an object. The time-series signal used by the ECU to perform the cross-correlation with the electrical signal may be representative of the ultrasonic signal emitted into the environment of the autonomous vehicle.
At 608, at the ECU, a distance between the emitting ultrasonic sensor and the object is computed based upon a peak identified in the cross-correlation output produced by the ECU. Such peak can identify an amount of time between when the ECU instructed the emitting ultrasonic sensor to emit the ultrasonic signal and when the electrical signal was received by the ECU.
At 610, by a computing system of the autonomous vehicle, a mechanical system of the autonomous vehicle is controlled based upon the distance computed at 608. For instance, any of a steering system, a braking system, or a propulsion system may be controlled based upon the distance computed at 608. The methodology 600 completes at 612.
Referring now to
At 706 a second ultrasonic signal is emitted from a second ultrasonic emitter of the autonomous vehicle, where the second ultrasonic signal includes a second code. The second code may be unique to the second ultrasonic emitter.
At 708 at an ultrasonic transducer receives a superposition of the signals transmitted by the first and second ultrasonic emitters. This signal is transmitted to the ECU.
At 710, the ECU performs a first cross-correlation (or similar) in order to separate out the data transmitted by the first emitter, and performs a second cross-correlation (or similar) in order to separate out the signal transmitted by the second emitter. The first cross-correlation (or similar) is performed with respect to the first electrical signal and a first time-series signal assigned to the first ultrasonic emitter, and the second cross-correlation (or similar) is performed with respect to the second electrical signal and a second time-series signal that is assigned to the second ultrasonic emitter. Some examples of approaches similar to cross correlation are covariance or coherence, which may also be used to isolate energy from emitters.
At 712, in the ECU after having isolated energy originating from each emitter at each receiver, energy from one emitter can be considered. Delays or other spatial filtering can then be applied to the signals at each receiver (for that single emitter) to determine how much energy came from each direction. Thus the object's location and shape may be calculated. Ground reflections may be rejected. The methodology 700 completes at 714.
Referring now to
The computing device 800 additionally includes a data store 808 that is accessible by the processor 802 by way of the system bus 806. The data store 808 may include executable instructions, cross-correlation values, etc. The computing device 800 also includes an input interface 810 that allows external devices to communicate with the computing device 800. For instance, the input interface 810 may be used to receive instructions from an external computer device, from a user, etc. The computing device 800 also includes an output interface 812 that interfaces the computing device 800 with one or more external devices. For example, the computing device 800 may display text, images, etc. by way of the output interface 812.
Additionally, while illustrated as a single system, it is to be understood that the computing device 800 may be a distributed system. Thus, for instance, several devices may be in communication by way of a network connection and may collectively perform tasks described as being performed by the computing device 800.
Various functions described herein can be implemented in hardware, software, or any combination thereof If implemented in software, the functions can be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media includes computer-readable storage media. A computer-readable storage media can be any available storage media that can be accessed by a computer. By way of example, and not limitation, such computer-readable storage media can comprise RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Disk and disc, as used herein, include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk, and Blu-ray disc (BD), where disks usually reproduce data magnetically and discs usually reproduce data optically with lasers. Further, a propagated signal is not included within the scope of computer-readable storage media. Computer-readable media also includes communication media including any medium that facilitates transfer of a computer program from one place to another. A connection, for instance, can be a communication medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio and microwave are included in the definition of communication medium. Combinations of the above should also be included within the scope of computer-readable media.
Alternatively, or in addition, the functionally described herein can be performed, at least in part, by one or more hardware logic components, For example, and without limitation, illustrative types of hardware logic components that can be used include Field-programmable Gate Arrays (FPGAs), Program-specific Integrated Circuits (ASICs), Program-specific Standard Products (ASSPs), System-on-a-chip systems (SOCs), Complex Programmable Logic Devices (CPLDs), etc.
What has been described above includes examples of one or more embodiments. It is, of course, not possible to describe every conceivable modification and alteration of the above devices or methodologies for purposes of describing the aforementioned aspects, but one of ordinary skill in the art can recognize that many further modifications and permutations of various aspects are possible. Accordingly, the described aspects are intended to embrace all such alterations, modifications, and variations that fall within the spirit and scope of the appended claims. Furthermore, to the extent that the term “includes” is used in either the details description or the claims, such term is intended to be inclusive in a manner similar to the term “comprising” as “comprising” is interpreted when employed as a transitional word in a claim.
