Patent Application
20240132076
The present specification relates to road sound detection when a vehicle interacts with an anomaly.
As a vehicle is driven along a road, if the vehicle hits an anomaly (e.g., a speed bump or a pot hole), the vehicle may be damaged by the interaction with the anomaly. The damage caused to the vehicle may require the vehicle to be serviced by a vehicle technician to repair the damage. However, diagnosing the damage to the vehicle may be difficult, which may increase the time and/or cost of repairing the vehicle. As such, more efficient ways of diagnosing vehicle damage may be desirable.
One way to diagnose damage to a vehicle caused by interaction with an anomaly is to listen to a sound that occurs when the vehicle interacts with the anomaly and shortly thereafter. In particular, a vehicle technician may be able to identify the damage that occurred to the vehicle based on the sound caused by the vehicle interacting with the anomaly. However, this is typically not possible because the sound that occurs when a vehicle hits an anomaly cannot typically be reproduced later for a vehicle technician to listen to. Accordingly, an improved method of road sound detection may be desirable.
In an embodiment, a vehicle system of a vehicle may include a controller. The controller may be programmed to receive sensor data from one or more vehicle sensors as the vehicle drives along a road, and determine whether the vehicle is approaching a road anomaly based on the sensor data. Upon determination that the vehicle is approaching the road anomaly, the controller may begin recording audio and continue recording audio until after the vehicle interacts with the road anomaly.
In another embodiment, a vehicle system may include a controller. The controller may be programmed to continually record audio, store a predetermined amount of the most recently recorded audio in a buffer, receive sensor data from one or more vehicle sensors, and determine whether the vehicle has interacted with an anomaly based on the sensor data. Upon determination that the vehicle has interacted with an anomaly, the controller may extract a portion of the audio from the buffer.
In another embodiment, a method may include receiving sensor data from one or more vehicle sensors of a vehicle as the vehicle drives along a road, and determining whether the vehicle is approaching a road anomaly based on the sensor data. Upon determination that the vehicle is approaching the road anomaly, the method may include beginning to record audio and continuing to record audio until after the vehicle interacts with the road anomaly.
The embodiments set forth in the drawings are illustrative and exemplary in nature and not intended to limit the disclosure. The following detailed description of the illustrative embodiments can be understood when read in conjunction with the following drawings, where like structure is indicated with like reference numerals and in which:
The embodiments disclosed herein include a method and system for road sound detection. As a vehicle drives along a road, a vehicle system may detect when the vehicle is approaching a road anomaly. When the vehicle system detects that the vehicle is about to interact with a road anomaly, the vehicle may begin recording audio so as to record audio of the vehicle interacting with the anomaly. The recorded audio may be stored by the vehicle system. If the interaction with the anomaly causes any damage to the vehicle, the recorded audio of the interaction between the vehicle and the anomaly can later be played back for a vehicle technician in order to assist the technician in diagnosing the damage caused to the vehicle.
Turning now to the figures,
Each of the one or more processors 202 may be any device capable of executing machine readable and executable instructions. Accordingly, each of the one or more processors 202 may be a controller, an integrated circuit, a microchip, a computer, or any other computing device. The one or more processors 202 are coupled to a communication path 204 that provides signal interconnectivity between various modules of the vehicle system 200. Accordingly, the communication path 204 may communicatively couple any number of processors 202 with one another, and allow the modules coupled to the communication path 204 to operate in a distributed computing environment. Specifically, each of the modules may operate as a node that may send and/or receive data. As used herein, the term “communicatively coupled” means that coupled components are capable of exchanging data signals with one another such as, for example, electrical signals via conductive medium, electromagnetic signals via air, optical signals via optical waveguides, and the like.
Accordingly, the communication path 204 may be formed from any medium that is capable of transmitting a signal such as, for example, conductive wires, conductive traces, optical waveguides, or the like. In some embodiments, the communication path 204 may facilitate the transmission of wireless signals, such as Wi-Fi, Bluetooth®, Near Field Communication (NFC) and the like. Moreover, the communication path 204 may be formed from a combination of mediums capable of transmitting signals. In one embodiment, the communication path 204 comprises a combination of conductive traces, conductive wires, connectors, and buses that cooperate to permit the transmission of electrical data signals to components such as processors, memories, sensors, input devices, output devices, and communication devices. Accordingly, the communication path 204 may comprise a vehicle bus, such as for example a LIN bus, a CAN bus, a VAN bus, and the like. Additionally, it is noted that the term “signal” means a waveform (e.g., electrical, optical, magnetic, mechanical or electromagnetic), such as DC, AC, sinusoidal-wave, triangular-wave, square-wave, vibration, and the like, capable of traveling through a medium.
The vehicle system 200 includes one or more memory modules 206 coupled to the communication path 204. The one or more memory modules 206 may comprise RAM, ROM, flash memories, hard drives, or any device capable of storing machine readable and executable instructions such that the machine readable and executable instructions can be accessed by the one or more processors 202. The machine readable and executable instructions may comprise logic or algorithm(s) written in any programming language of any generation (e.g., 1GL, 2GL, 3GL, 4GL, or 5GL) such as, for example, machine language that may be directly executed by the processor, or assembly language, object-oriented programming (OOP), scripting languages, microcode, etc., that may be compiled or assembled into machine readable and executable instructions and stored on the one or more memory modules 206. Alternatively, the machine readable and executable instructions may be written in a hardware description language (HDL), such as logic implemented via either a field-programmable gate array (FPGA) configuration or an application-specific integrated circuit (ASIC), or their equivalents. Accordingly, the methods described herein may be implemented in any conventional computer programming language, as pre-programmed hardware elements, or as a combination of hardware and software components.
Referring still to
The vehicle system 200 comprises one or more vehicle sensors 210. Each of the one or more vehicle sensors 210 is coupled to the communication path 204 and communicatively coupled to the one or more processors 202. The one or more vehicle sensors 210 may include, but are not limited to, LiDAR sensors, RADAR sensors, optical sensors (e.g., cameras, laser sensors), proximity sensors, location sensors (e.g., GPS modules), and the like. In embodiments, the vehicle sensors 210 may monitor the surroundings of the vehicle and may detect road anomalies, as disclosed herein. In some examples, the vehicle sensors 210 may also monitor the behavior of other vehicles, as disclosed herein.
Still referring to
Still referring to
Still referring to
Now referring to
The sensor data reception module 300 may receive sensor data from the one or more vehicle sensors 210. In particular, the sensor data reception module 300 may receive sensor data that may indicate that the vehicle 102 is approaching an anomaly. In one example, the sensor data reception module 300 may receive image data captured by a forward-facing camera. However, in other examples, the sensor data reception module 300 may receive other sensor data.
The anomaly detection module 302 may analyze the sensor data received by the sensor data reception module 300 to determine whether the vehicle 102 is approaching an anomaly. In one example, the anomaly detection module 302 may perform image analysis on one or more images captured by front-facing cameras. That is, the anomaly detection module 302 may perform image analysis of a portion of the road being approached by the vehicle 102 to determine if an anomaly is present. In some examples, a machine learning algorithm may be used to identify road anomalies based on image data (e.g., the “You Only Look Once” algorithm). The anomaly detection module 302 may be an image segmentation model, an object detection model, an object classification model, or any other model for recognizing an anomaly. The anomaly detection module 302 may include a machine learning model including, but not limited to, supervised learning models such as neural networks, decision trees, linear regression, and support vector machines, unsupervised learning models such as Hidden Markov models, k-means, hierarchical clustering, and Gaussian mixture models, and reinforcement learning models such as temporal difference, deep adversarial networks, and Q-learning. In other examples, other types of image processing techniques may be performed to identify road anomalies. In some examples, the anomaly detection module 302 may analyze data from other types of sensors (e.g., LiDAR, proximity sensors, and the like) to determine if the vehicle 102 is approaching an anomaly.
In another example, the anomaly detection module 302 may analyze the sensor data received by the sensor data reception module 300 to identify unusual behavior of other vehicles, which may indicate the presence of a road anomaly. For instance, in the example of
If an anomaly is detected, the anomaly detection module 302 may determine a location of the anomaly (e.g., a distance between the vehicle 102 and the detected anomaly) based on the received sensor data. In some examples, the anomaly detection module 302 may determine an estimated time when the vehicle 102 will reach a detected anomaly based on the distance between the vehicle 102 and the detected anomaly and a speed of the vehicle 102.
The audio recording module 304 may cause the microphone 214 to begin recording audio. In particular, the audio recording module 304 may cause the microphone 214 to record audio when the vehicle 102 interacts with the anomaly, as disclosed herein.
In one example, the audio recording module 304 may cause the microphone 214 to begin recording audio when the anomaly detection module 302 determines that a distance between the vehicle 102 and a detected anomaly is less than a predetermined threshold distance (e.g., less than 20 feet). In another example, the audio recording module 304 may cause the microphone 214 to begin recording audio when the anomaly detection module 302 determines that a time when the vehicle 102 is expected to reach the anomaly is less than a predetermined threshold time in the future (e.g., less than 5 seconds).
In some examples, the audio recording module 304 may begin recording audio upon being triggered by an occupant of the vehicle 102. For example, the interior of the vehicle 102 may have a button that can be pressed to begin recording audio (e.g., on the vehicle dashboard). As such, if an occupant of the vehicle 102 notices that the vehicle 102 is approaching an anomaly, the occupant may press the button, which may trigger the audio recording module 304 to cause the microphone 214 to begin recording audio.
In some examples, the audio recording module 304 may cause the microphone 214 to begin recording audio after receiving a notification indicating that the vehicle 102 is approaching an anomaly. For example, the vehicle 102 may receive map data from an edge server indicating that a particular location is known to contain a road anomaly. As such, the audio recording module 304 may cause the microphone 214 to begin recording audio when the vehicle 102 approaches the location of the known anomaly. In other examples, the audio recording module 304 may cause the microphone 214 to begin recording audio when the vehicle 102 or the vehicle 104 receives an indication from another vehicle that an anomaly has been detected at a particular location. For instance, in the example of
Once the audio recording module 304 causes the microphone 214 to begin recording audio, the microphone 214 may continue recording audio until after the vehicle 102 passes the detected anomaly, such that the microphone 214 records the entire interaction between the vehicle 102 and the anomaly. In some examples, the microphone 214 may continue recording audio for a time period after the vehicle 102 passes the anomaly. This may allow a technician to not only hear the sound when the vehicle 102 hit the anomaly, but also to hear the sound of the vehicle shortly after the anomaly was hit, which may give a stronger indication as to the nature of the damage to the vehicle.
In one example, once the microphone 214 starts recording audio, the microphone 214 may continue recording audio for a predetermined time period (e.g., 20 seconds). In another example, once the microphone 214 starts recording audio, the microphone 214 may continue recording audio until the vehicle 102 travels a predetermined distance (e.g., 40 feet). In another example, once the microphone 214 starts recording audio, the microphone 214 may continue recording audio until the vehicle 102 has traveled a certain distance past the anomaly (e.g., 20 feet). In another example, once the microphone 214 starts recording audio, the microphone 214 may continue recording audio until the vehicle 102 has traveled past the anomaly for a certain period of time (e.g., 5 seconds). In embodiments, once the audio recording module 304 causes the microphone 214 to start recording audio, the audio recording module 304 may cause the microphone 214 to stop recording audio once the appropriate time has passed, or the vehicle has traveled the appropriate distance, as discussed above.
Referring still to
In these examples, the microphone 214 may continually record audio, and the recorded audio may be stored in the data storage component 216. In particular, the data storage component 216 may store an audio buffer of a certain length. That is, as the microphone 214 records audio, the most recently recorded audio may be stored in a buffer (e.g., the most recently recorded 5 minutes of audio). As new audio is recorded, the oldest previously recorded audio may be deleted, such that the buffer continually stores the most recently recorded audio. If the vehicle 102 does not encounter any anomalies, then the audio in the buffer can be deleted, thereby limiting the amount of storage needed to maintain the audio buffer. In some examples, the length of the audio buffer may be based on a speed of the vehicle 102.
However, if the anomaly detection module 302 detects that the vehicle 102 has hit an anomaly, the audio buffer extraction module 306 may extract a portion of the recorded audio to be permanently stored. In particular, the audio buffer extraction module 306 may extract a portion of the audio based on a timing of when the anomaly detection module 302 detects that the vehicle 102 interacted with the anomaly. For example, when the anomaly detection module 302 detects that the vehicle 102 has interacted with an anomaly, the audio buffer extraction module 306 may extract audio beginning at a predetermined time before the vehicle 102 interacted with the anomaly and ending at a predetermined time after the vehicle 102 interacted with the anomaly (e.g., 10 seconds before and 10 seconds after the vehicle 102 interacted with the anomaly). As such, audio of the interaction between the vehicle 102 and the anomaly may be stored even if the vehicle system 200 is not able to determine that the vehicle 102 is approaching the anomaly ahead of time. In some embodiments, the audio buffer extraction module 306 may send the extracted audio to an edge server or a cloud server via V2X communication such that the extracted audio is stored in the edge server or the cloud server.
Referring still to
Referring still to
At step 402, the anomaly detection module 302 determines whether an anomaly is detected along the trajectory of the vehicle 102. In one example, the anomaly detection module 302 may detect an anomaly based on data received by the sensor data reception module 300. In another example, the anomaly detection module 302 may detect an anomaly based on data received by the data reception module 310 (e.g., a notification from a remote computing device or another vehicle about the location of an anomaly). In addition to detecting the presence of an anomaly, the anomaly detection module 302 may determine the location of the anomaly (e.g., the distance between the vehicle 102 and the anomaly).
If the anomaly detection module 302 determines that an anomaly has not been detected (No at step 402), then control returns to step 400 and the sensor data reception module 300 continues to receive additional sensor data. However, if the anomaly detection module 302 determines that an anomaly has been detected (Yes at step 402), then control passes to step 404.
At step 404, after an anomaly has been detected, the audio recording module 304 causes the microphone 214 to begin recording audio. In some examples, the audio recording module 304 causes the microphone 214 to begin recording audio immediately after the anomaly detection module 302 detects an anomaly. In some examples, the audio recording module 304 causes the microphone 214 to begin recording audio when the distance between the vehicle 102 and the detected anomaly is less than a predetermined threshold distance. In some examples, the audio recording module 304 causes the microphone 214 to begin recording audio when an expected time before the vehicle 102 reaches the detected anomaly is less than a predetermined threshold time.
At step 406, the audio recording module 304 causes the microphone 214 to stop recording audio after the vehicle 102 has passed the anomaly. In some examples, the audio recording module 304 may cause the microphone 214 to stop recording audio after the vehicle 102 has traveled beyond the anomaly by a predetermined threshold distance. In some examples, the audio recording module 304 may cause the microphone 214 to stop recording audio when the vehicle 102 has traveled past the anomaly for greater than a predetermined threshold time. Once the microphone 214 stops recording audio, the recorded audio may be stored in the data storage component 216. Accordingly, if any damage to the vehicle 102 occurs due to the interaction with the anomaly, the recorded audio may be played back for a vehicle technician in order to assist the technician in diagnosing the damage that was done to the vehicle. In some examples, the data transmission module 308 may transmit the determined location of the anomaly to a remote computing device and/or another vehicle in order to assist other vehicles in identifying the anomaly.
At step 502, the anomaly detection module 302 determines whether an anomaly has been detected based on the sensor data received by the sensor data reception module 300. In particular, the anomaly detection module 302 may determine whether the vehicle 102 has hit or otherwise interacted with an anomaly. For example, the anomaly detection module 302 may determine that the vehicle 102 has hit an anomaly based on a sudden change to a speed, acceleration, or trajectory of the vehicle 102.
If the anomaly detection module 302 determines that an anomaly has not been detected (No at step 502), then control returns to step 500 and the sensor data reception module 300 continues to receive additional sensor data. Alternatively, the anomaly detection module 302 determines that an anomaly has been detected (Yes at step 502), then control passes to step 504.
At step 504, after an anomaly has been detected, the audio buffer extraction module 306 extracts a portion of the audio stored in the buffer and stores it in the data storage component 216. In some examples, the audio buffer extraction module 306 may extract audio from the buffer recorded over a certain length of time. In some examples, the audio buffer extraction module 306 may extract audio from the buffer recorded while the vehicle 102 has traveled a certain distance. As such, an audio recording of an interaction between the vehicle 102 and the anomaly may be recorded even if the anomaly detection module 302 does not detect the anomaly until after the vehicle 102 reaches the anomaly.
It should now be understood that embodiments described herein are directed to road sound detection. A vehicle system of a vehicle may detect that the vehicle is approaching a road anomaly and may cause a microphone to begin recording audio as the vehicle approaches the anomaly. The audio may continue to be recorded as the vehicle reaches and interacts with the anomaly, and for a short time after reaching the anomaly. As such, the microphone may record the sound of the vehicle interacting with the anomaly. If the anomaly causes any damage to the vehicle, the recorded audio may later be played back to a vehicle technician. The audio may assist the technician in diagnosing the specific damage that was done to the vehicle and determining what repairs are needed.
It is noted that the terms “substantially” and “about” may be utilized herein to represent the inherent degree of uncertainty that may be attributed to any quantitative comparison, value, measurement, or other representation. These terms are also utilized herein to represent the degree by which a quantitative representation may vary from a stated reference without resulting in a change in the basic function of the subject matter at issue.
While particular embodiments have been illustrated and described herein, it should be understood that various other changes and modifications may be made without departing from the spirit and scope of the claimed subject matter. Moreover, although various aspects of the claimed subject matter have been described herein, such aspects need not be utilized in combination. It is therefore intended that the appended claims cover all such changes and modifications that are within the scope of the claimed subject matter.
