Patent Grant
11780452
This application claims the benefit of priority from Chinese Patent Application No. 202011134946.9, filed on Oct. 21, 2020. The content of the aforementioned application, including any intervening amendments thereto, is incorporated herein by reference in its entirety.
The present application relates to fault diagnoses for systems, particularly to data analysis of sensors of intelligent vehicles, and more particularly to a method and a system for a fault diagnosis of an intelligent vehicle.
An intelligent vehicle involves high-tech technologies, such as sensor technologies, computer technologies, communication technologies, information processing technologies, and control technologies. The development of the intelligent vehicle is promising in the field of transportation. However, the safety of intelligent vehicles still cannot be guaranteed, and there is no systematic theory to ensure the safety of the intelligent vehicles. Thus, it is of great significance to provide a method for fault diagnoses of the intelligent vehicles to ensure the safety of the intelligent vehicles in practical use.
Existing fault diagnosis methods for intelligent vehicle are generally based on fault diagnosis methods applied in traditional industrial processes and mainly include model-based methods, signal-based methods, and data-driven methods. However, since the intelligent vehicle has a complex system, possible faults of various subsystems of the intelligent vehicle cannot be effectively diagnosed using a single fault diagnosis method. In addition, it is difficult to accurately position faults of subsystems of the intelligent vehicle through data of sensors carried by the intelligent vehicle.
In order to overcome shortcomings in the prior art, the present disclosure aims to provide a method for a fault diagnosis of an intelligent vehicle.
Provided is a method for a fault diagnosis of an intelligent vehicle, comprising:
1) establishing a model of a system of the intelligent vehicle; acquiring system operation data of the intelligent vehicle in a normal running state; training and optimizing the model using the system operation data of the intelligent vehicle in a normal running state;
2) collecting the system operation data of the intelligent vehicle in a running state in real time; de-noising sensor data of the operation data, and performing feature extraction and screening for a fatal sensor fault to reconstruct the system operation data; if a screened feature is the fatal sensor fault, outputting sensor unit information corresponding to the screened feature; inputting the reconstructed system operation data into the trained model to output system state data of the intelligent vehicle in the running state; comparing the system state data with a set threshold; and if the system state data exceeds the set threshold, determining that an actuator corresponding to the system state data has a fault, thereby completing the fault diagnosis of the intelligent vehicle.
In some embodiments, the system operation data of the intelligent vehicle in the normal running state is obtained through a storage medium of a control system of the intelligent vehicle; and a useful field in the system operation data is extracted, and data cleaning and data transformation are carried out for the useful field.
In some embodiments, during the data cleaning, incomplete records are directly removed, and duplicate records are merged into one piece; the data transformation is to carry out mathematical transformation for a directly extracted field to obtain required verification information.
In some embodiments, before training and optimizing the model, the sensor data of the system operation data of the intelligent vehicle in the normal running state is de-noised, and feature extraction and screening are performed for the fatal sensor fault of the system operation data of the intelligent vehicle in the normal running state.
In some embodiments, the feature extraction is performed for the system operation data of the intelligent vehicle in the normal running state at different scales using discrete wavelet transform (DWT); a signal from extracted features are reconstructed using an approximation coefficient and a detail coefficient to obtain de-noised system operation data; a signal of the system operation data of the intelligent vehicle in the normal running state is reconstructed using a threshold method.
In some embodiments, the feature extraction is performed for the system operation data of the intelligent vehicle in the normal running state at different scales using DWT, and a sliding window method is adopted for the DWT.
In some embodiments, the model comprises a plurality of subsystem models, and each of the subsystem models corresponds to an independent actuator.
In some embodiments, the fatal sensor fault is determined according to formulas (6) and (7):
thBM<abs(Σi=k-(W-1)kdi) (6),
thJ>Σi=k-W
wherein, thBM and thJ are two set thresholds and are set to be 3 and 1×10−6 respectively; W and Wj are sizes of two sliding windows and are set to be 100 and 50 respectively; di represents a three-level detail coefficient obtained through DWT at moment i; x(k) represents the sensor data at moment k.
The present disclosure further provides a system for a fault diagnosis of an intelligent vehicle, comprising:
a state monitoring module for a sensor;
an abnormality detection module for the intelligent vehicle; and
a fault test module for an actuator;
wherein the state monitoring module is configured to acquire system operation data of the intelligent vehicle in a running state and transmit the acquired system operation data to the abnormality detection module and the fault test module;
the fault test module is configured to:
determine the system operation data through screening; if a screened feature of the system operation data is a fatal sensor fault, output sensor unit information corresponding to the screened feature; and
the abnormality detection module is configured to determine the system operation data; if system state data corresponding to the system operation data of the intelligent vehicle in the running state exceeds a set threshold, determine that an actuator corresponding to the system state data has a fault.
Compared to the prior art, the present invention has the following advantages.
A model of a system of the intelligent vehicle is established, and system operation data of the intelligent vehicle in a normal running state is acquired. The system operation data of the intelligent vehicle in a normal running state is adopted to train and optimize the model. The system operation data of the intelligent vehicle in a running state is collected in real time. Sensor data of the system operation data is de-noised, and feature extraction and screening are performed for a fatal sensor fault, to reconstruct the system operation data. If a screened feature is the fatal sensor fault, sensor unit information corresponding to the screened feature is outputted. The reconstructed system operation data is inputted into the trained model to output system state data of the intelligent vehicle in the running state. The system state data is compared with a set threshold. If the system state data exceeds the set threshold, an actuator corresponding to the system state data is determined to have a fault. Sensor data of the collected system operation data is de-noised, and features of a fatal sensor fault are extracted and screened, so that fatal abnormality of the sensor data and state abnormality of the vehicle can be effectively detected. The fault of the intelligent vehicle can be positioned according to the system operation data.
The DWT can be used for extracting features of the sensor fault and de-noising the sensor data, and the sliding window method can greatly improve the accuracy of the data.
The model includes a plurality of subsystem models, and each of the subsystem models corresponds to an independent actuator. In this way, the sensor data de-noising and the feature extraction and screening are performed for each independent subsystem, so that the subsystem having faults can be quickly screened out.
The state monitoring module is configured to acquire system operation data of the intelligent vehicle in a running state and transmit the acquired system operation data to the abnormality detection module and the fault test module. The fault test module is configured to screen the system operation data; and if a screened feature of the system operation data is a fatal sensor fault, output sensor unit information corresponding to the screened feature; and the abnormality detection module is configured to determine the system operation data; and if system state data corresponding to the system operation data of the intelligent vehicle in the running state exceeds a set threshold, determine that an actuator corresponding to the system state data has a fault. In this way, faults of the intelligent vehicles can be quickly diagnosed.
The present disclosure will be further described below with reference to the accompany drawings.
As shown in
1) A model of a system of the intelligent vehicle is established. System operation data of the intelligent vehicle in a normal running state is obtained. The model is trained and optimized using the system operation data of the intelligent vehicle in the normal running state.
Specifically, the system operation data of the intelligent vehicle in the normal running state is obtained through a storage medium of a control system of the intelligent vehicle. A useful field in the system operation data is extracted, and data cleaning and data transformation are carried out for the useful field.
Specifically, during the data cleaning, incomplete records are directly removed, and duplicate records are merged into one piece of record. The data transformation is to carry out the mathematical transformation for the directly extracted field to obtain required verification information.
Before training and optimizing the model, sensor data of the system operation data of the intelligent vehicle in the normal running state is de-noised, and feature extraction and screening are performed for the fatal sensor fault of the system operation data of the intelligent vehicle in the normal running state, so as to eliminate optimization training defects of the model caused by the fatal sensor fault.
Specifically, feature extraction is performed for the acquired system operation data of the intelligent vehicle in the normal running state at different scales using DWT. Signals from the extracted features are reconstructed using approximation coefficients and detail coefficients to obtain the de-noised system operation data. The signal of the obtained system operation data of the intelligent vehicle in the normal running state is reconstructed using a threshold method.
2) The system operation data of the intelligent vehicle in the normal running state is collected in real time. The sensor data is de-noised, and feature extraction and screening are performed for the fatal sensor fault. If a screened feature is a fatal sensor fault, a sensor unit corresponding to the screened feature needs to be replaced. The reconstructed system operation data is inputted into the trained and optimized model to output state data of the system of the intelligent vehicle in the running state. The operation data of the intelligent vehicle in the running state is compared with a set threshold; if the operation data exceeds the set threshold, the system corresponding to the operation data is determined to have a fault and needs to be repaired. The model is trained and optimized using the system operation data of the intelligent vehicle in the normal running state, and the approximation is performed for the input and the corresponding output, so that the set threshold is obtained.
The model includes multiple subsystem models, and each subsystem model corresponds to an independent actuator of the intelligent vehicle.
Embodiment
In this embodiment, collected sensor data refers to GNSS information, and the actuator data refers to data of a hydraulic brake of the vehicle. The method for fault diagnoses of intelligent vehicles is described in detail as follows.
The collection of the operation data of the GNSS is described as follows. The data acquisition devices used include an integrated navigation system Micro Electro Mechanical System (MEMS) installed in the vehicle, a set of GNSS antennas for receiving satellite signals, an industrial computer for recording logs of the integrated navigation system GNSS, and a power supply device for powering the above devices. As shown in
The collection of the operation data of the hydraulic brake is described as follows. The operation data of the hydraulic brake includes an oil pressure of the vehicle, a vehicle speed and the time when the vehicle begins to brake and the time when the vehicle is fully stopped, and the data acquisition device used herein is a vehicle-mounted CAN bus analyzer. The CAN bus of the intelligent vehicle is connected to the vehicle-mounted CAN bus analyzer which is connected to the computer via a USB cable, so that relevant data of the vehicle can be obtained from the CAN bus.
Step 1) System operation data is extracted and pretreated.
The system operation data in this embodiment includes operation data of the GNSS and operation data of the hydraulic brake. Useful fields are extracted from the operation data of the GNSS and the operation data of the hydraulic brake, respectively, and data cleaning and data conversion are performed for the useful fields. The useful fields are used for training the model.
For the operation data of GNSS, the useful fields extracted include a protocol header, sampling moment, latitude, longitude, a heading angle, an eastward speed and a northward speed. For the collected operation data of the hydraulic brake, the extracted useful fields include an oil pressure of the vehicle, a vehicle speed and the time when the vehicle begins to brake and the time when the vehicle is fully stopped (i.e., the vehicle speed is 0). The extracted fields still have many problems including incomplete, missing, and duplicate records, and cannot be directly inputted to the model for training. The incomplete records are directly removed. The duplicate records are merged, and only one piece is kept. The input data required for subsequent training and verification of the model includes: the speed, an angular velocity and the oil pressure, the vehicle speed when the vehicle begins to brake and an average acceleration during braking of the vehicle in an unmanned state. Attribute cannot be obtained directly from the extracted fields, and is acquired through a certain transformation based on the extracted fields.
The transformation method includes the following steps.
(1) The eastward speed and the northward speed are transformed to obtain a speed value. The eastward speed is denoted as νe and the northward speed is denoted as νn. The speed value is:
ν=√{square root over (νe2+νn2)} (1).
The angular velocity is obtained by calculating a difference quotient of the heading angles at two consecutive sampling moments. Considering that the heading angles of 360° and 0° actually represent the same heading angle, but the numerical value shows an abrupt change, the phase unwrap function of the MATLAB R2016a platform is used to avoid such abrupt change between two consecutive time frames, and then the difference quotient of heading angles is calculated. The angular velocity of the intelligent vehicle is calculated as follows:
where ωk represents the angular velocity at moment k; tk and tk-1 represent two consecutive sampling moments, φk and φk-1 represent the heading angles at moments k and k−1, respectively.
3) The average acceleration a during braking is calculated from the vehicle speedy, and the moment t0 when the vehicle begins to brake, and the moment t1 when the vehicle is fully stopped:
α=νs/(t1−t0) (3).
Step 2) Sensor data is de-noised, and feature extraction and screening is performed for the fatal sensor fault.
Using the multi-resolution analysis of the DWT, the feature extraction of the operation data is performed at different scales. The approximation coefficients (also known as low-frequency coefficients) and the detail coefficients (also known as high-frequency coefficients) are adopted to reconstruct the signal, so as to obtain the de-noised operation data. The threshold method is used to reconstruct the signal of the system operation data of the intelligent vehicle in the normal running state, that is, the detail coefficients greater than the threshold are set to zero, and then the inverse wavelet transform is adopted to obtain the reconstructed signal. The threshold is selected based on an empirical formula:
th=std(x)·√{square root over (2 log2(N))} (4),
where, th is the determined threshold, std(x) is a standard deviation of a signal x, and N is a signal length. In practice, in order to facilitate the adjustment of the threshold, an adjustment coefficient α (usually around 1) is often added to the formula (4) to obtain:
th=α·std(x)·√{square root over (2 log2(N))} (5).
The fatal sensor fault means that when the fault occurs, the sensor itself or related electronic devices (such as power supply) are in an abnormal state, for example, internal components are damaged and the power supply fails. Once the fatal sensor fault occurs, the sensor cannot work normally, and accordingly the intelligent vehicle cannot work since it cannot effectively perceive the environment. It is necessary to replace the corresponding sensor according to the fatal sensor fault. Common fatal faults include: bias (a fixed deviation from the actual value), jam (outputs of the sensor are a constant value), miss (lack of available data of the sensor in a period of time), and as shown in
thBM<abs(Σi=k-(W-1)kdi) (6),
thJ>Σi=k-W
where, thBM and thJ are two set thresholds and are set to be 3 and 1×10−6 respectively. W and Wj are sizes of two sliding windows and are set to be 100 and 50 respectively. di represents a three-level detail coefficient obtained through DWT at time i (the DWT decomposes the data at three levels). x(k) represents the sensor data at time k. If formula (6) is true, it means that a “bias” fault or a “miss” fault has been detected; if formula (7) is true, it means that a “jam” fault has been detected.
Step 3: An abnormality detection module of the vehicle is constructed, that is, a vehicle system model is constructed and trained as follows.
Based on an extreme learning framework, the speed and the angular velocity of the vehicle in normal running state obtained in step 1 are used to train an autoencoder. The trained autoencoder performs pattern comparison on new samples to identify data that clearly bias from the data in the normal running state.
Step 3.1) The autoencoder is trained offline. The autoencoder, which is a neural network in structure, is the vehicle system model. As shown in
hj=g(Σi=1dwjixi+bj);j=1,2, . . . ,L (8),
{tilde over (x)}k=Σj=1Lβjkhj;k=1,2, . . . ,d (9),
ε=∥X−{tilde over (X)}∥ (10).
The feature of the autoencoder is that the input of the network is consistent with the target output (or label). The input data is reconstructed by minimizing the reconstruction error defined by equation (10), so that normal data can be learned. When training the vehicle system model, only health data (that is, the operation data of the system of the vehicle in the normal running state) is used as inputs and target outputs. After the vehicle system model is trained, if input data is similar to the training data (i.e., normal data), the vehicle system model reconstructs the input data well; otherwise, there is a large error between the actual input data and the reconstructed input data. According to the error between the actual input data and the reconstructed input data, whether the samples are abnormal is determined.
A hidden layer parameter β* of the extreme learning framework is directly calculated based on the least square method:
β*=H†X (11),
where, H=[hT(x1), . . . , hT(xd)]T, H†=(HTH)−1HT is a Moore-Penrose generalized inverse of the matrix H. The output y of the vehicle system model is expressed as:
y=hβ*=hH†X (12).
Step 3.2) The state of the vehicle is detected. For the intelligent vehicle, the input vector X is described by the speed ν and the angular velocity ω which represent the state of the vehicle at the time of sampling (the speed ν represents the state of the vehicle in the longitudinal direction, and the angular velocity ω represents the state of the vehicle in the transverse direction). Whether the sample point is abnormal (different from the normal data pattern) can be judged by calculating the 2-norm of the output y of the vehicle system model, that is, for a new sample Xnew=[νnew, ωnew]T, a decision variable can be formalized as:
D=sign(δ−∥y−Xnew∥) (13),
where, δ is a threshold value set according to the actual situation, where δ takes 5 herein; sign represents a sign function; D is the decision variable; if D is −1, the current state of the vehicle is considered to be significantly deviated from the normal state.
Step 4) Approximation is carried out for the system of the actuator. In this embodiment, the system of the hydraulic brake is taken an example to illustrate the approximation of the system of the actuator. A fully connected neural network with two hidden layers is built. Each hidden layer has 16 nodes, and the number of nodes in the input layer and the output layer is the same as the corresponding input and output dimensions. The oil pressure and the speed when the vehicle begins to brake obtained in step 1 are taken as the input data, and the average acceleration during braking is taken as the output data to train the fully connected neural network. The activation function adopts a rectified linear unit (ReLU), and the weight parameters and the bias parameters are initialized randomly at the beginning, and are continuously modified as the training process progresses. The training process involves back propagation (BP), and the BP process involves adaptive momentum (Adam); when an error between mean squares of the output of the vehicle system model and the actual value is less than 1e-3, the training is stopped.
Step 5) As shown in
After the fuzzy rules are determined, the membership function should be selected according to the actual situation, and may cause a greater impact on the performance of the fuzzy PID controller. Generally, if there is a higher requirement for the response sensitivity of the system, a membership function with a sharper shape may be selected; otherwise, a membership function with a smoother shape may be selected. In this embodiment, the triangular membership function is selected, and the inference rule is the Mamdani fuzzy inference rule. Finally, the three parameters Kp, Ki, Kd of the PID controller are obtained through the center of gravity defuzzification method.
Step 6) The fault of the actuator is positioned. For each actuator in a healthy state, a neural network is used to approximate its input and output relationship. After training, the neural network will be used as the reference system of the actual actuator unit. The controller acts on the actual actuator and the reference system at the same time. If the actual system fails, there is a large bias between the actual output and the output of the system in the normal running state. In order to allow a controlled object to achieve a desired performance, the output of the controller will have a larger change from the output in the normal running state. That is, once a fault occurs, there is a large difference between the output of the controller for the actual actuator and the output of the controller for the reference system. Taking the brake system as an example, as shown in
ud=∥ũ−ũref∥ (14),
where, ũ and ũref are standardized u and uref, respectively; ∥·∥ is a certain norm and takes the 2-norm herein. If ud is greater than the threshold uth, the actuator is abnormal, and the system will alarm and record the abnormal information of this specific actuator in the log. After the fault is located, the vehicle slows down and pulls to the side to stop for subsequent processing.
According to the steps 4 and 5, for the fuzzy PID controller, three components of the output u=[Kp, Ki, Kd]T are normalized to the interval [0,1] to obtain ũ. Similarly, the output uref of the reference system is normalized to obtain ũref. The 2-norm is calculated after the subtraction of the respective components of ũ and ũref. The 2-norm is compared with the set threshold uth. In this embodiment, the set threshold is 2. If it is greater than the threshold, the fault may occur in the system of the actual brake. Then, the fault is recorded in the log, and an alarm is issued.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202011134946.9 | Oct 2020 | CN | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 8356207 | Hosek | Jan 2013 | B2 |
| 20080004840 | Pattipatti | Jan 2008 | A1 |
| 20190340392 | Khorrami | Nov 2019 | A1 |
| Number | Date | Country |
|---|---|---|
| 103455026 | Dec 2013 | CN |
| 110501169 | Nov 2019 | CN |
| 110658807 | Jan 2020 | CN |
| Number | Date | Country | |
|---|---|---|---|
| 20220118987 A1 | Apr 2022 | US |
