Patent Application
20250171029
The present invention relates to driving behavior monitoring system. More particularly the invention relates to driving behavior of a driver driving a vehicle by calibrating, correcting and analyzing the data obtained from the sensors during the drive.
In modern days, road safety is becoming an increasing concern for everyone with the rising number of accidents. Driving behavior and road accidents are directly related to each other. Around 1.35 million people get killed in road accidents around the world each year. It is possible to avoid most by following certain safety measures. However, these measures are often overlooked by drivers in today's hectic world. Existing methods of monitoring Driving Behavior of a driver driving a vehicle involve use of standalone Telematics Data processing system, driving policy, training etc. to overcome accident risk and safe driving practices.
The methods involve using a telematics device in the Vehicle is expensive and involves additional logistics of fitting the device in the vehicle. Further, the conventional Smartphone based Driving Behavior monitoring solutions are not accurate.
In light of the foregoing discussion, there is a need to develop economic and accurate solution for safe driving practices and reduce accident risk, which offers high accuracy of driving behavior monitoring without the use of any dedicated telematics device in the vehicle. Also, the solution is required which can also be used to rank drivers based on safe driving score and reward them for safe driving, more specifically as a part of Fleet Management System or Usage Based Insurance program to ensure safe driving practices and reduce accident risk.
The object of the invention is to provide a system for determining behavior of a driver driving a vehicle.
Another object of the invention is to provide the system that can determine the driving behavior using the data from mobile communication device in the vehicle such as Driver's Smartphone with a high degree of accuracy and robustness. Optionally, the mobile communication device is communicably paired with low-cost Inertial Measurement Unit (IMU) sensor, to further improve the accuracy of the driving behavior with minimal cost addition and inconvenience to the driver.
Another object of the invention is to provide a system for calibrating the accelerometer data of the vehicle to prepare it for determining the driving behavior of the driver driving the vehicle, wherein the vehicle could be four-wheeler and above vehicles (such as cars, trucks and buses) as well as two-wheeler vehicles (such as motorcycles and bicycles).
A further object of the invention is to provide the system which combines the different rash and unsafe driving behavior to rate the drivers for safe driving.
A system for efficiently determining a driving behaviour of a driver driving a vehicle is provided. The system comprises a 3-axis accelerometer generating acceleration data, direction sensor generating direction data, orientation sensor generating orientation data. The system's processing unit processes the data through a sequence of processing steps to generate a calibrated accelerometer data where the 3 axes of accelerometer are aligned with the direction of the vehicle. System uses the calibrated accelerometer data along with the direction data to determine various inappropriate driving behaviour instances using a sequence of processing steps. System is adapted to generate Driving Deficiency Alerts, including but not limited to Over-speeding, Hard acceleration, Sudden Braking, Fast Cornering, Quick Lane Changing, Distracted Driving, Dangerous Driving over bumps and Crash Detection. System is adapted to determine driving behaviour for a vehicle that could be four-wheeler and above vehicles (such as cars, trucks and buses) as well as two-wheeler vehicles (such as motorcycles and bicycles) which can rotate laterally. System is adapted to determine the driving behavior using the data from mobile communication device in the vehicle such as Driver's Smartphone. Optionally, the mobile communication device is communicably paired with low-cost Inertial Measurement Unit (IMU) sensor, to further improve the accuracy of the driving behaviour.
The novel features and characteristics of the disclosure are set forth in the description. The disclosure itself, however, as well as a preferred mode of use, further objectives and advantages thereof, will best be understood by reference to the following description of an illustrative embodiment when read in conjunction with the accompanying drawings. One or more embodiments are now described, by way of example only, with reference to the accompanying drawings wherein like reference numerals represent like elements and in which:
The figures depict embodiments of the disclosure for purposes of illustration only. One skilled in the art will readily recognize from the following description that alternative embodiments of the assemblies, structures and methods illustrated herein may be employed without departing from the principles of the disclosure described herein.
For the purpose of promoting an understanding of the principles of the invention, reference will now be made to the embodiment illustrated in the figures and specific language will be used to describe them. It will nevertheless be understood that no limitation of the scope of the invention is thereby intended. Such alterations and further modifications in the illustrated system, and such further applications of the principles of the invention as would normally occur to those skilled in the art are to be construed as being within the scope of the present invention.
It will be understood by those skilled in the art that the foregoing general description and the following detailed description are exemplary and explanatory of the invention and are not intended to be restrictive thereof.
The terms “comprises”, “comprising”, or any other variations thereof, are intended to cover a non-exclusive inclusion, such that a process or method that comprises a list of steps does not include only those steps but may include other steps not expressly listed or inherent to such a process or method. Similarly, one or more sub-systems or elements or structures or components preceded by “comprises . . . a” does not, without more constraints, preclude the existence of other, sub-systems, elements, structures, components, additional sub-systems, additional elements, additional structures or additional components. Appearances of the phrase “in an embodiment”, “in another embodiment” and similar language throughout this specification may, but not necessarily do, all refer to the same embodiment.
Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention belongs. The system, methods, and examples provided herein are only illustrative and not intended to be limiting. The system, methods, and examples provided herein are only illustrative and not intended to be limiting.
Embodiments of the present invention will be described below in detail with reference to the accompanying figures.
The present disclosure relates to monitoring of driving Behavior of a driver driving a vehicle using data from Driver Smartphone. Optionally, a low-cost Inertial Measurement Unit (IMU), placed in the vehicle which is combined with Driver Smartphone data. The solution also generates composite and normalized Safe Driving score by combining the different driving behavior parameters.
The system (1) also includes a processing unit (8) which processes the accelerometer data (3), the direction data (5) and the orientation data (7) and re-calibrates the accelerometer data (3), and further generates a calibrated accelerometer data (9). The processing unit (8) recalibrate the accelerometer data (3) to align its 3 axes with the 3 axes of the orientation of vehicle so that the calibrated accelerometer data (9) is useful for determining the inappropriate driving behavior events.
The driving behavior events which are considered for analysis and determination may include over-speeding, hard acceleration, sudden braking, fast cornering, quick lane change, fatigue driving, crash detection and distracted driving etc. These events are required to be captured and analyzed as part of driver behavior monitoring.
These data (3, 5, 7) may be received from the sensors (2, 4, 6) inside a mobile communication device being carried by the driver or from the sensors (2, 4, 6) inside a separate IMU placed in the vehicle, as well as some combination of both. The mobile communication device may have the 3-axis accelerometer (2), direction sensor (4) and the orientation sensor (6). The direction sensor (4) can be a GPS/GNSS sensor that can be used to obtain direction and speed data. The orientation sensor can be Gyroscope sensor that generates angular velocity data. The IMU unit may have the 3-axis accelerometer (2) and the orientation sensor (6) to provide accelerometer and orientation data respectively. The IMU unit can be battery powered. The IMU unit may use Bluetooth low energy connectivity to send the sensor data to the driver's mobile communication device, which can also be his/her smartphone.
It is to be noted that the mobile communication device is generally picked up by the driver multiple times during commutation, which leads to change in orientation of smartphone. This means that the system (1) of calibration of accelerometer data takes into account such changes in orientation so that its 3 axes are aligned correctly with the orientation of vehicle after each such change in orientation of smartphone. In case, if the data (3, 5, 7) is received from the fixed IMU unit in the vehicle, it requires calibration only once during initial placement, or maximum when it is displaced for any repairs. Hence, the embodiment where IMU unit's data is used, the computational requirements of the system (1) reduce while computing driver behavior. Also, such an embodiment ensures that the data is captured from a designated vehicle in which the IMU is placed, thus improving accuracy in the determination of driver's behavior.
Further, whenever a driver starts driving the vehicle, the processing unit (8) may automatically start receiving the data from the sensors (2, 4, 6) for further determination of the driving behavior. An advantage of using IMU unit is that even if the GPS/GNSS sensor of the Smartphone is switched off (to conserve battery of the Smartphone), the sensor data received from the IMU unit enables the system (1) to identify certain key driving behavior events (Alerts) such as Hard-Acceleration, Sudden Braking, Fast Cornering and Quick Lane Change.
After calibrating the accelerometer data (3), the processing unit (8) compares the corrected accelerometer data (9) in a XY plane of the vehicle at each time instance with a fifth predefined threshold (not shown in the figure). If a magnitude of the corrected accelerometer data (9) is above the fifth predefined threshold, the processing unit (8) determines such time instance as an inappropriate driving behaviour instance (30).
Discussing
In this embodiment, the processing unit (8) uses the acceleration data (3), direction data (5) and the orientation data (7) to generate angular data relating to the three angles of rotation (namely, phi, theta and psi corresponding to Euler Z-Y-Z transformation) for each time instant by which Z, Y and Z axes of accelerometer needs to be rotated in sequence to align the accelerometer with the three axes of the orientation of the vehicle at each instant and generate the calibrated accelerometer data (9).
For still better performance of the calibration of the accelerometer data (3), only the orientation data (7) which represents the data from the sequential instances where the difference in the orientation data (7) within a first predefined threshold (12) is considered for calibration. For this, the processing unit (8) receives the orientation data (7) for a predefined period time, and thereafter matches a difference of sequential orientation data (7) for various instances in the predefined time period with the first predefined threshold (12). Where for sequential instances or sequential stretch of instances, the difference in orientation data (7) is within the first predefined threshold (12), those sections of instances are labelled as stable orientation data (10). And, where the difference of the orientation data (7) for sequential instances is outside the first predefined threshold (12), they are labelled as unstable orientation data (11). The processing unit (8) uses the stable orientation data (10) along with the accelerometer data (3) to re-calibrate the accelerometer data (3) and to generate the calibrated accelerometer data (9).
For still better performance of the calibration of the accelerometer data (3), the processing unit (8) processes the stable orientation data (10) in the following way. The processing unit (8) processes the stable orientation data (10) and generates an angular data (18). The angular data (18) includes a phi-pre angle of orientation (19) of the 3-axis accelerometer (2) with respect to a frame of reference of the vehicle, which is representative of first rotation angle around Z-axis of the frame of reference, and a theta angle of orientation (20) of the 3-axis accelerometer (2) with respect to the frame of reference of the vehicle, which is representative of tilt around a Y-axis of the frame of reference. The processing unit (8) compares a change in angular data (21) with a second predefined threshold (22). If the change in angular data (21) is within the second predefined threshold (22), then the processing unit (8) splits the angular data (18) into smaller sub-sections periods and calculates a moving average of the angular data (23). If the change in angular data (21) is within the second predefined threshold (22), then the processing unit (8) generates a median value of the angular data (24). The processing unit (8) processes the moving average of the angular data (23) or the median value of the angular data (24) along with the accelerometer data (2) and re-calibrates the accelerometer data (2) and generates the intermediate calibrated accelerometer data in the plane of the vehicle (9-1).
For still better performance of the calibration of the accelerometer data (3), the processing unit (8) compares the intermediate calibrated accelerometer data in the plane of the vehicle (9-1) with a third predefined threshold (25), and simultaneously compares the direction data (5) for corresponding period with a fourth predefined threshold (26). If the corrected accelerometer (9) is above the third predefined threshold (25) and the direction data (5) is below the fourth predefined threshold (26), the processing unit (8) determines a psi angle of orientation (27) of the 3-axis accelerometer (2) with respect to the frame of reference of the vehicle for the stable period. The psi angle of orientation (27) of the 3-axis accelerometer (2) is representative of second rotation around Z-axis of the frame of reference. The processing unit (8) further determines the most frequent or most likelihood psi angle (28). Thereafter, the processing unit processes the angular data (18) along with the most likelihood psi angle (28) and the accelerometer data (3) and re-calibrates the accelerometer data (3) and generates the final calibrated accelerometer data (9). In one embodiment of the present invention, the psi value may be selected from multiple psi values using a Kernel Density Estimation method for probability density distribution and then finding the maximum likelihood psi.
In case, where the accelerometer data (2) is captured from smartphone of driver, the orientation of smartphone in the vehicle can undergo a change anytime the smartphone is picked up or moved by the driver during the drive. This implies that the accelerometer data (2) needs to be re-calibrated so that its 3 axes are aligned with the orientation of vehicle. GPS/GNSS sensor and Gyroscope sensor in Smartphone provide the direction data (5) and orientation data (7) respectively. The above methodology mentioned is applied to such situation as elaborated herein below:
In case of fixed IMU unit, the accelerometer data from the IMU unit needs to be corrected so that its 3 axes are aligned with the direction of vehicle. However, this correction needs to be performed only once and is simpler as compared to Smartphone sensor data correction method. The following are the steps for this one-time correction:
(X, Y, Z).
Discussing
The processing unit (8) compares the corrected accelerometer data (9) in the horizontal (XY) plane of the vehicle at each time instance with a fifth predefined threshold (29), and if a magnitude of the corrected accelerometer data (9) is above the fifth predefined threshold (29), the processing unit (8) is adapted to determine such time instance as an inappropriate driving behaviour instance (30).
The processing unit (8) determines a first pattern (31) in the corrected accelerometer data (9) in direction of vehicle movement for successive time intervals for various inappropriate driving behaviour instances (30). If the first pattern (31) matches a first predefined pattern (32), to determine the time interval as hard acceleration or deacceleration interval (33). This helps in determining sudden braking and hard acceleration events.
The processing unit (8) determines a second pattern (34) in the corrected accelerometer data (9) perpendicular to direction of vehicle movement for successive time intervals for various inappropriate driving behaviour instances (30). If the second pattern (34) is beyond a second predefined pattern (35), to determine the time interval as fast cornering or quick lane changing interval (36). This helps in determining the Fast Cornering and Quick Lane Change type of Events The pattern matching technique used can be based on Dynamic Time Warp (DTW) based pattern matching techniques to classify each time instant of corrected Accelerometer data into relevant categories. This technique uses a sliding window to match the corrected Accelerometer pattern with the reference DTW pattern. Sliding window is a set of corrected accelerometer readings around a center point. For each point, the sliding window is shifted by certain number of points and DTW pattern matched. Consecutive sliding windows that match a DTW pattern pertaining to a Driving Alert are combined so as to label one instant of continuous Driving Alert as a single instance. This enables high accuracy of identifying the Driving Alert and at the same time avoiding any double counting of Driving Alerts.
It is to be noted that for determining hard acceleration or deacceleration interval (33) and fast cornering or quick lane change interval (36), the processing unit (8) does not use location data from GPS/GNSS sensor. Therefore, in the embodiment of where the fixed IMU unit's sensors are used and the accelerometer data (2) correction is required only once (not needed for each trip), the processing un it (8) can generate Driving Alerts even if the GPS/GNSS sensor of the mobile communication device is switched off for battery saving purpose.
For sections of the drive, for which accelerometer data (2) is not available or corrected accelerometer data (9) cannot be determined, GPS sensor data can used for determining the hard acceleration or deacceleration intervals (33), and the fast cornering or quick lane changing intervals (36). Vehicle Speed is calculated from GPS sensor data by differentiating the distance between two consecutive GPS readings by the time difference between the two consecutive readings. This Vehicle Speed at each instant is used to further calculate the Acceleration and Deacceleration of the moving vehicle.
For Fast Cornering detection or Quick Lane change intervals (36), the direction data (5) of GPS is used to calculate an angular velocity and speed data obtained from GPS is used to define the tangential velocity. These two are multiplied to calculate the centripetal acceleration which is the indicative of the cornering force applied. If the cornering force is greater than a certain threshold, it's considered as Fast Cornering or Quick Lane changing.
The processing unit (8) processes the hard acceleration or deacceleration intervals (33), and the fast cornering or quick lane change intervals (36), and if intervals (33, 36) of same type are lying within a predefined period (37), the processing unit (8) combines the intervals of the same type and generates a driving deficiency alert (38) of that type for the combined interval.
Further, the processing unit (8) receives a speed change information (39) and a direction change information (40) for various inappropriate driving behaviour instances (30), and correlates the driving deficiency alert (38) with the speed change information (39) and a direction change information (40), and determines a correctness and severity (41) of the driving deficiency alert (38). In one embodiment, the processing unit (8) may use either of the speed change information (39) and the direction change information (40) for determining the correctness and severity (41) of the driving deficiency alert (38).
The processing unit (8) processes the corrected accelerometer data (9) and the direction data (5) for various inappropriate driving behaviour instances (30) and determines a speed of the vehicle (42) during those instances. In one embodiment, the processing unit (8) uses filter such as Kalman or Complementary filter to determine to determine speed of the vehicle (42). The processing unit (8) compares the speed of the vehicle (42) with a first predefined threshold speed (43), and if the speed of the vehicle (42) is above first predefined threshold speed (43), the processing unit (8) determines the instance as over speeding instance (44). In one embodiment, the first predefined threshold speed (43) is dynamically established based on the location of the vehicle provided by GPS sensor and location-based road speed limit information available from external sources.
The processing unit (8) processes the orientation data (7) for unstable orientation data periods (11) and determines a orientation data pattern (45). Further, the processing unit (8) compares the orientation data pattern (45) with a predefined orientation pattern (46), and if a match occurs, it determines the time interval as a phone usage time interval (47). Thereafter, the processing unit (8) correlates the phone usage time interval (47) with a second pre-defined speed threshold (43-1) and determines the phone usage time interval as drive distraction time interval (48).
For bump detection, the processing unit (8) processes the corrected accelerometer data (9) in the direction of ground for small sub-sections of stable orientation data periods (10), and determines a data pattern (49) in the corrected accelerometer data in the direction of ground. Further, the processing unit (8) compares the data pattern (49) with a fifth predefined data pattern (50), and if a match occurs, it determines the interval as a bump detection interval (51). Thereafter, the processing unit (8) correlates the bump detection interval (51) with a third pre-defined speed threshold (43-2) and determines the bump detection interval as dangerous driving over the bump instance (52).
For determining a crash detection instance (55), the processing unit (8) processes the speed (42) of the vehicle at for small intervals and determines an abrupt drop in speed (53). The processing unit (8) further processes the corrected accelerometer data (9) around same time intervals to determine an acceleration pattern (54). Thereafter, the processing unit (8) correlates the abrupt change in speed (53) with the acceleration pattern (54) and matches the acceleration pattern (54) with a sixth pre-defined pattern set (61) to determine the crash detection instance (55). In one embodiment, the processing unit (8) may specifically match the acceleration pattern in the horizontal (XY) plane of the vehicle with a sixth pre-defined pattern set (61) to determine the direction with respect to the vehicle from which the crash occurred.
In one embodiment, this crash detection instance (55) can be transmitted in real time to a Cloud Server using wireless connectivity provided by the mobile communication device of the user. Also, processed data around the time of the Crash Alert such as the approaching speed, direction, location and driving behavior is sent to the Cloud Server. Depending upon the use, this information in the Cloud Server is used to notify Emergency response service, to provide roadside assistance. It may also be used for accident reconstruction and auto insurance claims management.
It is to be noted that the processing unit (8) is further enabled to process all the inappropriate driving behaviour instances to generate a composite score or a ranking to rate driver's driving skills.
The processing unit (8) compares the corrected accelerometer data (9) in a three-dimensional frame of the vehicle at each time instance with a ninth predefined threshold (56). If the magnitude of the corrected accelerometer data (9) is above the ninth predefined threshold (56), the processing unit (8) determines such time instance as an inappropriate driving behaviour instance (30).
The processing unit (8) determines a third pattern (57) in the corrected accelerometer data (9) in direction of vehicle movement for successive time intervals. If the third pattern (57) matches a third predefined pattern (58), the processing unit (8) determines the time interval as hard acceleration or deacceleration interval (33). Further, the processing unit (8) determines a fourth pattern (59) in the corrected accelerometer data (9) in the plane perpendicular to direction of vehicle movement for successive time intervals. If the fourth pattern (59) is beyond a fourth predefined pattern (60), the processing unit (8) determine the time interval as fast cornering interval (36). The pattern matching technique used can be based on Dynamic Time Warp based pattern matching techniques to classify each time instant of corrected Accelerometer data into relevant categories.
The system allows flexibility to add newer safe driving factors to be considered. Also, the safe driving factors calculation are vectorized (parallelized) over a large number of raw sensor data points in order to improve the performance of safe driving factor computation. Varying thresholds are applied depending upon the factor being considered. The thresholds depend upon the type of vehicle and the geographic location where the vehicle is driven. In general, additional criteria can be added to modify the thresholds considered for each type of factor.
Further the processing unit can be configured to assign a score for each individual factor and calculating a composite safe driving score. For each of the factors affecting safety (such as Over-speeding, Hard acceleration, Sudden Braking, Fast Cornering, Distracted Driving, Dangerous Driving over bumps) a score can be assigned out of 100. The logic for computing the score related to each factor varies based on the factor. Generally, a score of 100 can be assigned for a factor for a given driver/vehicle in case there is no abnormal or rash driving event found in the Drive with respect to that variable. In case rash driving event is found in the Drive with respect to a variable, the number, duration and severity of each event is rated and combined to assign an overall percentage rating for that drive on that parameter.
Scores of individual factors are combined to generate an overall safe driving score of a drive or a trip. Scores across trips can be combined by the processing unit by using weighted average of distance travelled in each drive to give an overall safe driving score. The trend of safe driving score across trips and time is used to improve driving skills and encourage safe driving behavior from the driver.
In some embodiments, techniques for normalizing the safe driving scores for different driving conditions can be used. Different driving conditions are referred to be such as highway vs non-highway, weight of vehicle, driving during day or night time, prevalent weather condition during the drive based on the location of the drive. These normalizations enable safe driving score to reflect the risk more accurately. For example, Over-speeding under rainy weather condition or night time is riskier than Over-speeding under fair weather and day time. These normalizations can be compared across drivers in order to rate and rank drivers in a group for the purpose of their safe driving ability, gamification and training.
The system 1 can utilize sensor data received from the Driver's Smartphone itself where the sensor data is already aggregated. This reduces the cost and time of transferring the voluminous data to another computing resource. This also addresses the privacy concern of users by avoiding sending any private information such as GPS location to the server. At the same time, system can use the GPS location information locally to generate the Driving Score which can be sent to the server. User can still see the Analyzed Driving Behavior Events including each trip details and route inside a user interface of an application running on the Smartphone of the driver. In another embodiment, the captured sensor data is transferred in real time to the Vehicle's Head Unit using connectivity technologies such as Bluetooth or Wifi and the sensor data is processed in the processing resources of the Vehicle's Head Unit. In yet another embodiment of the Invention, the captured sensor data is transferred in real time or on batch basis to a Cloud based server using connectivity technologies such as Cellular/Wifi and the sensor data processing is executed on the processing resources of the Cloud based Server. In another embodiment of the Invention, the sensor data processing can switch between execution on Driver's Smartphone and Cloud based execution depending upon the availability of resources and connectivity between Smartphone and Cloud.
Analyzed Driving Behavior Events and Safe Driving Scoring trends from the Drivers trips are displayed to the Driver using a smartphone based mobile application running on their smartphone. The Application enables Driver to view his/her Driving behavior for each trip as well as aggregated score across trips. Driver can analyze which component of his/her driving needs improvement by drilling down to the detailed alert view of each trip. Also, the composite Driving score is used to compare and indicate ranking of a Driver within a Group. Similar User Interface can be provided using other applications such as Web Based Dashboard Application. The analyzed data can be sent to a cloud server and further shared with third parties such as Insurance and Automotive companies with Driver's consent.
Irrespective of capturing data from either mobile communication device (smartphone) or the IMU unit, the steps related to Driving Behavior (Alerts) Generation and Safe Driving Score Generation remains common. This increases flexibility of the system and reducing the memory footprint of the overall system to enable it being deployed on memory constrained mobile devices.
In another embodiment, the sensor data is processed during an ongoing drive and audio or visual alerts are generated on Driver's Smartphone during the ongoing drive to warn the driver when driving behavior is unsafe.
It is to be noted that for personal owned vehicles, the system can be used for giving guidance to vehicle owner or driver to drive in a safe manner. For fleet vehicles that are owned by an organization, the system can be used as part of fleet Management System to ensure safe driving practices and reduce accident risk. For Motor Insurance companies, the system can be used to rate the accident risk of each driver or vehicle that is insured by the Insurance company. The System can be used to create underwriting models as part of Usage Based Insurance policies wherein the premium of the policy is varied based on the Driving behavior of the Driver or Drivers driving a certain vehicle. Vehicle Manufacturers can use this system to understand how drivers drive the vehicles in the field under various driving conditions and hence design vehicles with adequate safety measures, controls, and warning systems. Invention is also useful for various other players in the automotive ecosystem who are interested in monitoring driving behavior and providing solutions based on driving behavior.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202141040988 | Jan 2022 | IN | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/IB2023/050551 | 1/23/2023 | WO |
