Patent Application
20230271618
This invention relates generally to the vehicular activity monitoring field, and more specifically to a new and useful system and method for detecting lateral driving behavior in the vehicular activity monitoring field.
As mobile user devices continue to evolve and the quality of onboard sensors continues to increase, the use cases for which these sensors can be used continually expands. One such use case is in determining kinematic and trajectory information associated with vehicles in which a mobile user device is present, such as a personal mobile user device of a driver of a vehicle. If available, this information can be useful in numerous applications, such as in the early detection of vehicular accidents, an assessment of a driver's driving behavior, and other applications. Collecting and interpreting the data from mobile user devices has numerous challenges, however, which is compounded by the numerous types of risky driving behavior in which drivers can engage.
Thus, there is a need in the vehicle activity monitoring field to create an improved and useful system and method for detecting lateral driving behavior.
The following description of the preferred embodiments of the invention is not intended to limit the invention to these preferred embodiments, but rather to enable any person skilled in the art to make and use this invention.
As shown in
As shown in
The method 200 can function to detect and assess the (lateral) driving behavior associated with a user, which can be used in numerous ways, examples of which include: determining a driving score for a user (e.g., for use by an insurance company); detecting that a driver is in danger and/or is putting other drivers in danger (e.g., to trigger an emergency response action, etc.), understanding risky roads and/or locations and/or scenarios (e.g., based on aggregated data from multiple drivers), and/or any other use cases.
Lateral deviations and/or lateral accelerations as referenced herein are preferably substantially orthogonal relative to gravity (i.e., orthogonal to a gravity vector; independent of an orientation of the mobile device) and a longitudinal axis of the vehicle (e.g., occurring perpendicular to the longitudinal axis). As an example, lateral accelerations can be aligned with and/or parallel to the rear axle of the vehicle (e.g., where the front axle is a steering axle; where lateral accelerations and heading changes may result from steering adjustments of the front axle). As a second example, lateral accelerations may be substantially perpendicular to a centerline of the road (e.g., where a centerline may be approximated as straight; perpendicular to the tangent of a curving road, etc.). Additionally or alternatively, lateral deviations can reference: angular deviations, steering/heading adjustments, and/or any other suitable deviations associated with vehicle lane changes/swerves. However, the lateral deviations and/or accelerations may be otherwise suitably used and/or referenced herein.
In a first set of variants (e.g., an example is shown in
In a second set of variants (e.g., an example is shown in
In one variant of the second set of variants, the set of lateral movement events can include a set of lane change events, wherein detecting each event of the set includes detecting a motion signature of a lane change behavior. In an example of the first variant, the motion signature is a pair of opposite sign lateral accelerations (e.g., in opposing directions, such as left and right; driver-side and passenger-side; etc.). In a second example, the motion signature is a pair of opposite sign lateral accelerations, wherein the pair of opposite sign lateral accelerations occur consecutively and within a frequency bandwidth (e.g., a bounded range of frequencies), wherein high frequency vibrations above the frequency bandwidth are filtered out of the sensor dataset, wherein the motion signature comprises higher-frequency peaks, within the frequency bandwidth, which occurring along a lower-frequency acceleration signal (e.g., below the frequency bandwidth).
In one variant, at least one lateral movement event is detected as a higher-frequency peak within a lower-frequency signal. In an example, the lower-frequency signal corresponds to the lateral acceleration (and/or angular velocity/acceleration) associated with a roadway curvature, wherein the higher-frequency peak corresponds to a lane change maneuver. In a second example, the lateral movement event is a pair of lateral movements (e.g., corresponding to a lane change) which are detected as a pair of higher-frequency peaks (e.g., opposite sign lateral accelerations) within a lower-frequency signal.
The system and method for detecting lateral driving behavior can confer several benefits over current systems and methods.
First, variants of the system and/or method can confer the benefit of enabling accurate, robust, and/or near-real time detections that a driver has participated in risky driving behavior, and optionally triggering a suitable action in response. This can, include, for instance dynamically detecting that a driver is participating in (and/or has a tendency to participate in) aggressively changing lanes and/or swerving, and taking this behavior into account in determining and/or maintaining one or more scores associated with the driver and/or his or her environment (e.g., particular stretch of road associated with a high incidence of risky lateral driving behavior). In examples, a score associated with a user's driving behavior can be calculated, maintained, and/or used by a car insurance provider of the user (e.g., to determine/adjust the user's monthly premium). In other examples, aggregated data from a set of users can be used to assess the riskiness of particular driving locations (e.g., road segments), such that drivers can be aware of locations at which drivers tend to drive in the riskiest manner. In other examples, determining that a driver is behaving in a risky manner can be used to dynamically alert the driver (e.g., to notify him or her to stop driving in a risky manner) or other users (e.g., other drivers nearby the risky driver, police, etc.). Alternatively, the detection of the lateral driving event can be otherwise used. In some examples, for instance, rather than detecting and/or analyzing lateral events which indicate risky driving, lateral events can be generally detected and/or analyzed for other purposes (e.g., for understanding traffic patterns, detecting locations which serve as bottlenecks, etc.).
Second, variants of the system and/or method can additionally or alternatively confer the benefit of differentiating between types of lateral events, such as distinguishing an aggressive lane change lateral event from a swerving driving behavior. Additionally or alternatively, variations of the system and/or method can confer the benefit of filtering out and/or not detecting similar events which are not of interest. For instance, in some examples (e.g., for use cases in which only risky lateral events are desired to be detected), the system (e.g., algorithms, models, satisfaction criteria, etc.) and/or method can be configured to lateral events which correspond to risky driving behavior while ignoring (e.g., filtering out, identifying and preventing further processing of, etc.) lateral events which are allowed/permitted and/or not associated with higher driving risk (e.g., turning, merging onto/off of a freeway, etc.). This can, in turn, function to prioritize computational resources, calculate accurate scores for assessing driver behavior, and/or confer any other benefits.
Third, variants of the system and/or method can additionally or alternatively confer the benefit of robustly assessing the driving behavior associated with a user based on sensor data collected at a mobile device (e.g., smartphone) in a variety of conditions without explicitly knowing said conditions, such as: when the mobile device is stationary relative to the vehicle, when the mobile device is moving relative to the vehicle, and/or any other conditions.
Fourth, variants of the system and/or method can additionally or alternatively improve the technical fields of at least vehicle telematics, inter-vehicle networked communication, computational modeling of vehicle-related events, and vehicle-related event determination with mobile computing device data. The technology can take advantage of the non-generic sensor data and/or be used with supplemental data (e.g., maps; vehicle sensor data, weather data, traffic data, environmental data, biometric sensor data, etc.) to better improve the understanding of correlations between such data and traffic-related events and/or responses to such events, leading to an increased understanding of variables affecting user behavior while driving and/or riding in a vehicle (e.g., bus, train, etc.) and/or traffic behavior at the scale of a population of users driving vehicles.
Fifth, the technology can provide technical solutions necessarily rooted in computer technology (e.g., automatic data collection via a mobile computing platform, utilizing computational algorithms/models to characterize and/or determine traffic-related events from non-generic sensor datasets collected at mobile computing devices, updating the computational models based on event determination and/or communication accuracy, etc.) to overcome issues specifically arising with computer technology (e.g., issues surrounding how to leverage location data collected by a mobile computing device to accurately determine vehicle-related events, etc.).
Sixth, the technology can leverage specialized computing devices (e.g., computing devices with GPS location capabilities, computing devices with motion sensor functionality, wireless network infrastructure nodes capable of performing edge computation, etc.) to collect specialized datasets for characterizing behavior associated with a driver.
Additionally or alternatively, the system and method can confer any other benefit.
The system 100, an example of which is shown in
The system 100 preferably functions to collect and/or process data which is used in any or all processes of the method 200. Additionally or alternatively, the system 100 can function to calculate driver behavior metrics, share driver behavior metrics with 3rd party entities (e.g., insurance providers, emergency responders, etc.), and/or can perform any other functions.
The system can include or be used with a mobile device 120 which functions to collect sensor data and/or any other data. Examples of the mobile device include a mobile phone (e.g., smartphone), user device, tablet, laptop, watch, wearable devices, or any other suitable mobile device. The mobile device can include power storage (e.g., a battery), processing systems (e.g., CPU, GPU, memory, etc.), sensors, wireless communication systems (e.g., a WiFi transceiver(s), Bluetooth transceiver(s), cellular transceiver(s), etc.), or any other suitable components. The set of sensors of the mobile device can include movement sensors, which can include: location sensors (e.g., GPS, GNSS, etc.), inertial sensors (e.g., IMU, accelerometer, gyroscope, magnetometer, etc.), motion sensors, force sensors, orientation sensors, altimeters, and/or any other suitable movement sensors; user-facing sensors, which can include: cameras, user input mechanisms (e.g., buttons, touch sensors, etc.), and/or any other suitable user-facing sensors; and/or any other suitable sensors.
Additionally, the set of sensors can include any or all the sensors onboard the mobile device, such as but not limited to, any or all of: inertial sensors and/or motion sensors (e.g., accelerometer, gyroscope, magnetometer, orientation sensor, etc.), which can function to detect any or all of: mobile device movement, mobile device orientation, vehicle movement, vehicle orientation, arrangement of the mobile device within the vehicle (e.g., dash-mounted; in-hand; etc.), and/or any other suitable information; proximity sensors (e.g., optical sensors, capacitive sensors, etc.), which can function to detect and/or classify a user's handling of a mobile device; location sensors (e.g., GPS); any or all of the sensors described above; any or all of the sensors described below; and/or any other suitable sensors. In preferred variations, the set of sensors includes any or all of: a GPS sensor, an accelerometer, a gyroscope, a magnetometer, and a gravity sensor. Additionally or alternatively, the sensor system can include any other suitable sensors.
In a preferred set of variations, the set of sensors includes a set of inertial sensors onboard the mobile device. The set of inertial sensors preferably includes one or more accelerometers (e.g., tri-axial accelerometer), which function to measure the specific forces experienced by the phone (e.g., with respect to an inertial reference frame). In specific examples, the accelerometer includes a tri-axial accelerometer arranged at the center of gravity of the mobile device which records the specific forces experienced by the phone with respect to an inertial reference frame. Additionally or alternatively, the set of inertial sensors can include: one or more gyroscopes; one or more magnetometers; and/or any other inertial sensors. The set of sensors further preferably includes one or more location sensors, such as a GPS receiver onboard the mobile device.
The sensors (e.g., location sensor; GPS/GNSS) of the mobile device can generate data: periodically (e.g., greater than 10 Hz, 10 Hz, 1 Hz, 0.1 Hz, less than 0.1 Hz, any range bounded by the aforementioned values, etc.), aperiodically, in response to a geofence trigger (e.g., every 10 meters, every 100 meters, etc.) and/or other trigger (e.g., minimum speed threshold, etc.), and/or with any other suitable timing/frequency.
However, the system can include and/or be used with any other suitable mobile device(s); and/or can receive location data from any other suitable devices, sources, and/or endpoint(s).
In variants, the system can optionally include or be used with a trip detection system, such as the trip detection system as described in U.S. application Ser. No. 16/201,955, filed 27 Nov. 2018, which is incorporated herein in its entirety by this reference. Accordingly, any or all of the method 200 can be triggered in response to a trip detection by the trip detection system. Additionally or alternatively, the method can be implemented independently of a vehicular trip, asynchronously with vehicle trips and/or vehicular navigation, and/or with any other suitable timing. As an example, trip detection can be based on vehicle movement and/or vehicle traversal (e.g., which is substantially aligned with a longitudinal axis of the vehicle and/or a roadway; above a particular speed; above a threshold speed; longitudinal velocity component above a threshold—where the longitudinal velocity is substantially aligned with a roadway and/or lane/centerline thereof). As an illustrative example, sensor data (e.g., inertial and/or GPS) collected with a mobile device arranged onboard a vehicle can detect a vehicle trip based on the velocity (e.g., substantially aligned with the longitudinal axis of the vehicle) and/or speed of the mobile device as it moves with the vehicle; in-trip lateral accelerations occurring during the trip (e.g., substantially orthogonal to the longitudinal axis and/or direction of traversal; as measured by mobile device sensors) can be separately analyzed to evaluate driver behavior during the trip (e.g., aggressive lane changes, swerves, etc.; in addition to any behavioral analyses based on the longitudinal traversal of the vehicle, such as may be based on longitudinal acceleration, average speed, speed fraction relative to roadway speed limit, etc.).
However, the system can include or be used with any other suitable trip detection system(s) and/or model(s), and/or can be otherwise configured.
In variants, the system can optionally include or be used with the screen interaction system, tap detection system (and/or in-hand versus stationary classification system) and/or method(s) as described in U.S. application Ser. No. 16/700,991, filed 2 Dec. 2019, and/or U.S. application Ser. No. 17/831,731, filed 3 Jun. 2022, each of which is incorporated herein in its entirety by this reference. Accordingly, any or all of the method 200 can be occur for/during portions of a vehicular trip where the mobile device may be classified as ‘stationary’ relative to the vehicle and/or based on any other suitable period(s) of user interaction (or non-interaction) with the mobile device. For example, the method 200 can be executed for periods of a vehicular trip where the mobile device is arranged in a cup holder, a user's pocket, mounted to the vehicle dash, or otherwise arranged in a substantially stationary position relative to the vehicle (e.g., for the purpose of inertial-derived heading estimation, where inertial signals during in-hand periods may be noisy, etc.). Additionally or alternatively, the method can be implemented independently of any user-interaction or user-behavior classifications, asynchronously with user-interaction with the mobile device, and/or with any other suitable timing. However, the system can include or be used with any other screen interaction (or tap detection) system(s) and/or model(s), and/or can be otherwise configured.
In variants, the system can optionally include or be used with a sensor fusion system, which can function to aggregate data (e.g., in accordance with Block S200) and/or derive vehicle motion parameters (e.g., trajectory, PVA data, etc.; longitudinal speed and acceleration, lateral acceleration, angular velocity, heading/course, etc.) from the set of sensors (e.g., of the mobile device). For example, the sensor fusion system can include a set of Kalman filters (e.g., EKF, etc.) and/or models. Alternatively, the system can include any other suitable sensor fusion systems/modules, and/or can otherwise exclude a sensor fusion system(s) (e.g., where the mobile device can be assumed to be stationary relative to the vehicle). However, the system can include or be used with any other suitable sensor fusion system(s) and/or model(s), and/or can be otherwise configured.
In variants, the system can optionally include or be used with: a criteria detection system, which can function to execute Block S300. For example, the criteria detection system can include a heuristic classifier, decision-tree, and/or rule-based model which checks for a set of criteria and/or facilitates detection/classification of lateral driving events in accordance with the method 200. However, the system can include or be used with any other suitable criteria detection system(s) and/or model(s), and/or can be otherwise configured.
In variants, the system can optionally include or be used with: a lateral driving assessment system, which can function to execute one or more of Block S400, S500 and/or S600 of the method. In a first example, the driving assessment system can be a software module onboard the mobile device (e.g., facilitating executing of method elements S400, S500, and/or S600 onboard the mobile device). In a second example, the driving assessment system can be remote (e.g., facilitating remote assessment of driving behaviors based on the mobile driving data). However, the system can include or be used with any other suitable lateral driving assessment system(s) and/or model(s), and/or can be otherwise configured.
However, the system can include any other suitable set of models/subsystems.
In variants, the system can optionally include or be used with a set of databases, which can include: map data (e.g., route maps, acceleration profiles, etc.), historical data (e.g., historical behavior patterns, historical acceleration data for a set of users in a particular region or driving context), contextual data (e.g., weather data, traffic data, etc.), and/or any other suitable databases or supplemental data. The set of databases can function to facilitate lateral driving behavior determinations and/or criteria detection, evaluation, and/or analysis of (lateral) driving events. However, the system can alternatively access and/or receive data from a remote database, and/or can otherwise be used without a database(s). For example, in variants the system and/or method processes can be entirely local to the mobile device (e.g., edge computing), without requiring access to external data sources/information.
However, the system can additionally or alternatively include any other suitable set of components.
As shown in
The method 200 preferably functions to detect and assess the (lateral) driving behavior associated with a user, which can be used in numerous ways, examples of which include: determining a driving score for a user (e.g., for use by an insurance company); detecting that a driver is in danger and/or is putting other drivers in danger (e.g., to trigger an emergency response action, etc.), understanding risky roads and/or locations and/or scenarios (e.g., based on aggregated data from multiple drivers), and/or any other use cases.
In preferred variations of the method, the driving behavior is assessed using only data collected at a mobile device (e.g., user device) of the user, but alternatively any other information (e.g., from an OBD port of the vehicle, from remote sensors, etc.) can be used.
The method 200 is further preferably configured to detect driving behavior which includes lateral events (a.k.a. lateral movement events), with lateral events preferably herein referring to behaviors and/or actions which involve the vehicle moving, accelerating, steering, or otherwise deviating in a lateral direction (e.g., non-parallel with respect to the vehicle's course, perpendicular or substantially perpendicular with respect to the vehicle's heading/trajectory, etc.). These can include, but are not limited to: lane change behaviors and/or actions (e.g., aggressive and/or unsafe lane changing, fast lane changing, high speed and/or high acceleration lane changing, lane changing without signaling, etc.), swerving behaviors and/or actions, driving off-center of a lane (e.g., at the edge of a lane, between lanes, etc.), drifting within and/or out of a lane, and/or any other events involving lateral motion. Additionally or alternatively, the method 200 can detect any other driving behavior.
As an example, lateral movement events can be associated with lateral deviations relative to the longitudinal vehicle movement (e.g., aligned with a lane centerline/path, which the vehicle may substantially tracks by longitudinal movement/traversal; where the direction of longitudinal vehicle movement is substantially aligned with a lane and/or generally tangent to a path of a road; aligned with a longitudinal vehicle axis).
The method 200 is preferably performed with a system 100 as described above, and further preferably with a set of computing and/or processing subsystems (e.g., onboard a mobile device, remote from a mobile device, etc.) configured to execute a set of algorithms and/or models (e.g., which can individually or collectively form a sensor fusion system configured to execute S200 and/or a criteria detection system configured to execute S300 and/or a lateral driving assessment system configured to execute S400, S500, and/or S600). Additionally or alternatively, the method 200 can be performed with any other suitable components and/or system(s).
In some variations of the method 200 (e.g., as shown in
In variants, all or a portion of method 200 can be executed onboard the mobile device (e.g., processing for each method element can be performed onboard the mobile device and/or the mobile device can include each processing module and/or system element). Additionally or alternatively, method process/elements can be performed at least partially remotely (e.g., at a remote server, cloud computing system, etc.), with centralized or distributed computing nodes (e.g., central processing, distributed processing, etc.). In variants, the method is preferably performed contemporaneously and/or concurrently with a driving trip (e.g., during a driving trip; analyzing driver behavior in real time, near real time, etc.), but various elements can additionally or alternatively be performed asynchronously with a driving trip, periodically (e.g., daily, weekly, monthly, annually, etc.), aperiodically, after and/or in response to completion of a driving trip, and/or with any other suitable timing/frequency.
The method 200 can include collecting data from a set of sensors S100, which functions to receive information with which to detect and assess driving behavior associated with a user (e.g., a driver). Additionally or alternatively, S100 can perform any other functions.
The sensor data (equivalently referred to herein as telematic data) can include data from any or all of the sensors described above, and/or any other sensors. The sensor data is preferably at least partially received via a software development kit (SDK) and/or a client application executing on the user device, but can additionally or alternatively be received from an OBD-II port (e.g., via a wireless connection, via a wired connection, etc.) and/or from any other sources.
The sensor data is preferably received at a processing system (e.g., as described above) for processing, but can additionally or alternatively be received at any other locations.
S100 can optionally additionally include pre-processing (e.g., filtering, clipping, normalizing, etc.) any or all of the sensor data.
S100 can optionally additionally include receiving one or more inputs from a user and/or any other information from a user. These inputs can be received at a client application executing on the user device, determined based on information from one or more sensors, and/or otherwise received. S100 can further optionally include collecting information from memory, storage, and/or databases, such as historical information (e.g., associated with the user, aggregated from multiple users, etc.), a set of maps, and/or any other sources.
Additionally or alternatively, S100 can include collecting any other information.
In a preferred set of variations, the data in S100 is collected entirely from a mobile device and/or user device of the user. This can have the benefit of enabling easy, fast, and/or uniform data collection among an aggregated set of users, such as through a software development kit (SDK) executing on the mobile device.
In alternative variations, data in S100 can be collected from other devices, databases, memory/storage, and/or any other sources.
The method 200 can optionally include aggregating data S200, which functions to determine a set of intermediate outputs with which to inform any or all subsequent processes of the method 200. In some variations, for instance, S200 can function to determine a set of intermediate outputs with which to check for satisfaction of a set of criteria in S300. Additionally or alternatively, S200 can function to determine a set of intermediate outputs with which to determine a set of lateral event features in S400 and/or lateral event outcomes in S500. Additionally or alternatively, S200 can perform any other functions.
S200 is preferably performed in response to and based on S100, but can additionally or alternatively be performed in response to another process of the method 200, prior to and/or during any process of the method, multiple times, and/or at any other times. Alternatively, the method 200 can be performed in absence of S200.
S200 can optionally include performing a sensor fusion process, which functions to aggregate (e.g., fuse) data from any or all of the sensors to determine a set of intermediate outputs. Additionally or alternatively, the sensor fusion process can function to determine a set of metrics associated with the vehicle based on data collected directly at the mobile device. In variations in which the mobile device is moving relative to the vehicle (and/or stationary but rotated relative to the vehicle course), the sensor fusion process can function to determine (e.g., predict, approximate, etc.), the motion of the vehicle such that the user's driving behavior can be accurately assessed.
Each of the set of intermediate outputs is preferably defined for the vehicle, such that the intermediate outputs reflect motion of the vehicle. Additionally or alternatively, any or all of the intermediate outputs can reflect motion of the mobile device.
In some variations (e.g., for lateral event detection), these intermediate outputs include any or all of: a course parameter (e.g., direction of the vehicle relative to earth [e.g., relative to true North], direction of the mobile device relative to earth, etc.), a heading parameter, a speed parameter (e.g., speed of the mobile device and/or vehicle), an angular velocity parameter (e.g., of the mobile device and/or vehicle). For example, S200 can function to separate vehicle movements (e.g., in an earth frame) from mobile device movements (e.g., in a vehicle coordinate frame) by sensor fusion (e.g., with a fusion model and/or a set of EKFs).
In a first set of variations, S200 can implement a fusion model which is configured to separately model multiple states of interest associated with the vehicle, which preferably differ in terms of which reference frame the information is determined relative to, but can additionally or alternatively be otherwise configured. The multiple states of interest produced by the fusion model can include: accelerations with respect to a coordinate frame(s), such as an acceleration of the vehicle with respect to the earth n-frame. The fusion model can further produce as outputs: a location of the vehicle (e.g., with respect to earth), a velocity/trajectory of the vehicle (e.g., with respect to earth), an orientation of the mobile device with respect to the n-frame, and/or any other outputs. Additionally or alternatively, the fusion model can produce a subset of these outputs and/or any other outputs. In one example, the fusion model is a Kalman filter (e.g., extended Kalman filter as described above. Additionally or alternatively, the measurement model, state model, and/or any other components of the filter can be otherwise suitably determined/derived.
In a second set of variations, a set of multiple models can be implemented (e.g., in parallel), which are individually tuned and modeled based on different assumptions. In a first specific example of the second set of variations, the acceleration outputs produced by the multiple models are aggregated in a weighted fashion based on posterior probabilities (e.g., determined with a trained model) of each state being true given observations from any or all of the set of sensors. Additionally or alternatively, position and/or velocity can be produced by the multiple models and aggregated in a weighted fashion. In a second specific example of the second set of variations, the acceleration outputs (and/or velocity and/or position) produced by the multiple models are aggregated in a weighted fashion based on binary weights which are determined based on user information, such as any or all of: a user input of whether or not he had mounted his phone during a trip, historical information associated with the user (e.g., whether or not he has a mount for his phone), and/or any other information. Additionally or alternatively, only the model corresponding to an assumption having a weight of “1” (or the highest weight in non-binary use cases) is processed. In a third specific example of the second set of variations, the acceleration outputs (and/or velocity and/or position outputs) produced by the multiple models are aggregated in a weighted fashion based on weight values determined with a trained model (e.g., machine learning model, deep learning model, etc.).
In variants, S200 can optionally include one or more processes which function to enable any or all of S200 to be performed on the mobile device and/or with limited compute, limited latency, and/or based on any other goals or requirements. In preferred variations, such as those involving Kalman filters or other filters, a smoothing algorithm (equivalently referred to herein as a backward pass algorithm)—which functions to refine estimates of previous states in light of later observations.
Additionally or alternatively, the mobile device can be assumed to be stationary (relative to the vehicle) during one or more periods of a vehicle trip (e.g., such as when the device may be classified as stationary by an in-hand detection module, examples of which are shown in
In variants, S200 can optionally additionally or alternatively include aggregating data from multiple users (e.g., for determining an aggregated metric and/or outcome), from multiple time periods (e.g., for a single user), from multiple sources, and/or data can be otherwise aggregated and/or not aggregated.
S200 can additionally or alternatively include any other processes.
The method 200 can optionally include checking for a set of criteria S300, which functions to inform any or all of the other processes of the method 200 (e.g., whether or not to perform certain processes of the method, which data to use in determining lateral event features and/or outcomes, etc.). Additionally or alternatively, S300 can function to eliminate certain events and/or scenarios from being further processed in the method 200, such as upon detecting that the driving behavior of the vehicle does not correspond to an event of interest (e.g., aggressive lane change, swerve behavior, etc.). This can optionally further function to preserve and/or prioritize computational resources for other processing, prevent a false positive determination of an outcome (e.g., based on an irrelevant event), and/or perform any other functions. Additionally or alternatively, S300 can perform any other functions.
In preferred variations, for instance, S300 functions to determine whether or not the data (e.g., reflective of the user's driving behavior over one or more time steps) is a candidate for detecting an event of interest (e.g., aggressive lane change, swerving, etc.).
S300 can be performed in response to any number of other processes of the method 200, such as, but not limited to: in response to S200, in response to S100, in response to S400 (e.g., wherein criteria are evaluated based on lateral event features), in response to S500 (e.g., wherein criteria are evaluated based on lateral event outcomes), in response to S600 (e.g., wherein criteria are evaluated based on a calculated score), and/or in response to any other processes and/or combination of processes. Additionally or alternatively, S300 can be performed in parallel with any or all of the processes of the method 200, in response to a trigger, prior to any processes of the method 200, multiple times, and/or at any other times.
S300 preferably includes checking for a set of lateral deviation criteria, which functions to detect if a lateral event (e.g., any lateral event, lateral event of interest, lateral deviation, etc.; lane change, swerve, drift from lane center, etc.) has taken place. In a preferred set of variations, upon detecting that any or all of the set of lateral deviation criteria are not satisfied, the remaining processes of the method 200 are not performed. Alternatively, a subset of the subsequent processes is not performed, all processes are performed, and/or any other actions can be triggered.
One or more lateral deviation criteria are preferably configured to detect a motion signature associated with lane change events, such as a sequence of alternating turn directions (e.g., left turn followed by a right turn, right turn followed by a left turn, multiple such maneuvers, any sequence of alternating “turns,” etc.), which represent the turning and correcting motions of lane change behaviors, swerving behaviors, and/or any other lateral event behaviors.
A turn can be defined as a lateral deviation in heading and/or course, and can further optionally be defined by minimum and/or maximum angle values in the set of criteria. For instance, in order to detect lane changes and/or swerving, without detecting full turns or other behaviors (e.g., merging onto a freeway, merging off of a freeway, etc.), the set of lateral deviation criteria can additionally or alternatively include checking to see that an angle associated with each of the turning motions falls below a threshold (e.g., 10 degrees, between 5 and 15 degrees, 20 degrees, between 5 and 30 degrees, 30 degrees, 45 degrees, 90 degrees, any range bounded by these values, etc.). Additionally or alternatively, the set of criteria can check to see that the angle(s) exceed a threshold (e.g., to prevent detection of minor corrections of heading/course within a lane).
Additionally or alternatively, any other lateral deviation criteria can be evaluated.
S300 can optionally include checking for a set of criteria which indicate (e.g., determine, predict, etc.) whether or not the mobile device is stationary relative to the vehicle. This preferably functions to determine which data is used to determine a set of lateral features in S400 (e.g., as described below) and/or the lateral outcomes in S500. As an example, S300 can check for a ‘stationary’ classification output by a screen interaction system/method, such as described in U.S. application Ser. No. 16/700,991, filed 2 Dec. 2019, or U.S. application Ser. No. 17/831,731, filed 3 Jun. 2022, each of which is incorporated herein in its entirety by this reference. Additionally or alternatively, checking for the set of criteria can function to prescribe which intermediate outputs to determine in S200.
These criteria can be checked in parallel and/or substantially in parallel (e.g., overlapping in time, partially overlapping in time, etc.) with the lateral deviation criteria, prior to the checking the lateral deviation criteria (e.g., in response to S100, in response to S200, etc.), after checking the lateral deviation criteria, multiple times, and/or at any other times.
In a first set of variations, checking for whether or not the mobile device is stationary relative to the vehicle is performed in response to S100 and based on sensor data (e.g., raw sensor data, gravity and/or gyroscope data, etc.) collected at the mobile device. In a second set of variations, this determination is made based on one or more intermediate outputs produced by a sensor fusion process.
In a second set of variations, nonexclusive with the first, S300 can include checking for a motion signature associated with a lane change behavior (e.g., checking for or comparing against a motion signature of a lane change behavior, where lateral events are detected based on the presence of the motion signature). In an example, the motion signature is a pair of opposite sign lateral accelerations. In a second example, the motion signature is a pair of opposite sign lateral accelerations, wherein the pair of opposing accelerations (e.g., steering left and then right, which results in a counterclockwise acceleration and rotation followed by a clockwise rotation and acceleration) occur consecutively and within a frequency bandwidth, wherein high frequency vibrations above the frequency bandwidth are filtered out of the sensor dataset (e.g., neglecting high frequencies such as may be generated from a phone vibrating within a cupholder or in response to receiving a text message, for example; such as vibrations with a frequency above: 50 Hz, 100 Hz, 1000 Hz, etc.; low-pass filter), wherein the motion signature comprises higher-frequency peaks, within the frequency bandwidth, which occurring along a lower-frequency acceleration signal (e.g., below the frequency bandwidth; lateral acceleration signal associated with the vehicle tracking the road curvature, etc.; high-pass filtering, band-pass filter, etc.).
In one variant, at least one lateral movement event is detected as a higher-frequency peak (and/or a pair of higher frequency peaks of opposite sign) within a lower-frequency signal. In an example, the lower-frequency signal corresponds to the lateral acceleration (and/or angular velocity/acceleration) associated with a roadway curvature, wherein the higher-frequency peak corresponds to a maximal acceleration(s) and/or angular speed of a lane change maneuver (e.g., in both a clockwise and counterclockwise direction; steering towards an adjacent lane and counter-steering upon reaching the adjacent lane).
Additionally or alternatively, any other criteria can be checked for and/or otherwise suitably used.
The method 200 can include determining a set of lateral event features S400, which preferably functions to determine features with which to determine and/or assess (e.g., determine a severity of) a set of lateral event outcomes. Additionally or alternatively, S400 can function to determine information with which to compare with a set of criteria (e.g., in S200), determine information with which to trigger one or more processes of the method 200, and/or can perform any other functions.
S400 is preferably performed in response to and based on S300, but can additionally or alternatively be performed in response to another process of the method 200, prior to any other process of the method 200, during any process of the method 200, multiple times during the method 200, and/or at any other time(s) during the method 200. Alternatively, the method 200 can be performed in absence of S400.
The set of lateral event features preferably includes a lateral acceleration metric (e.g., lateral acceleration of vehicle) (e.g., as shown in
The lateral acceleration metric (and/or any other lateral event features) can be determined in any number of ways and based on any suitable data and/or metrics, such as, but not limited to, any or all of: a heading of the vehicle, a set of locations of the vehicle such as represented as a course (e.g., trajectory) of the vehicle, an angular velocity of the vehicle, a speed of the vehicle, derivatives of these metrics, any of these metrics relative to the mobile device, and/or any other metrics.
In some variations, the way in which the metric(s) are determined is dependent (at least in part) based on the results of S300, such as based on a determination of whether or not the phone is stationary relative to the vehicle.
In a first set of specific examples, the lateral acceleration is determined based on a heading of the mobile device (and/or heading of the vehicle as determined based on the heading of the mobile device) and a speed of the vehicle (e.g., speed multiplied by a derivative of heading). The first set of specific examples can optionally be triggered in response to (and/or based on) detecting that the mobile device is stationary relative to the vehicle (e.g., as shown in
In a second set of specific examples, the lateral acceleration is determined based on a set of locations (e.g., course/trajectory) of the vehicle and a speed of the vehicle (e.g., speed multiplied by a derivative of course). The second set of specific examples can optionally be triggered in response to (and/or based on) detecting that the mobile device is moving relative to the vehicle (e.g., in which case a heading metric may not be accurate and/or usable) (e.g., as shown in
In a third set of specific examples, the lateral acceleration is determined based on angular velocity and speed. Additionally or alternatively, an angular velocity can supplement the determination of lateral acceleration in the first and/or second specific examples.
In a fourth set of specific examples, the lateral acceleration is determined based on any or all of the inputs described above, such as with a trained model (e.g., machine learning model) and/or as an output of a sensor fusion process.
Additionally or alternatively, a lateral acceleration metric can be determined in absence of a determination of whether or not the phone is stationary, based on any other criteria, and/or otherwise suitably determined.
The set of lateral event features can optionally additionally include one or more temporal features, such as a frequency of occurrence of the lateral event, a duration of the lateral event, and/or any other features. In some variations, for instance, if a change in lateral acceleration occurs frequently (e.g., within a short time period), a determination in S500 can be made that the driver has been swerving (e.g., rather than engaging in a single lane change).
The lateral event features (and/or lateral acceleration metric[s] associated therewith) can be determined using one or more predetermined models[s] (e.g., a lateral driving assessment model; of the set of models no). For example, lateral event features can be extracted from mobile device sensor data (e.g., collected during S100 and/or aggregated during S200) with a classification model (e.g., pretrained and/or predetermined) which can include one or more: a binary classifier, a multi-class classifier, a neural network model (e.g., DNN, CNN, RNN, etc.), a logistic regression model, Bayesian networks (e.g., naïve Bayes model), a cascade of neural networks, compositional networks, Markov chains, decision trees, predetermined rules, probability distributions, heuristics, probabilistic graphical models, and/or other models.
Lateral event features (interchangeably referenced herein with “derived signals”) can relate a set(s) of sensor data which can include: time domain data, frequency domain data, localization data, motion data, user behavior data (e.g., device utilization, screen interaction, etc.), and/or any other suitable sensor data and/or mobile device data. Lateral event features can be extracted from a single sensor or a combination of multiple sensors (e.g., multiple sensor types, such as an accelerometer and a gyroscope; GPS fused with inertial sensing, etc.). Lateral event features can be generated algorithmically according to a predetermined set of rules/heuristics (e.g., manually assigned, pre-generated by a computer, etc.), using one or more pre-trained models (e.g., machine learning [ML] models, neural networks, fully convolutional network [FCN], convolutional neural network [CNN], recurrent neural network [RNN], artificial neural network [ANN], etc.), polynomial regression, and/or with any other suitable techniques. Features can be constructed from time segments of sensor data spanning: a full duration of a vehicle trip, a partial duration of a vehicle trip (e.g., between trip detection and/or initiation and an instantaneous time, dynamic time window between screen interactions, etc.), and/or any other suitable duration(s). More preferably, the lateral event features can include in-trip lateral events (e.g., associated with a time period of vehicular traversal and/or a duration of the vehicular trip), but can additionally or alternatively include or be evaluated in conjunction with pre-trip/post-trip features (e.g., entry/egress characteristics), supplemental classifications surrounding the trip (e.g., whether the user is a driver or a passenger), and/or any other suitable data features. The set of features is preferably fixed and/or predefined (e.g., based on a set of inputs used to train/update the models).
Additionally, the data features used to evaluate lateral driving events and/or used to determine lateral event outcomes can include any or all of the features determined/extracted as described in U.S. application Ser. No. 18/073,959, filed 2 Dec. 2022, U.S. application Ser. No. 17/959,067, filed 3 Oct. 2022, U.S. Provisional Application Ser. No. 63/285,650, filed 3 Dec. 2021, and/or U.S. application Ser. No. 17/474,591, filed 14 Sep. 2021, each of which is incorporated herein in its entirety by this reference.
Lateral event features are preferably generated at a local processor of the mobile device but can alternatively be generated at a remote processor and/or at any other suitable processing endpoint. In one variant, all features can be generated locally at the user device. Features can be generated/updated: continuously (e.g., during a trip), periodically, aperiodically, in response to satisfaction of a trigger condition (e.g., time trigger, speed threshold, etc.), during a vehicle trip (e.g., during movement of the mobile device within a vehicle, between detection of a vehicle trip and satisfaction of a trip termination condition), and/or with any other suitable timing.
In one example, the lateral acceleration metric is determined with a pretrained machine learning (ML) classifier at the mobile user device.
In one example, the lateral acceleration metric is determined based at least in part on an angular velocity (e.g., of the vehicle) and/or a heading (e.g., of the vehicle).
However, any other suitable features can be extracted and/or generated. Additionally or alternatively, the set of lateral event features can include any other features and/or be determined in any other suitable way(s) and based on any suitable metric(s).
The method 200 can include determining a set of lateral event outcomes S500, which functions to make a determination of a particular driving behavior associated with the user.
S500 can optionally be performed in response to S400, wherein the set of lateral events and/or lateral event outcomes are determined, at least in part, based on a set of lateral event features determined in S400. Additionally or alternatively, S500 can be performed in absence of S400, in response to and/or based on another process of the method 200, prior to any process of the method 200, during any process of the method 200, multiple times, and/or at any other times.
The set of lateral event outcomes can include any or all of: a binary determination/classification (e.g., indicating a determination of whether or not the driver has participated in a lateral driving event), a severity score (e.g., level of risk, level of danger, level of responsibility of the user, etc.) of the lateral event, a classification (e.g., a type of lateral driving event the driver has participated in, such as a swerve, lane change, multi-lane change, etc.), an event frequency (e.g., indicating how commonly a user participates in the driving event, such as may be used to evaluate whether or not it is a persistent habit), and/or any other determinations. As an example, determining the set of lateral event outcomes can include determining a severity score for the set of lateral (movement) events based on a magnitude of lateral deviation (e.g., lateral acceleration).
The set of lateral event outcomes is preferably determined based on a set of lateral event features (e.g., as determined in S400), but can additionally or alternatively be determined based on any other information (e.g., data received in S100, data aggregated in S200, etc.), in absence of a set of lateral event features, any combination of information, and/or otherwise suitably determined.
In a preferred set of variations, for instance, the set of lateral event outcomes preferably includes any or all of: a determination of whether or not one of a set of lateral events has occurred, a categorization of the type of lateral event that has occurred, a level of risk associated with the lateral event, and/or any other outcomes.
Determining any or all of the set of lateral event outcomes preferably includes comparing any or all of the set of lateral event features (and/or raw data received in S100, aggregated data produced in S200, etc.) with a set of thresholds, such as, but not limited to, any or all of: a minimum heading/course change threshold, a maximum heading/course change threshold, a minimum lateral acceleration threshold, a maximum lateral acceleration threshold, a minimum frequency threshold, a maximum frequency threshold, and/or any other thresholds. The thresholds can be any or all of: static, dynamic, and/or any combination thereof. In some variations, for instance, any or all of the thresholds (e.g., lateral acceleration threshold) reflect and/or are dynamically determined based on any or all of: the speed limit at the location of the vehicle, the road geometry (e.g., to what degree the road is banked, curvatures of the road, etc.; determined from a map and/or a path of the vehicle), road scenarios (e.g., intersections, freeways, residential roads, etc.), traffic conditions, and/or any other information.
S500 preferably includes comparing a lateral acceleration with a set of one or more magnitude thresholds, which characterizes a lateral event (e.g., lane change) as aggressive (e.g., risky, dangerous, unsafe, etc.) if it exceeds the threshold (e.g., a lateral acceleration threshold of between 0.2-0.4 g, a lateral acceleration threshold of greater than 0.2 g, etc.). The set of magnitude thresholds can additionally or alternatively function to distinguish lateral events. For instance, a lateral event of swerving can be distinguished from a lane change if it is below a maximum lateral acceleration threshold (and optionally above another minimum threshold).
S500 can optionally additionally or alternatively include comparing one or more temporal features with a set of thresholds. For instance, a swerving lateral event can be detected if a certain number of lateral acceleration events occurs within a predetermined threshold, such as by: exceeding a minimum frequency threshold, falling below a minimum duration between lateral acceleration events, and/or any other thresholds.
In variants, lateral event outcomes can be determined using a classifier, such as a pretrained ML classifier, decision trees (e.g., boosted decision tree), GBM, heuristic classifiers (e.g., rule-based classification), and/or other suitable classifier or classification technique(s). In some variants, lateral event outcomes can be determined in conjunction with the lateral event features and/or by the same process (e.g., using the same classification technique[s]/processes[s]; as another model output, etc.) and/or separately (e.g., by a subsequent downstream process/determination, by a different process etc.).
In variants, S500 can include determining/detecting a set of lateral events based on the presence of a motion signature (e.g., independently or and/or in conjunction with S300). For example lateral events can be determined based on the presence of a (lateral) motion signature associated with a lane change behavior (e.g., where the lateral motion signature is detected/classified using a predetermined model). In an example, the motion signature is a pair of opposite sign lateral accelerations. In a second example, the motion signature is a pair of opposite sign lateral accelerations, wherein the pair of opposite sign accelerations occur consecutively and within a frequency bandwidth, wherein high frequency vibrations above the frequency bandwidth are filtered out of the sensor dataset (e.g., neglecting high frequencies such as may be generated from a phone vibrating within a cupholder or in response to receiving a text message, for example; such as vibrations with a frequency above: 50 Hz, 100 Hz, 1000 Hz, etc.), wherein the motion signature comprises higher-frequency peaks, within the frequency bandwidth, which occurring along a lower-frequency acceleration signal below the frequency bandwidth (e.g., lateral acceleration signal associated with the vehicle tracking the road curvature, etc.). In one variant, at least one lateral movement event is detected based on a higher-frequency peak within a lower-frequency signal (e.g., a pair of peaks of opposite sign). In an example, the lower-frequency signal corresponds to the lateral acceleration (and/or angular velocity/acceleration) associated with a roadway curvature, wherein the higher-frequency peak corresponds to a lane change maneuver.
However, the set of lateral events and/or outcomes associated therewith can be otherwise determined.
Additionally or alternatively, S500 can include any other suitable processes.
The method 200 can optionally include triggering an action based on the set of lateral event outcomes S600, which functions to respond to a determination that a user has been driving in an aggressive and/or risky fashion. This can in turn function to appropriately and/or dynamically set an insurance premium for the user, prevent the risk of a collision, protect the driver and/or other users from harm, deduce insights into particularly risky driving environments, and/or can perform any other functions. In variants, S600 can occur at least partially based on (and/or in response to) checking for criteria, wherein the action is triggered based on satisfaction of one or more criteria. For example, actions can be triggered in response to satisfaction of a set of lateral deviation criteria (e.g., satisfaction of a severity threshold, lane change frequency threshold, driver aggression threshold, risk score threshold, etc.). However, actions can alternatively be triggered periodically (e.g., weekly), at the end of a driving trip, in response to a remote (pull) request, in response to a user request (e.g., at the mobile device) and/or with any other suitable frequency/timing.
Examples of actions include any or all of: calculating and/or updating a score (e.g., driver score), training and/or retraining a model, alerting a user (e.g., the driver, an insurance company associated with the driver, an employer [e.g., trucking company, rideshare service, public transit entity, etc.] associated with the driver, other drivers on the road near a risky driver, a passenger in the vehicle, etc.; via a push notification or alert on the mobile device; during a driving trip, after a driving trip, etc.), triggering an emergency response (e.g., alerting an ambulance of the vehicle's location and suspected accident, contacting the driver, contacting a family member of the driver, etc.), triggering an alert (e.g., to the driver, to a family member of the driver, to other drivers on the road, etc.), updating a score associated with a route and/or location driven by the driver, and/or any other actions.
In one variant, S600 can include providing driver feedback based on the lateral event outcomes such as updating a driver risk score, providing in-trip driver statistics/parameters, and/or any other suitable characterizations of a mobile device user (driver) based on the trip.
In one variant, S600 can include providing user/driver feedback based on a risk score (e.g., such as described in U.S. application Ser. No. 14/566,408, filed 10 Dec. 2014, which is incorporated herein in its entirety by this reference) which is determined based on the set of lateral event outcomes (e.g., risk score for a trip, updated driver risk score, etc.).
However, any other suitable action can be triggered based on the set of lateral event outcomes. Alternatively, no action can be triggered based on the set of lateral event outcomes, or actions can be otherwise suitably triggered for the method.
The method 200 can additionally or alternatively include any other processes.
Although omitted for conciseness, the preferred embodiments include every combination and permutation of the various system components and the various method processes, wherein the method processes can be performed in any suitable order, sequentially or concurrently.
Embodiments of the system and/or method can include every combination and permutation of the various system components and the various method processes, wherein one or more instances of the method and/or processes described herein can be performed asynchronously (e.g., sequentially), contemporaneously (e.g., concurrently, in parallel, etc.), or in any other suitable order by and/or using one or more instances of the systems, elements, and/or entities described herein. Components and/or processes of the following system and/or method can be used with, in addition to, in lieu of, or otherwise integrated with all or a portion of the systems and/or methods disclosed in the applications mentioned above, each of which are incorporated in their entirety by this reference.
Additional or alternative embodiments implement the above methods and/or processing modules in non-public transitory computer-readable media, storing computer-readable instructions. The instructions can be executed by computer-executable components integrated with the computer-readable medium and/or processing system. The computer-readable medium may include any suitable computer readable media such as RAMs, ROMs, flash memory, EEPROMs, optical devices (CD or DVD), hard drives, floppy drives, non-public transitory computer readable media, or any suitable device. The computer-executable component can include a computing system and/or processing system (e.g., including one or more collocated or distributed, remote or local processors) connected to the non-public transitory computer-readable medium, such as CPUs, GPUs, TPUS, microprocessors, or ASICs, but the instructions can alternatively or additionally be executed by any suitable dedicated hardware device.
As a person skilled in the art will recognize from the previous detailed description and from the figures and claims, modifications and changes can be made to the preferred embodiments of the invention without departing from the scope of this invention defined in the following claims.
This application claims the benefit of U.S. Provisional Application No. 63/314,716, filed 28 Feb. 2022, which is incorporated herein in its entirety by this reference.
| Number | Date | Country | |
|---|---|---|---|
| 63314716 | Feb 2022 | US |
