Patent Application
20220221281
This disclosure relates to methods and systems for processing vehicle event data for analysis of road segments and turn ratios.
The automotive industry is undergoing a radical change unlike anything seen before. Disruption is happening across the whole of the mobility ecosystem. The result is vehicles that are more automated, connected, electrified and shared. This gives rise to an explosion of car generated data. This rich new data asset remains largely untapped.
Vehicle location event data, such as GPS data, is extremely voluminous and can involve 200,000-400,000 records per second. The processing of location event data presents a challenge for conventional systems to provide substantially real-time analysis of the data, especially for individual vehicles. In particular, end user technology can require data packages. What is needed are system platforms and data processing algorithms and processes configured to process and store high-volume data with low latency while still making the high-volume data available for analysis and re-processing.
While there are systems for tracking vehicles, what is needed is virtually real-time and accurate trip and road information from high-volume vehicle data. What is needed is systems and algorithms configured to accurately identify journeys and journey destinations from vehicle movement and route analysis.
The following briefly describes embodiments to provide a basic understanding of some aspects of the innovations described herein. This brief description is not intended as an extensive overview. It is not intended to identify key or critical elements, or to delineate or otherwise narrow the scope. Its purpose is merely to present some concepts in a simplified form as a prelude to the more detailed description that is presented later. An exemplary advantage of the systems and methods described herein is optimized low latency. For example, the systems and methods described in the present disclosure are capable of ingesting and processing vehicle event data for up to at least 600,000 records per second for up to 12 million vehicles.
According to one aspect of the disclosure, there is provided a system that has at least one processor; and memory storing computer instructions that are accessible by the at least one processor. The at least one processor is configured to execute the computer instructions, and, when the at least one processor executes the computer instructions, the system: ingests location event data to a server, wherein the location event data for a given vehicle of the plurality of vehicles includes a plurality of vehicle event data points; for each of the plurality of vehicles, identifies a pair of road segments based on the location event data for the given vehicle, wherein the pair of road segments define at least in part an intersection; and calculates a turn ratio at the intersection based on the identified pairs of road segments for the plurality of vehicles.
According to various embodiments, the system may further include any one of the following features or any technically-feasible combination of some or all of the following features:
According to another aspect of the disclosure, there is provided a method for calculating a turn ratio at an intersection, wherein the method is carried out by at least one processor. The method includes: ingesting location event data to a server, wherein the location event data for a given vehicle of the plurality of vehicles includes a plurality of vehicle event data points; for each of the plurality of vehicles, identifying a pair of journey segments based on the location event data for the given vehicle, wherein the pair of journey segments define at least in part an intersection; and calculating a turn ratio at the intersection based on the identified pairs of journey segments for the plurality of vehicles.
According to another aspect of the disclosure, there is provided a method for modifying a base map based on a proportion of wayward points, wherein the method is carried out by at least one processor. The method includes the steps of: ingesting vehicle event movement data points for a plurality of vehicles; identifying a proportion of wayward points based on the vehicle event movement data points, wherein the identification includes determining a discrepancy between a heading as indicated by the vehicle event movement data and a heading for a road segment of a base map; determining that the proportion of wayward points exceeds a predetermined threshold amount; and when it is determined that the proportion of wayward points exceeds the predetermined threshold amount, modifying the base map.
According to various embodiments, the method may further include any one of the following features or any technically-feasible combination of some or all of the following features:
According to yet another aspect of the disclosure, there is provided a method for generating a lane index, wherein the method is carried out by at least one processor. The method includes the steps of: ingesting vehicle event movement data points for a plurality of vehicles; for each of the vehicle event movement data points, determining a distance between a location indicated by the vehicle event movement data point and a road segment selected from a plurality of road segments; based on a distribution of the determined distances, identifying one or more lanes of the corresponding road segment; and generating a lane index based on the identified lane(s) for each of the plurality of road segments.
According to various embodiments, the method may further include any one of the following features or any technically-feasible combination of some or all of the following features:
Preferred exemplary embodiments will hereinafter be described in conjunction with the appended drawings, wherein like designations denote like elements, and wherein:
Various embodiments now will be described more fully hereinafter with reference to the accompanying drawings, which form a part hereof, and which show, by way of illustration, specific embodiments by which the innovations described herein can be practiced. The embodiments can, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the embodiments to those skilled in the art. Among other things, the various embodiments can be methods, systems, media, or devices. The following detailed description is, therefore, not to be taken in a limiting sense.
Throughout the specification and claims, the following terms take the meanings explicitly associated herein, unless the context clearly dictates otherwise. The term “herein” refers to the specification, claims, and drawings associated with the current application. The phrase “in one embodiment” or “in an embodiment” as used herein does not necessarily refer to the same embodiment or a single embodiment, though it can. Furthermore, the phrase “in another embodiment” as used herein does not necessarily refer to a different embodiment, although it can. Thus, as described below, various embodiments can be readily combined, without departing from the scope or spirit of the present disclosure.
In addition, as used herein, the term “or” is an inclusive “or” operator, and is equivalent to the term “and/or” unless the context clearly dictates otherwise. The term “based on” is not exclusive and allows for being based on additional factors not described, unless the context clearly dictates otherwise. In addition, throughout the specification, the meaning of “a” “an” and “the” include plural references. The meaning of “in” includes “in” and “on.”
As used herein, the term “Host” can refer to an individual person, partnership, organization, or corporate entity that can own or operate one or more digital media properties (e.g., web sites, mobile applications, or the like). Hosts can arrange digital media properties to use hyper-local targeting by arranging the property to integrate with widget controllers or servers.
The following briefly describes various embodiments of a system, method, and computer program product for processing vehicle event data.
As used herein, a journey can include any trip, run, or travel to a destination.
Illustrative Logical System Architecture and System Flows
The Egress Server system 400 can be configured to be in communication with and provide data output to data consumers. The Egress Server system 400 can also be configured to be in communication with the Stream Processing Server 200.
The Analytics Server system 500 is configured to be in communication with and accept data from the Ingress Server system 100, the Stream Processing Server system 200, and the Egress Server system 400. The Analytics Server system 500 is configured to be in communication with and output data to a Portal Server system 600.
In at least one embodiment, Ingress Server system 100, Stream Processing Server system 200, Egress Server system 400, Analytics Server system 500, and Portal Server system 600 can each be one or more computers or servers. In at least one embodiment, one or more of Ingress Server system 100, Stream Processing Server system 200, Egress Server system 400, Analytics Server system 500, and Portal Server system 600 can be configured to operate on a single computer, for example a network server computer, or across multiple computers. For example, in at least one embodiment, the system 10 can be configured to run on a web services platform host such as Amazon Web Services (AWS) or Microsoft Azure. In an exemplary embodiment, the system 10 is configured on an AWS platform employing a Spark Streaming server, which can be configured to perform the data processing as described herein. In an embodiment, the system 10 can be configured to employ a high throughput messaging server, for example, Apache Kafka.
In at least one embodiment, Ingress Server system 100, Stream Processing Server system 200, Egress Server system 400, Analytics Server system 500, and Portal Server system 600 can be arranged to integrate and/or communicate using API's or other communication interfaces provided by the services.
In at least one embodiment, Ingress Server system 100, Stream Processing Server system 200, Egress Server system 400, Analytics Server system 500, and Portal Server system 600 can be hosted on Hosting Servers.
In at least one embodiment, Ingress Server system 100, Stream Processing Server system 200, Egress Server system 400, Analytics Server system 500, and Portal Server system 600 can be arranged to communicate directly or indirectly over a network to the client computers using one or more direct network paths including Wide Access Networks (WAN) or Local Access Networks (LAN).
As described herein, embodiments of the system 10, processes and algorithms can be configured to run on a web services platform host such as Amazon Web Services (AWS)™ or Microsoft Azure™. A cloud computing architecture is configured for convenient, on-demand network access to a shared pool of configurable computing resources (e.g. networks, network bandwidth, servers, processing, memory, storage, applications, virtual machines, and services). A cloud computer platform can be configured to allow a platform provider to unilaterally provision computing capabilities, such as server time and network storage, as needed automatically without requiring human interaction with the service's provider. Further, cloud computing is available over a network and accessed through standard mechanisms that promote use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs). In a cloud computing architecture, a platform's computing resources can be pooled to serve multiple consumers, partners or other third party users using a multi-tenant model, with different physical and virtual resources dynamically assigned and reassigned according to demand. A cloud computing architecture is also configured such that platform resources can be rapidly and elastically provisioned, in some cases automatically, to quickly scale out and rapidly released to quickly scale in.
Cloud computing systems can be configured with systems automatically control and optimize resource use by leveraging a metering capability at some level of abstraction appropriate to the type of service (e.g., storage, processing, bandwidth, and active user accounts). Resource usage can be monitored, controlled, and reported. As described herein, in embodiments, the system 10 is advantageously configured by the platform provider with innovative algorithms and database structures configured for low-latency.
A cloud computing architecture includes a number of service and platform configurations.
A Software as a Service (SaaS) is configured to allow a platform provider to use the provider's applications running on a cloud infrastructure. The applications are accessible from various client devices through a thin client interface such as a web browser (e.g., web-based e-mail). The consumer typically does not manage or control the underlying cloud infrastructure including network, servers, operating systems, storage, or even individual application capabilities, with the possible exception of limited user-specific application configuration settings.
A Platform as a Service (PaaS) is configured to allow a platform provider to deploy onto the cloud infrastructure consumer-created or acquired applications created using programming languages and tools supported by the provider. The consumer does not manage or control the underlying cloud infrastructure including networks, servers, operating systems, or storage, but can a have control over the deployed applications and possibly application hosting environment configurations.
An Infrastructure as a Service (IaaS) is configured to allow a platform provider to provision processing, storage, networks, and other fundamental computing resources where the consumer is able to deploy and run arbitrary software, which can include operating systems and applications. The consumer does not manage or control the underlying cloud infrastructure but has control over operating systems, storage, deployed applications, and possibly limited control of select networking components (e.g., host firewalls).
A cloud computing architecture can be provided as a private cloud computing architecture, a community cloud computing architecture, or a public cloud computing architecture. A cloud computing architecture can also be configured as a hybrid cloud computing architecture comprising two or more clouds platforms (private, community, or public) that remain unique entities but are bound together by standardized or proprietary technology that enables data and application portability (e.g., cloud bursting for load-balancing between clouds).
A cloud computing environment is service oriented with a focus on statelessness, low coupling, modularity, and semantic interoperability. At the heart of cloud computing is an infrastructure comprising a network of interconnected nodes.
Referring now to
Referring now to
A hardware and software layer 60 can comprise hardware and software components. Examples of hardware components include, for example: mainframes 61; servers 62; servers 63; blade servers 64; storage devices 65; and networks and networking components 66. In some embodiments, software components include network application server software 67 and database software 68.
Virtualization layer 70 provides an abstraction layer from which the following examples of virtual entities can be provided: virtual servers 71; virtual storage 72; virtual networks 73, including virtual private networks; virtual applications and operating systems 74; and virtual clients 75.
In one example, management layer 80 can provide the functions described below. Resource provisioning 81 provides dynamic procurement of computing resources and other resources that are utilized to perform tasks within the cloud computing environment. Metering and Pricing 82 provide cost tracking as resources are utilized within the cloud computing environment, and billing or invoicing for consumption of these resources. In one example, these resources can comprise application software licenses. Security provides identity verification for cloud consumers and tasks, as well as protection for data and other resources. User portal 83 provides access to the cloud computing environment for consumers and system administrators. Service level management 84 provides cloud computing resource allocation and management so that required service levels are met. Service Level Agreement (SLA) planning and fulfillment 85 provides pre-arrangement for, and procurement of, cloud computing resources for which a future requirement is anticipated in accordance with an SLA.
Workloads layer 90 provides examples of functionality for which the cloud computing environment can be utilized. Examples of workloads and functions that can be provided from this layer include mapping and navigation 91; ingress processing 92, stream processing 93; portal dashboard delivery 94—same number; data analytics processing 95; and egress and data delivery 96.
Although this disclosure describes embodiments on a cloud computing platform, implementation of embodiments as described herein are not limited to a cloud computing environment.
One of ordinary skill in the art will appreciate that the architecture of system 10 is a non-limiting example that is illustrative of at least a portion of an embodiment. As such, more or less components can be employed and/or arranged differently without departing from the scope of the innovations described herein. However, system 10 is sufficient for disclosing at least the innovations claimed herein.
Referring to
In at least one embodiment, events that can be received and/or determined from one or more event sources includes vehicle event data from one or more data sources, for example GPS devices, or location data tables provided by third party data source 15, such as OEM vehicle sensor data source 14. Vehicle event data can be ingested in database formats, for example, JSON, CSV, and XML. The vehicle event data can be ingested via APIs or other communication interfaces provided by the services and/or the Ingress Server system 100. For example, Ingress Server system 100 can offer an API Gateway 102 interface that integrates with an Ingress Server API 106 that enables Ingress Server system 100 to determine various events that can be associated with databases provided by the vehicle event source 14. An exemplary API gateway can include, for example AWS API Gateway. An exemplary hosting platform for an Ingress Server system 100 system can include Kubernetes and Docker, although other platforms and network computer configurations can be employed as well.
In at least one embodiment, the Ingress Server system 100 includes a Server 104 configured to accept raw data, for example, a Secure File Transfer Protocol Server (SFTP), an API, or other data inputs can be configured accept vehicle event data. The Ingress Server system 100 can be configured to store the raw data in data store 107 for further analysis, for example, by an Analytics Server system 500. Event data can include Ignition on, time stamp (T1 . . . TN), Ignition off, interesting event data, latitude and longitude, and Vehicle Information Number (VIN) information. Exemplary event data can include Vehicle Movement data from sources as known in the art, for example either from vehicles themselves (e.g. via GPS, API) or tables of location data provided from third party data sources 15.
In at least one embodiment, the Ingress Server system 100 is configured to clean and validate data. For example, the Ingress Server 100 can be configured to include Ingress API 106 that can validate the ingested event and location data and pass the validated location data to a server queue 108, for example, an Apache Kafka queue, which is then outputted to the Stream Processing Server 200. The server 108 can be configured to output the validated ingressed location data to the data store 107 as well. The Ingress Server can also be configured to pass invalid data to a data store 107. For example, invalid payloads can be stored in data store 107. Exemplary invalid data can include, for example, data with bad fields or unrecognized fields, or identical events.
In an embodiment, the system 10 is configured to detect and map vehicle locations with enhanced accuracy. In order to gather useful aggregates about the road network, for example expected traffic volumes and speeds across the daily/weekly cycle, the system 10 can be configured to determine how vehicles are moving through a given road network. As noted herein, a naive approach of associating or “snapping” each data point with a nearest section of a road can fail because vehicle GPS data has an inherent degree of error due to various known physical effects. Further, a road network often approaches and crosses itself in complicated geometries leading to locations with multiple road snapping candidates.
In an embodiment, the system 10 can be configured to include a base map given as a collection of line segments for road segments. The system 10 includes, for each line segment, geometrical information regarding the line segment's relation to its nearest neighbors. For each line segment, statistical information regarding expected traffic volumes and speeds is generated from an initial iteration of the process. As shown above, vehicle movement event data comprises longitude, latitude, heading, speed and time-of-day.
In an embodiment, the system 10 is configured to take a collection of line segments, which corresponds to road segments, and create an R-Tree index over the collection of line segments. R-trees are tree data structures used for spatial access methods, i.e., for indexing multi-dimensional information such as geographical coordinates, rectangles or polygons. The R-tree is configured to store spatial objects as bounding box polygons to represent, inter alia, road segments. The R-Tree is first used to find road segment candidates within a prescribed distance of a coordinate in order to snap a data point. The candidates are then further examined using a refined metric that considers event data, such as the heading, to select a road segment, which is most likely based on all known information. Event data, such as speed and/or time-of-day, can also be employed to select a road segment.
In an embodiment, the system 10 can also be configured to penalize road segments that are not aligned in the direction of travel of the given data point by adding a fixed penalty to the actual distance between the point and the road segment. This makes line segments that do not align with the direction of travel to appear further away and therefore less likely to be selected as the correct one. In cases where there is still some question over which segment is the best fit, the system 10 can be configured to weigh additional information regarding the expected speed of the given point and additional geometrical considerations before selection takes place.
The system 10 is configured to predefine distances between bounding box road segments, for example using an R-tree as described above. For precalculated distances for the road segments, the system 10 can be configured to select a nearest neighbor for a closest distance. In an embodiment, the system 10 can also be configured to add a penalty to determine if the road segment with the closest distance is the correct road segment for the vehicle.
The system 10 is configured to identify a distance between a point (lat/long) and a road segment (line segment). An Item Distance artery implementation allows any two points in distance to be identified to a road segment. In an embodiment, the system can also be configured to implement a penalty for a heading in order to override choosing a road segment based on a naive or default selection of a closest point from the lat/long data point. As noted above, a road segment can be defined as a bounding box or line segment.
For example, the system 10 can be configured to allow an angular range of deviation between a car heading and road heading to determine whether to apply a penalty in selecting the road segment. For example, where the deviation is small, no penalty is applied, as the car heading and the road heading are highly likely to be accurate when the angle of deviation is small. Thus, the system 10 can be configured to choose a smallest angle to identify a segment heading. However, if the smallest angle is less than a predetermined angle, for example in the range of 10-40 degrees out of 360 degrees, the system 10 can be configured to select that road segment or preferentially weight that road segment for selection.
In an embodiment, other event data can be employed to weight the selection of the penalty, for example the speed of vehicle (mph). For instance, when a vehicle speed indicates a high speed and a high angle of deviation, the penalty can be applied. If the road heading is more than 30 degrees from the car heading, and the speed is higher than a given speed threshold, it is highly likely that the road segment is not accurate, and so the penalty should be applied. On the other hand, if the angle of deviation between the car heading and the road heading is small and the speed is high, it is highly likely that the vehicle is indeed moving in the proper direction at that speed.
Accordingly, in an embodiment, an angle differential, for example over 30 degrees and under 180 degrees for a heading, can be employed to determine a “one way” or “wrong way” penalty using directional information from associated map data for a road segment. For example, if a closest point between the two points for selecting a road segment results in angle differential between 30 degrees to 150 degrees, and that angle would place the vehicle on the wrong direction for the segment, the system can be configured to apply a wrong way penalty.
An exemplary penalty algorithm is as follows:
The output from the algorithm comprises: a road segment chosen as the best match; a new (longitude, latitude) pair that represents the original point snapped to the chosen line segment; and the error or distance between the original point and the snapped point. As noted above, the system 10 is configured to apply a penalty to obtain the most likely road segment.
In an embodiment, the algorithm can also include a measure of confidence in the chosen road segment based on the number of other potential matches that closely match the criteria for selection. For example, a weight could comprise a road knowledge weight, for example, time-of-day, miles-per-hour and/or road type weight. For instance, a road knowledge weight might include a highway or residential road weight. Thus, if a road segment is known to be a residential segment, a selection could be weighted to penalize choosing a nearest highway segment when a vehicle is identified as going 30 miles per hour.
The Ingress Server 100 can be configured to output the stored invalid data or allow stored data to be pulled to the Analysis Server 500 from the data store 107 for analysis, for example, to improve system performance. For example, the Analysis Server 500 can be configured with diagnostic machine learning configured to perform analysis on databases of invalid data with unrecognized fields to newly identify and label fields for validated processing. The Ingress Server 100 can also be configured to pass stored ingressed location data for processing by the Analytics server 500, for example, for Journey analysis as described herein.
In an embodiment, the Ingress Server 100 is configured to process event data to derive vehicle movement data, for example speed, duration, and acceleration. For example, in an embodiment, a snapshot is taken on the event database every x number of seconds (e.g. 3 seconds). Lat/long data and time data can then be processed to derive vehicle tracking data, such as speed and acceleration, using vehicle position and time.
In an embodiment, the Ingress Server system 100 is configured to accept data from devices and third party platforms. The Ingress Server API 106 can be configured to authenticate devices and partner or third-party platforms and platform hosts to the system 10.
Accordingly, in an embodiment, the Ingress Server system 100 is configured to receive raw data and perform data quality checks for raw data and schema evaluation. Ingesting and validating raw data is the start of a data quality pipeline of quality checks for the system as shown in
In another embodiment, vehicle event data from an ingress source can include less information. For example, as shown in Table 2, the raw vehicle event data can comprise a limited number of attributes, for example, location data (longitude and latitude) and time data (timestamps).
An exemplary advantage of embodiments of the present disclosure is that information that is absent can be derived from innovative algorithms as described herein. For example, vehicle event data may not include a journey identification, or may have a journey identification that is inaccurate. Accordingly, the system 10 can be configured to derive additional vehicle event attribute data when the initially ingressed data has limited attributes. For example, the system 10 can be configured to identify a specific vehicle for ingressed vehicle event data and append a Vehicle ID or Device ID. The system 10 can thereby trace vehicle movement—including starts and stops, speed, heading, acceleration, and other attributes using, for example, only location and timestamp data associated with a Vehicle ID or Device ID.
In an embodiment, at block 702, data received can conform to externally defined schema, for example, Avro or JSON. The data can be transformed into internal schema and validated. In an embodiment, event data can be validated against an agreed schema definition before being passed on to the messaging system for downstream processing by the data quality pipeline. For example, an Apache Avro schema definition can be employed before passing the validated data on to an Apache Kafka messaging system. In another embodiment, the raw movement and event data can also be processed by a client node cluster configuration, where each client is a consumer or producer, and clusters within an instance can replicate data amongst themselves.
For example, the Ingress server system 100 can be configured with a Pulsar Client connected to an Apache Pulsar end point for a Pulsar cluster. In an embodiment, the Apache Pulsar end point keeps track of the last data read, allowing an Apache Pulsar Client to connect at any time to pick up from the last data read. In Pulsar, a “standard” consumer interface involves using “consumer” clients to listen on topics, process incoming messages, and finally acknowledge those messages when the messages have been processed. Whenever a client connects to a topic, the client automatically begins reading from the earliest unacknowledged message onward because the topic's cursor is automatically managed by a Pulsar Broker module. However, a client reader interface for the client enables the client application to manage topic cursors in a bespoke manner. For example, a Pulsar client reader can be configured to connect to a topic to specify which message the reader begins reading from when it connects to a topic. When connecting to a topic, the reader interface enables the client to begin with the earliest available message in the topic or the latest available message in the topic. The client reader can also be configured to begin at some other message between the earliest message and the latest message, for example by using a message ID to fetch messages from a persistent data store or cache.
As noted above, in at least one embodiment, the Ingress Server system 100 is configured to clean and validate data. For example, the Ingress Server system 100 can be configured include an Ingress Server API 106 that can validate the ingested vehicle event and location data and pass the validated location data to a server queue 108, for example, an Apache Kafka queue, which is then outputted to the Stream Processing Server system 200. Server 104 can be configured to output the validated ingressed location data to the data store 107 as well. The Ingress Server system 100 can also be configured to pass invalid data to a data store 107.
The map database can be, for example, a point of interest database or other map database, including public or proprietary map databases. Exemplary map databases can include extant street map data such as Geofabric for local street maps, or World Map Database. The system can be further configured to egress the data to external mapping interfaces, navigation interfaces, traffic interfaces, and connected car interfaces as described herein.
The Ingress Server system 100 can be configured to output the stored invalid data or allow stored data to be pulled to the Analysis Server system 500 from the data store 107 for analysis, for example, to improve system performance. For example, the Analysis Server system 500 can be configured with diagnostic machine learning configured to perform analysis on databases of invalid data with unrecognized fields to newly identify and label fields for validated processing. The Ingress Server system 100 can also be configured to pass stored ingressed location data for processing by the Analytics Server system 500.
As described herein, the system 10 is configured to process data in both a streaming and a batch context. In the streaming context, low latency is more important than completeness, i.e. old data need not be processed, and in fact, processing old data can have a detrimental effect as it may hold up the processing of other, more recent data. In the batch context, completeness of data is more important than low latency. Accordingly, to facilitate the processing of data in these two contexts, in an embodiment, the system 10 can default to a streaming connection that ingresses all data as soon as it is available but can also be configured to skip old data. A batch processor can be configured to fill in any gaps left by the streaming processor due to old data.
In at least one embodiment, event information can be stored in one or more data stores 107, for later processing and/or analysis. Likewise, in at least one embodiment, event data and information can be processed as it is determined or received. Also, event payload and process information can be stored in data stores, such as data store 107, for use as historical information and/or comparison information and for further processing.
In at least one embodiment, the Stream Processing Server system 200 is configured to perform vehicle event data processing.
In an embodiment, as shown at block 703 of
In an embodiment, at block 704 the system 10 is configured to perform Attribute Value Filtering. Attribute Value Filtering checks to ensure attribute values are internally set or bespoke defined ranges. For example, while a date of 1970 can pass an Attribute Bounds Filter check for a date Attribute of the event, the date is not a sensible value for vehicle tracking data. Accordingly, Attribute Value Filtering is configured to filter data older than a predefined time, for example 6 weeks or older, which can be checked and filtered. An example Attribute Bounds Filtering is given in Table 4.
At block 705, the system 10 can perform further validation on Attributes in a record to confirm that relationships between attributes of record data points are coherent. For example, a non-zero trip start event does not make logical sense for a Journey determination as described herein. Accordingly, as shown in Table 5, the system 10 can be configured to filter non-zero speed events recorded for the same Attributes for a captured timestamp and a received timestamp for a location as “TripStart” or Journey ignition on start event.
Returning to
The geohashing algorithm encodes latitude and longitude (lat/long) data from event data to a short string of n characters. In an embodiment, the geohashed lat/long data is geohashed to a shape. For example, in an embodiment, the lat/long data can be geohashed to a rectangle whose edges are proportional to the characters in the string. In an embodiment, the geohash can be encoded from to 4 to 9 characters.
A number of advantages flow from employing geohashed event data as described herein. For example, in a database, data indexed by geohash will have all points for a given rectangular area in contiguous slices, where the number of slices is determined by the geohash precision of encoding. This improves the database by allowing queries on a single index, which is much easier or faster than multiple-index queries. The geohash index structure is also useful for streamlined proximity searching, as the closest points are often among the closest geohashes.
At block 206, in at least one embodiment, the Stream Processing Server system 200 performs a location lookup. As noted above, in an embodiment, the system 10 can be configured to encode the geohash to identify a defined geographical area, for example, a country, a state, or a zip code. The system 10 can geohash the lat/long to a rectangle whose edges are proportional to the characters in the string.
For example, in an embodiment, the geohashing can be configured to encode the geohash to 5 characters, and the system 10 can be configured to identify a state to the 5-character geohashed location. For example, the geohash encoded to 5 slices or characters of precision is accurate to +/−2.5 kilometers, which is sufficient to identify a state. A geohash to 6 characters can be used to identify the geohashed location to a zip code, as it is accurate to +/−0.61 kilometers. A geohash to 4 characters can be used to identify a country. In an embodiment, the system 10 can be configured to encode the geohash to uniquely identify a vehicle with the geohashed location. In an embodiment, the system 10 can be configured to encode the geohash to 9 characters to uniquely identify a vehicle.
In an embodiment, the system 10 can be further configured to map the geohashed event data to a map database. The map database can be, for example, a point of interest database or other map database, including public or proprietary map databases as described herein. The system 10 can be further configured to produce mapping interfaces. An exemplary advantage of employing geohashing as described herein is that it allows for much faster, low latency enrichment of the vehicle event data when processed downstream. For example, geographical definitions, map data, and other enrichments are easily mapped to geohashed locations and Vehicle IDs. Feed data can be also be combined into an aggregated data set and visualized using an interface, for example a GIS visualization tool (e.g.: Mapbox, CARTO, ArcGIS, or Google Maps API) as shown in
In at least one embodiment, at block 208, the Stream Processor Server system 200 can be configured to anonymize the data to remove identifying information, for example, by removing or obscuring personally identifying information from a Vehicle Identification Number (VIN) for vehicle data in the event data. In various embodiments, event data or other data can include VIN numbers, which include numbers representing product information for the vehicle, such as make, model, and year, and also includes characters that uniquely identify the vehicle, and can be used to personally identify it to an owner. The system 10 can include, for example, an algorithm that removes the characters in the VIN that uniquely identify a vehicle from vehicle data but leaves other identifying serial numbers (e.g. for make, model and year), for example, a Squish Vin algorithm. In an embodiment, the system 10 can be configured to add a unique vehicle tag to the anonymized data. For example, the system 10 can be configured to add unique numbers, characters, or other identifying information to anonymized data so the event data for a unique vehicle can be tracked, processed and analyzed after the personally identifying information associated with the VIN has been removed. An exemplary advantage of anonymized data is that the anonymized data allows processed event data to be provided externally while still protecting personally identifying information from the data, for example as may be legally required or as may be desired by users.
In at least one embodiment, as described herein, a geohash to 9 characters can also provide unique identification of a vehicle without obtaining or needing personally identifying information such as VIN data. Vehicles can be identified via processing a database event data and geohashed to a sufficient precision to identify unique vehicles, for example to 9 characters, and the vehicle can then be identified, tracked, and their data processed as described herein.
In an embodiment, data can be processed as described herein. For example, un-aggregated data can be stored in a database (e.g. Parquet) and partitioned by time. Data can be validated in-stream and then reverse geocoded in-stream. Data enrichment, for example by vehicle type, can be performed in-stream. The vehicle event data can aggregated, for example, by region, by journey, and by date. The data can be stored in Parquet, and can also be stored in Postgres. Reference data can be applied in Parquet for in-stream merges. Other reference data can be applied in Postgres for spatial attributes.
As noted above, for real-time streaming, at block 202, the data validation filters out data that has excess latency, for example a latency over 7 seconds. However, batch data processing can run with a full set of data without gaps, and thus can include data that is not filtered for latency. For example, a batch data process for analytics as described with respect to
In an embodiment, at block 212, both the transformed location data filtered for latency and the rejected latency data are input to a server queue, for example, an Apache Kafka queue. At block 214, the Stream Processing server system 200 can split the data into a data set including full data 216—the transformed location data filtered for latency and the rejected latency data—and another data set of the transformed location data 222. The full data 216 is stored in data store 107 for access or delivery to the Analytics Server system 500, while the filtered transformed location data is delivered to the Egress Server system 400. In another embodiment, the full data set or portions thereof including the rejected data can also be delivered to the Egress Server system 400 for third party platforms for their own use and analysis. In such an embodiment, at block 213 transformed location data filtered for latency and the rejected latency data can be provided directly to the Egress Server system 400.
For example, in an embodiment, the Egress Server system 400 can provide pattern data based on filters provided by the system 10. For example, the system 10 can be configured to provide a geofence filter 412 to filter event data for a given location or locations. As will be appreciated, geofencing can be configured to bound and process journey and event data as described herein for numerous patterns and configurations. For example, in an embodiment, the Egress Server system 400 can be configured to provide a “Parking” filter configured restrict the data to the start and end of journey (Ignition—key on/off events) within the longitude/latitudes provided or selected by a user. Further filters or exceptions for this data can be configured, for example by state (state code or lat/long). The system 10 can also be configured with a “Traffic” filter to provide traffic pattern data, for example, with given states and lat/long bounding boxes excluded from the filters.
In an embodiment, the Egress Server 400 can be configured to process data with low-latency algorithms configured to maintain and improve low latency real-time throughput. The algorithms can be configured to process the data for low-latency file output that can populate downstream interfaces requiring targeted, real-time data that does not clog computational resources or render them inoperable. In an embodiment, the system 10 is configured to provide low latency average road speed data for road segments for output in virtually real time from a live vehicle movement data stream from the Stream Processing Server 200. The Egress Server 400 can also be configured to delete raw data in order and provide lightweight data packages to partners 20 and configured for downstream interfaces, for example via the Push Server.
As shown in
In an embodiment, at block 412 the system is configured to track data points for a vehicle as described herein with respect to
In an example shown and described with reference to
Each road segment comprises a speed threshold. Speed thresholds are defined per segment, so each segment can have a different speed threshold. The speed threshold is configured as a threshold for counting data points where the vehicle speed is above the threshold. A speed threshold could be defined to correspond, for example, to posted speed limits for the road the segment corresponds to, though it need not be so defined. A speed threshold can be set for any speed, and thus defined to capture speed data for any purpose.
The system is configured to calculate at segment traversal for a vehicle by monitoring a plurality of data points from the vehicle event data. A segment traversal is when a vehicle passes all the way through a road segment from one end to the other.
In an embodiment, as shown in
At any point where a vehicle speed crosses a speed threshold, the system is configured to increment a speeding count for the vehicle for that point. As shown in
At calculation triggering data point F, the system then calculates data for a segment event record for segment 2. The segment event record includes a traversal time and average speed for segment 2. A traversal time is the amount of time taken for a segment traversal. Traversal time is the captured time stamp of the first data point exiting outside the road segment minus the captured time stamp of the first data point inside the road segment in milliseconds. For example, in
Average speed is the segment distance divided by the traversal time. The average speed can be multiplied to obtain a desired order of magnitude. For a given capture rate for vehicle movement data points (e.g., 3 seconds), the exact distance driven will vary by record, and a fixed distance can be used when calculating average speed through the segment. For example, at 50 MPH a vehicle will have travelled approximately 73.3 yards in 3 seconds. In the example shown in
A worked example for a segment event record as shown in
Returning to
As shown in
As shown in
As explained herein, improved latency is not incidental to the design and implementation of the algorithm and segment event record container employed to egress segment events, as low latency is an important technical feature of the system. Further, light segment event record containers allow downstream consoles, for example traffic management consoles, to operate. For example, at block 416 of
As shown in the example above, at least three consecutive segments are employed to calculate an average speed for a given segment.
In an embodiment, the system is configured to end vehicle tracking for a number of exceptions. As shown at block 431, in an embodiment, the system is configured to stop tracking vehicle event data if the system has not received a data point for that vehicle in 30 minutes. In this case, the system proceeds to block 440 and stops tracking that vehicle.
At block 433, the system is also configured to delete or ignore a vehicle that returns to a previous segment (i.e. a U-turn). In this case, the system proceeds to block 440 and stops tracking that vehicle.
In an embodiment, at block 437 and block 439, if the event data includes an engine on or an engine off event, the road segment data for that segment is deleted. The presence of an engine on/off event signals that there was a loss of connectivity when a vehicle stopped in the segment. In this case, the system again proceeds to block 440 and stops tracking that vehicle.
As will be appreciated, the system 10 provides extremely accurate insights at low latency, and can account for variation and possible bias of vehicles, additionally ensuring that edge cases do not introduce bias. Edge cases include: late or out of order data, wide variations of speed due to multi-lane interstates, and low volume vehicles that skew results.
In an embodiment, the Analytics Server system 500 can be configured to accept vehicle event payload and processed information, which can be stored in data stores, such as data stores 107. As shown in
Accordingly, at block 502 of
Referring to block 504 of
At block 510, in at least one embodiment, the Analytics Server 500 can perform a Journey Segmentation analysis of the event data. At block 512, the Analytics Server 500 is configured to perform calculations to qualify a Journey from event information.
Returning to
In at least one embodiment, a Journey can comprise one or more Journey Segments from a starting point to a final destination. A Journey Segment comprises a distance and a duration of travel between engine on/start movement and engine off/stop movement events for a vehicle.
However, a real driver may have one or more stops when travelling to a destination. A Journey can have two or more Journey Segments, such as when there is a trip with multiple stops. For example, a driver may need to stop for fuel when travelling from home to work or stop at a traffic light. As such, a problem and challenge in vehicle event analysis includes developing accurate vehicle tracking for embodiments as described herein. While other Journey algorithms or processes have been employed in the art, for example reverse engineering a journey from a known destination of an identified vehicle, the present disclosure includes embodiments and algorithms that have been developed and advantageously implemented for agnostic vehicle tracking using the technology described herein, including the data analysis, databases, interfaces, data processing, and other technological products.
At block 512, the Analytics Server 500 is configured to perform calculations to qualify a Journey from event information. In an embodiment, the system 10 is configured with Journey detection criteria, including a duration criterion, a distance criterion, and a dwell time criterion. In at least one embodiment, the duration criterion includes a minimum duration criterion, where a minimum duration of travel is required for the system to include a Journey Segment in a Journey. A minimum duration of travel after engine on or a start movement can comprise a duration of time for travel, for example, from about 60 to about 90 seconds. In an exemplary embodiment, the system 10 can be configured require a vehicle travel more than 60 seconds for it to be included as a Journey Segment. For example, if an (1) engine on/ignition event or (2) an identified vehicle's first movement after a known last movement (e.g. from a previous trip or journey) or (3) a newly identified vehicle's first movement is identified for a vehicle, and the event is followed by a short duration of travel of less than 60 seconds, the system 10 is configured to exclude this Journey Segment from a Journey determination. The system 10 is configured to determine that the vehicle's short duration of movement is not a Journey start or destination.
In an embodiment, the Journey detection criterion includes a distance of travel criterion, for example 200 meters. The system 10 can be configured to exclude distances of 200 meters or less from a Journey segment. A minimum distance of travel criterion can comprise a predetermined duration of distance for travel, for example, from about 100 meters to about 300 meters. The minimum distance x (e.g. 200 meters) can be defined to an index including about 50% tolerance of the minimum distance x.
In an embodiment, a dwell time criterion can include a stop time for a vehicle. For example, a dwell time criterion can be from about 30 to about 90 seconds. A maximum dwell time can comprise a duration of stopping between an engine off/stop movement and engine on/start movement for the same vehicle, for example, from about 20 to about 120 seconds. For example, if the system 10 determines a vehicle is stopped or its engine is off for less than 30 seconds, the system can be configured not to include that stop period as the end of a Journey or in a Journey object.
As described above, in an embodiment, the system 10 is configured to process vehicle event data to determine if one or more Journey Segments comprise a Journey for a vehicle. For example, an engine on or start movement event can be followed by a distance exceeding a distance criterion (e.g. over 200 meters). Thus, the system's duration criterion does identify this segment for a Journey. However, if the car stops thereafter and continues to stay stationary for over 30 seconds, the system 10 is configured not to count that as a segment for a Journey. If the vehicle subsequently stops for less than 30 seconds and then moves again, the Dwell time criterion is met, and the system 10 is configured to include that Journey Segment in the Journey for that vehicle's travel to its final destination. Thus, the algorithm can join a plurality of Journey Segments for a Journey or a Journey object for an everyday real time drive a destination, for example, when a driver turns a car on (engine on/start movement) at home, drives for 10 miles (Distance criterion), stops at a stop light for 29 seconds, travels on to a final destination at work (engine off/stop movement). However, the system 10 is configured to ignore events that are unlikely to represent an interruption in a Journey, for example stopping at a stop light for 29 seconds (Dwell criterion) or movement less than 200 meters (Distance criterion) or less than 60 seconds (Duration criterion).
In an embodiment, the system 10 can include a plurality of criteria for each of the dwell criterion, the distance criterion, or the time criterion, for example, based on variable data. Thus, the algorithm can join a plurality of Journey Segments for a Journey for a common real time drive to a destination where additional data is known about the vehicle and the location. For example, if a vehicle is identified as a road legal electric vehicle such as an electric car, the dwell criteria can include a dwell time maximum criterion of 20 minutes at a location identified as an electric charging station. Thus, the dwell time can be extended up to between 2-20 minutes, based on, for example, other data about the location (e.g., data indicating the stop is a point of interest such as a gas station, rest area, or restaurant). The system 10 can be configured to identify a Journey when a driver of an electric car turns the car on (engine on or first movement) at home, drives for 100 miles (Distance criterion) to a charging station for charging (engine off/stop movement, 12 minutes, Dwell criterion, variable, charging station), then starts again (engine on/start movement) and travels on to a final destination at a sales meeting (engine off/stop movement). Accordingly, as will be appreciated, each of the criteria above can be configured to be variable depending on, inter alia, knowledge derived or obtained about an event vehicle data point.
In an embodiment, the system 10 is configured to identify candidate chains of Journey segments for a given device according to the criteria described above. Also, a compound Journey object can be instantiated with its start being the beginning of the chain and its end being the end of the final segment in the chain. A separate table of Journey objects can be extracted from event data and derived compound Journeys can be generated into a further table. In an embodiment, a data set including all engine on/engine off or start movement/stop movement events are identified to a unique vehicle ID. For example, each of the engine on/engine off or start movement/stop movement events for a vehicle can be placed on a single row including the candidate Journey segments. Then, row of engine on/engine off or start movement/stop movement events can be processed by each of the distance criterion, duration criterion, and dwell criterion to determine which Journey segments can be included or excluded from a Journey determination for a Journey object. In an embodiment, the system 10 can generate a further Journey Table, which is populated with Journey objects as determined from the events for the vehicle that meet the Journey criteria above.
In at least one embodiment, at block 514, the system 10 is configured to provide active vehicle detection by analyzing a database of vehicle event data and the summarizing of a journey of points into a Journey object with attributes, such as start time, end time, start location, end location, data point count, average interval and the like. In an embodiment, Journey objects can be put into a separate data table for processing.
In an exemplary embodiment, the system 10 can be configured to perform vehicle tracking without the need for pre-identification of the vehicle (e.g. by a VIN number). As described above, geohashing can be employed on a database of event data to geohash data to a precision of 9 characters, which corresponds to a shape sufficient to uniquely correlate the event to a vehicle. In an embodiment, the active vehicle detection comprises identifying a vehicle path from a plurality of the events over a period of time. In an embodiment, the active vehicle detection can comprise identifying the vehicle path from the plurality of events over the period of a day (24 hours). The identification comprises using, for example, a connected components algorithm. In an embodiment, the connected components algorithm is employed to identify a vehicle path in a directed graph including the day of vehicle events, in which in the graph, a node is a vehicle and a connection between nodes is the identified vehicle path. For example, a graph of journey starts and journey ends is created, where nodes represent starts and ends, and edges are journeys undertaken by a vehicle. At each edge, starts and ends are sorted temporally. Edges are created to connect ends to the next start at that node, ordered by time. Nodes are 9 digit geohashes of GPS coordinates. A connected components algorithm finds the set of nodes and edges that are connected and, a generated device ID at the start of a day is passed along the determined subgraph to uniquely identify the journeys (edges) as being undertaken by the same vehicle.
An exemplary advantage of this approach is it obviates the need for pre-identification of vehicles to event data. Journey Segments from vehicle paths meeting Journey criteria as described herein can be employed to detect Journeys and exclude non-qualifying Journey events as described above. For example, a geohash encoded to 9 digits (highest resolution) for event data showing a vehicle had a stop movement/engine off to start movement/engine on event within x seconds of each other (30 seconds) can be deemed the same vehicle for a Journey. For a sequence of arrives and leaves, a Journey can be calculated as the shortest path of Journey Segments through the graph.
In at least one embodiment, at block 515, the system 10 can be configured to store the event data and Journey determination data in a data warehouse 517. Data can be stored in a database format. In an embodiment, a time column can be added to the processed data. In an embodiment, the database can also comprise Point of Interest (POI) data.
The Analytics Server system 500 can include an analytics server component 516 to perform data analysis on data stored in the data warehouse 517, for example a Spark analytics cluster. The Analytics Server system 500 can be configured to perform evaluation 530, clustering 531, demographic analysis 532, and bespoke analysis 533. For example, a date column and hour column can be added to data to processed Journey data and location data stored in the warehouse 517. This can be employed for bespoke analysis 533, for example, determining how many vehicles at intersection x by date and time. The system 10 can also be configured to provide bespoke analysis 533 at the Egress Server system 400, as described with respect to
In an embodiment, a geospatial index row can be added to stored warehouse 517 data, for example, to perform hyper local targeting or speeding up ad hoc queries on geohashed data. For example, location data resolved to 4 decimals or characters can correspond to a resolution of 20 meters or under.
In at least one embodiment, at block 514, the system is configured to provide active vehicle detection by analyzing a database of vehicle event data and the summarizing of a journey of points into a Journey object with attributes such as start time, end time, start location, end location, data point count, average interval and the like. In an embodiment, Journey objects can be put into a separate data table for processing.
In an exemplary embodiment, the system 10 can be configured to perform vehicle tracking without the need for pre-identification of the vehicle (e.g. by a VIN number). As described above, geohashing can be employed on a database of event data to geohash data to a precision of 9 characters, which corresponds to a shape sufficient to uniquely correlate the event to a vehicle. In an embodiment, the active vehicle detection comprises identifying a vehicle path from a plurality of the events over a period of time. In an embodiment, the active vehicle detection can comprise identifying the vehicle path from the plurality of events over the period of a day (24 hours). The identification comprises using, for example, a connected components algorithm. In an embodiment, the connected components algorithm is employed to identify a vehicle path in a directed graph including the day of vehicle events, in which in the graph, a node is a vehicle and a connection between nodes is the identified vehicle path. For example, a graph of journey starts and journey ends is created, where nodes represent starts and ends, and edges are journeys undertaken by a vehicle. At each edge, starts and ends are sorted temporally. Edges are created to connect ends to the next start at that node, ordered by time. Nodes are 9 digit geohashes of GPS coordinates. A connected components algorithm finds the set of nodes and edges that are connected and, a generated device ID at the start of a day is passed along the determined subgraph to uniquely identify the journeys (edges) as being undertaken by the same vehicle.
An exemplary advantage of this approach is it obviates the need for pre-identification of vehicles to event data. Journey Segments from vehicle paths meeting Journey criteria as described herein can be employed to detect Journeys and exclude non-qualifying Journey events as described above. For example, a geohash encoded to 9 digits (highest resolution) for event data showing a vehicle had a stop movement/engine off to start movement/engine on event within x seconds of each other (30 seconds) can be deemed the same vehicle for a Journey. For a sequence of arrives and leaves, a Journey can be calculated as the shortest path of Journey Segments through the graph.
In at least one embodiment, at block 515, the system 10 can be configured to store the event data and Journey determination data in a data warehouse 517. Data can be stored in a database format. In an embodiment, a time column can be added to the processed data. In an embodiment, the database can also comprise Point of Interest (POI) data.
The Analytics Server system 500 can include an analytics server component 516 to perform data analysis on data stored in the data warehouse 517, for example a Spark analytics cluster. The Analytics Server system 500 can be configured to perform evaluation 530, clustering 531, demographic analysis 532, and bespoke analysis 533. For example, a date column and hour column can be added to data to processed Journey data and location data stored in the warehouse 517. This can be employed for bespoke analysis 533, for example, determining how many vehicles at intersection x by date and time. The system 10 can also be configured to provide bespoke analysis 533 at the Egress Server system 400, as described with respect to
In an embodiment, a geospatial index row can be added to stored warehouse 517 data, for example, to perform hyper local targeting or speeding up ad hoc queries on geohashed data. For example, location data resolved to 4 decimals or characters can correspond to a resolution of 20 meters or under. The Analytics Server 500 can be configured with diagnostic machine learning configured to perform analysis on databases of invalid data with unrecognized fields to newly identify and label fields for validated processing.
In an embodiment, the system 10 can be configured to process vehicle event data to provide enhanced insights and efficient processing. Exemplary processes and systems for processing event data comprise:
The Analytics Server system 500 can be configured to perform road snapping as described with respect to the Ingress Server system 100 hereinabove. The algorithm as described above advantageously can use individual points for snapping, and extracts as much information as possible from each data point by comparing each data point to road geometry. The data point can also be compared to statistics formed from aggregated data. In an embodiment, the snapping algorithm is implemented at an ingress server to provide, inter alia, advantages in substantially real-time, low latency feeds. In an embodiment, the snapping algorithm can also be provided at the Stream Processing server system 200, Egress Server system 400, or Analytics Server system 500. In an embodiment, the system 10 can be further configured to map the event data to a map database as described herein.
In an embodiment, a base map is given as a collection of line segments. The system 10 can be configured to include a base map given as a collection of line segments for road segment, for example employing an R-Tree index as described herein. As disclosed herein, the system 10 includes, for each line segment, geometrical information regarding the line segment's relation to its nearest neighbors. For each line segment, statistical information regarding expected traffic volumes and speeds is generated from an initial iteration of the process. Vehicle movement event data comprises longitude, latitude, heading, speed, and time-of-day. As described herein, vehicle movement event data is geohashed, for example to a 6 character geohash. Vehicle movement data enriched with the geohash can be map-matched to the base map.
One exemplary advantage of the system 10 is that among large data set of vehicle movement data, the system can be configured to be highly selective and yet correct map interfaces at a high degree of resolution. For example, the system 10 can identify and correct map data and interfaces from at least and as many as 10 million geohashes in the United States.
Another exemplary advantage is map interfaces and navigation systems can be improved to accurately navigate vehicles.
In an embodiment, from the map matching enrichment, the Analytics Server 500 can be configured to identify vehicle event movement data points that do not correlate well on the base map. For example, the distance of a given vehicle's snapped point from a road segment centerline can be high and/or have a large discrepancy in heading between and original vehicle heading and a snapped point. These vehicle event data points are flagged as “wayward”. The Analytics Server 500 can be configured to identify vehicle event movement data points having a threshold speed, for example greater than 30 kph, to filter out vehicle events less likely to correlate to road segments (e.g. parking lots, driveways, off road driving). The Analytics Server 500 can be configured to execute periodic (e.g. daily) aggregates for each 6 character geohash containing a measure of the proportion of “wayward” points.
The Analytics Server 500 identifies geohashes with a high proportion of wayward points. As the wayward points are active vehicle data, a large proportion of such wayward points indicates that general vehicle location is not well captured by the underlying base map. The Analytics Server 500 is configured to identify and pass a geohash with a given proportion of wayward points to the next step of the algorithm. An example of a road map with a high proportion of wayward points is shown in
A clustering algorithm is run on all wayward points in a given, qualified geohash to remove noise points and further retain only clusters meeting a threshold on both the number of points they contain and their geographical extent, thus avoiding small clusters around junctions. Threshold criteria for clusters in a qualified geohash can include, for example, at least 3 data points, clusters having at least a 10 meter radius, clusters that are at least 50 meters in length, and/or clusters containing data points from at least 2 distinct vehicles. An exemplary clustering algorithm employed a cluster threshold of 5 data points within a 15 meter radius and having data points from at least 3 distinct vehicles, where only clusters that grew to over 60 meters in length were retained.
The clusters meeting this test are the outputs of the algorithm. Clusters show real vehicle activity that does not correlate to the existing map data and map interface. Where a cluster indicates the road shape is inaccurate, the system can be configured to correct the shape of the road. For example, the cluster may indicate a road segment should be curved in a geohash where map data indicates it is straight. As shown in
In one embodiment, the wayward points may be determined when snapping vehicle movement event data to a base map or road segment, such as by incorporating the wayward point detecting into the snapping routine or as a concurrent separate routine. In this way, the wayward point determination may be characterized as an ad hoc process. In another embodiment, the wayward points may be determined after snapping vehicle movement event data to a base map or road segment has been carried out. For example, wayward points within a geographic region, such as that defined by/corresponding to a predetermined geohash, may be determined by analyzing already-snapped vehicle movement event data and the corresponding road segment to which it has been snapped. This may be characterized as a batch process.
In an embodiment, the system can be configured to generate a lane index. Through map matching, the system can be configured to analyze vehicle event data to locate each floating car data point onto a road segment. Each point has associated with it a distance it has been moved in order to make the match. As explained above, roads are represented as a single line segment in the map, so a match distance can show the presence of lanes on a road.
As will be appreciated, GPS vehicle event data has signal noise associated with it, and vehicles are not always driven in the center of a lane. As shown in the graph of
As shown in
As shown in
As shown in
In this way, the model can predict the number of lanes. It was discovered that the model works well even when sampling down to the order of hundreds of data points (shown as 500 data points in
In an embodiment, the Analytics Server 500 can be configured to identify vehicles travelling along a section of roadway in the opposite direction to that expected.
Proceeding from the map matching enrichment described above, the Analytics Server 500 can be configured to identify, from wayward points, vehicle event movement data points that can correlate well on the base map, but have a large heading discrepancy. For example, the Analytics Server 500 can be configured to identify when distance of a vehicle's snapped point from a road segment centerline correlates well, but has a large discrepancy in a heading between an original vehicle heading and a snapped point. From the map matching enrichment, the Analytics Server 500 is configured to identify points from vehicle location event data for vehicles that have a speed of greater than a given speed threshold, for example, 30 kph, and are also are well-located according to distance from centerline, but differ by 180 degrees between the vehicle heading and a snapped heading. The system is configured to identify these points as “wrong way” for the vehicle. When a trace of datapoints for a given vehicle contains a continuous section of “wrong way” points, the system can be configured to identify that the vehicle has been travelling in the wrong direction.
In another embodiment, proceeding from the map matching enrichment described above, the Analytics Server 500 or Egress Server 400 can be configured to determine accurate road speeds from vehicle event movement data points. As shown in
As shown in
At block 458, the system 10 is configured to, for each segment ID and time period, to output: a calculated mean flow rate of traffic, a number of vehicles observed, and a transit time.
By summing estimated transit times over a sequence of road segments, for example as shown with respect to
By the use of a harmonic mean over pooled vehicle speeds, the system 10 advantageously ensures that the reported flow rate value tends to favor lower estimates. The system 10 is thus configured to be more responsive to sudden reductions in speed in observed vehicle flow and diminishes the effect of higher speed vehicles pulling the overall average up. It was found that in free flow traffic, where speed variance is low, the harmonic mean value approached the arithmetic mean, but as variance increases and traffic flow instability ensued, the harmonic mean value dropped accordingly. Accordingly, the system 10 is configured to provide accurate traffic notifications, as information on low speeds is more prescient than high speeds. Thus, the system 10 measurements are configured to underestimate the average, and thereby are more responsive and more accurate.
It was also found that the use of a harmonic mean provided an improved result over usage of speed traps where instantaneous speed is captured, because instant capture data points failed to account for upstream and downstream changes in velocity until they are observed at the trap.
The system 10 thus advantageously can measure traffic flow accurately and frequently in order to both build up a dataset of historical flow rates for modeling and forecasting purposes, and also provide timely notifications of unexpected changes in flow rate (e.g. for mapping and traffic interfaces). Accordingly, the system 10 can be configured to provide the speed calculation as described with respect to
In an embodiment, the system can be configured to employ point-snapping and a journey segmentation analysis to advantageously perform automated identification of turn counts and/or turn ratios at intersections. In an embodiment, a map-matched journey segment can include a short section of road, where at either end there is either an intersection or a dead end.
The raw point data is first passed through the journey segmentation and aggregation process, for example as described with respect to
This journey trace is then input into a journey snapping algorithm. Journey segments for journeys as described with respect to
A consecutive pair of journey segments in a journey trace can describe a transition through an intersection. For a consecutive pair of journey segments describing such a transition, the first segment describes the path into the intersection and the second segment describes the path out of the intersection. Each journey trace can then be divided into consecutive pairs of segments. The system can then to count how many times each pairing occurs in a given time frame. The system thereby provides a count of how many of each type of transition is made through a given intersection on a per vehicle basis. The system can then group the transition types by intersection to get the turn ratios for each different transition through an intersection.
For example, as shown on the mapping interfaces of
As shown in
An intersection can have many types of turns. For example, at a 4 way intersection of two streets, a vehicle can, for each of the 4 incoming roads, go straight through or turn left or right. As will be appreciated, types of turns are identified and counted by journey analysis of the data itself. As shown in
As will be appreciated, the system can be configured to count turns and turn ratios using mass vehicle event data, including historical data going back years, with no implementation of hardware or personnel on road networks as is conventionally done. For example, the system as described includes vehicle event data going back at least two years covering 95% of US road networks.
The low latency provides a super-fast connection delivering information from vehicle source to end-user customer. Further data capture has a high capture rate of 3 seconds per data point, capturing up to, for example, 330 billion data points per month. As described herein, data is precise to lane-level with location data and 95% accurate to within a 3-meter radius, the size of a typical car.
In at least one embodiment, a dashboard display can render a display of the information produced by the other components of the system 10. In at least one embodiment, dashboard display can be presented on a client computer accessed over network. In at least one embodiment, user interfaces can be employed without departing from the spirit and/or scope of the claimed subject matter. Such user interfaces can have any number of user interface elements, which can be arranged in various ways. In some embodiments, user interfaces can be generated using web pages, mobile applications, GIS visualization tools, mapping interfaces, emails, file servers, PDF documents, text messages, or the like. In at least one embodiment, Ingress Server system 100, Stream Processing Server system 200, Egress Server system 400, Analytics Server system 500, or Portal Server system 600 can include processes and/or API's for generating user interfaces.
For example, as shown in the flow chart 800 of
Embodiments described with respect to systems 10, 50, 100, 200, 400, 500, 600, 700, 800 and 900 are described in conjunction with
It will be understood that each block of the flowchart illustration, and combinations of blocks in the flowchart illustration, can be implemented by computer program instructions. These program instructions can be provided to a processor to produce a machine, such that the instructions, which execute on the processor, create means for implementing the actions specified in the flowchart block or blocks. The computer program instructions can be executed by a processor to cause a series of operational steps to be performed by the processor to produce a computer-implemented process such that the instructions, which execute on the processor to provide steps for implementing the actions specified in the flowchart block or blocks. The computer program instructions can also cause at least some of the operational steps shown in the blocks of the flowchart to be performed in parallel. Moreover, some of the steps can also be performed across more than one processor, such as might arise in a multi-processor computer system or even a group of multiple computer systems. In addition, one or more blocks or combinations of blocks in the flowchart illustration can also be performed concurrently with other blocks or combinations of blocks, or even in a different sequence than illustrated without departing from the scope or spirit of the disclosure.
Accordingly, blocks of the flowchart illustration support combinations for performing the specified actions, combinations of steps for performing the specified actions and program instruction means for performing the specified actions. It will also be understood that each block of the flowchart illustration, and combinations of blocks in the flowchart illustration, can be implemented by special purpose hardware-based systems, which perform the specified actions or steps, or combinations of special purpose hardware and computer instructions. The foregoing example should not be construed as limiting and/or exhaustive, but rather, an illustrative use case to show an implementation of at least one of the various embodiments.
All references to processing or executing instructions by a processor means that said processing or executing instructions is performed by at least one processor and, in at least such implementations and according to some embodiments, said processing or executing instructions is performed by a plurality of processors, any or all of which may be co-located or remotely located. Any one or more of the processors discussed herein are electronic processors that may be implemented as any suitable electronic hardware that is capable of processing computer instructions and may be selected based on the application in which it is to be used. Examples of types of electronic processors that may be used include central processing units (CPUs), graphics processing units (GPUs), field-programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), microprocessors, microcontrollers, etc. Any one or more of the computer-readable memory discussed herein may be implemented as any suitable type of non-transitory memory that is capable of storing data or information in a non-volatile manner and in an electronic form so that the stored data or information is consumable by the electronic processor.
The memory may be any a variety of different electronic memory types and may be selected based on the application in which it is to be used. Examples of types of memory that may be used include including magnetic or optical disc drives, ROM (read-only memory), solid-state drives (SSDs) (including other solid-state storage such as solid-state hybrid drives (SSHDs)), other types of flash memory, hard disk drives (HDDs), non-volatile random access memory (NVRAM), etc. It should be appreciated that the computers or servers may include other memory, such as volatile RAM that is used by the electronic processor, and/or may include multiple processors.
It is to be understood that the foregoing description is of one or more embodiments of the invention. The invention is not limited to the particular embodiment(s) disclosed herein, but rather is defined solely by the claims below. Furthermore, the statements contained in the foregoing description relate to the disclosed embodiment(s) and are not to be construed as limitations on the scope of the invention or on the definition of terms used in the claims, except where a term or phrase is expressly defined above. Various other embodiments and various changes and modifications to the disclosed embodiment(s) will become apparent to those skilled in the art.
As used in this specification and claims, the terms “e.g.,” “for example,” “for instance,” “such as,” and “like,” and the verbs “comprising,” “having,” “including,” and their other verb forms, when used in conjunction with a listing of one or more components or other items, are each to be construed as open-ended, meaning that the listing is not to be considered as excluding other, additional components or items. Other terms are to be construed using their broadest reasonable meaning unless they are used in a context that requires a different interpretation. In addition, the term “and/or” is to be construed as an inclusive OR. Therefore, for example, the phrase “A, B, and/or C” is to be interpreted as covering all of the following: “A”; “B”; “C”; “A and B”; “A and C”; “B and C”; and “A, B, and C.”
| Number | Date | Country | |
|---|---|---|---|
| 63137493 | Jan 2021 | US |
