Vehicle event playback apparatus and methods

Description

FIELD

The following invention disclosure is generally concerned with multi-media playback systems and specifically concerned with playback systems for vehicle event record playback and analysis.

BACKGROUND

Multi-media players are known. Multi-media players are designed to run on computer systems and play back electronic audio and video files of specific types. The multi-media players typically include user interfaces for users to control playback. For example, a user may control ‘Windows Media Player®’ to play video encoded media files consistent with a format defined in the MP4 standard.

In some multi-media players the user interface controls are customized based on a user's internet browsing habits, computer system characteristics, and network authentication information. In some multi-media players, a user may control the visual appearance of the information displayed to the user.

SUMMARY

One aspect of the disclosure relates to vehicle event record playback systems including devices and methods. It is a primary function of these systems to provide review and analysis means suited and arranged in view of information captured by vehicle event recorders, active safety systems, and/or other systems installed in and/or coupled to a vehicle. In contrast to the prior art, systems presented here do not suffer limitations which prevent their use for synchronized display of data captured by vehicle event recorders, active safety systems, and/or other systems.

Vehicle event recorders may produce event record datasets in response to anomalies and/or other events detected during vehicle operation. Active safety systems may alert a vehicle driver to potentially dangerous conditions detected in and/or around a vehicle during operation of the vehicle. Because event record datasets and/or information generated by the active safety system may contain considerable amounts of data in many data formats and arrangements, they may be difficult to parse, read, interpret, and/or use. In known systems, some vehicle event recorder and/or active safety system data was manually reviewed and certain data subsets of greatest importance were manually developed from raw data into visual presentations by way of standard software such as spreadsheet plotting utilities.

Vehicle event data playback systems described herein may include advanced playback of vehicle event recorder data, active safety system data, vehicle subsystem data and/or other data. The playback of the vehicle event recorder data, the active safety system data, the vehicle subsystem data, and/or other data may be in real time, and/or may be played back at a later time. The playback of the vehicle event recorder data may be played back via a user interface located in the vehicle, a remotely located user interface, a remotely located computing device, and/or via other systems. These systems may include visual devices arranged to express vehicle performance data in graphical, image and/or alphanumeric forms. These systems may be arranged to present data in logical arrangements whereby many different types of datasets collected at various vehicle sensors may be presented in time synchronized schemes along with a plurality of complementary data. Data presented in this fashion may permit greater analysis and may enable reviewers to devise coaching feedback, for example.

In a first example, vehicle performance data may be collected by way of a vehicle's onboard electronic control unit (ECU). The ECU may include an engine control module (ECM), for example. The data from the ECU may be included in, and/or associated with other data collected at the same (or nearly same) instant in time from other sources such as data from aftermarket sensors (e.g., an accelerometer) coupled with the vehicle, data from an active safety system installed in and/or coupled with the vehicle, and/or data from other sources that generate information relevant to vehicle operation. An event record may be prepared in a manner where at least some data measurements are associated with a timestamp or other time synchronization scheme. In this way, synchronized playback of data from a plurality of data sources may be enabled.

On playback, a human reviewer may be presented with simultaneous views and expressions of multiple data elements. The individual data elements may be presented in a visual nature on a common display field of a graphical user interface (GUI). Since these data elements often have a high level of interdependence, simultaneous expression of their instantaneous values in a time sequenced series playback enables may enhance vehicle event record review.

Graphical user interfaces of these systems may include control objects. The control objects may be responsive to data values of vehicle event record datasets. The control objects may be expressed visually. A control object may be a visual element in the graphical user interface that is associated with data and/or a data stream from one or more sensors. Control objects may include parametric input couplings, data interfaces, underlying logic code which defines performance, a visual expression, an event set, triggers, an instantaneous state, and/or other control objects. Combinations of various related control objects may operate together in synchronization to characterize and/or describe actions and/or states associated with a vehicle and its many subsystems during a recorded event.

The control objects may be arranged to visually express data collected from vehicle subsystems in graphical and/or alphanumeric forms. In some implementations, the control objects may visually express data associated with standard ECU systems, an active safety system, a vehicle event recorder, and/or other systems, and present that data in one or more visual forms in conjunction with simultaneous playback of related video, for example.

The control objects of the graphical user interface may be coupled to an event timeline whereby presentation of data by the control objects conforms to the event timing. An event timeline may have a start-time, an end-time, and/or a continuous period of time therebetween. An event record dataset may include an event timeline definition for events recorded at a vehicle event recorder. For example, the controls presented by the GUI may be connected to a single timeline and that timeline may be coupled to the event time.

In some implementations of these systems, ‘video playback’ may be an included feature. Accordingly, video players may generally be included as a primary element of these graphical user interfaces. Video player systems of these devices may include those suitable for playback of common video captured at a vehicle event recorder video camera. These video players may also be suitable for playback of virtual video. Virtual video may include image series playback where the images are from sources other than a vehicle event recorder. These may include forward and aft video camera views, street view image series, mapview image series, and even bird's eye view playback. Video playback may be synchronized with the ECU data, active safety system data, and/or vehicle event recorder data, for example.

Other control objects found in the playback apparatus may include graphical representations of acceleration data in one or more coupled controls. One feature may relate to a notation facility which may be coupled to an event timeline. The notation facility permits replay of manually prepared notes to which a time Association has been made and assigned in synchronization with event replay.

These and other objects, features, and characteristics of the system and/or method disclosed herein, as well as the methods of operation and functions of the related elements of structure and the combination of parts and economies of manufacture, will become more apparent upon consideration of the following description and the appended claims with reference to the accompanying drawings, all of which form a part of this specification, wherein like reference numerals designate corresponding parts in the various figures. It is to be expressly understood, however, that the drawings are for the purpose of illustration and description only and are not intended as a definition of the limits of the invention. As used in the specification and in the claims, the singular form of “a”, “an”, and “the” include plural referents unless the context clearly dictates otherwise.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other features, aspects, and advantages of the present invention will become better understood with regard to the following description, appended claims and drawings where:

FIGS. 1A-1G illustrate examples of a view of a graphical user interface which illustrates a version of these systems;

FIG. 2 presents a timeline control time synchronized with images presented in related image control objects;

FIG. 3 illustrates side-by-side, time synchronized video players coupled to two video data sources related via a vehicle event record dataset;

FIGS. 4-6 show additional versions of side-by-side time synchronized video players similarly coupled to related video from sources other than vehicle event recorders;

FIG. 7 illustrates coupled control objects designed to express data both graphically and alphanumerically, wherein the control objects may be coupled to, and/or responsive to a vehicle event timeline;

FIG. 8 includes illustrations of a version of vehicle speed and engine speed control objects synchronized and commonly coupled to a timeline control;

FIG. 9 is a drawing of a plurality of control objects which are bound to standard vehicle sensors via on-board diagnostics and/or engine control module systems of a subject vehicle;

FIG. 10 presents a time responsive notation field type control object and its contents which are bound and responsive to an event timeline; and

FIG. 11 indicates additional auxiliary elements which may be included in some of these graphical user interface versions.

DETAILED DESCRIPTION

FIG. 1A-1G illustrate various views 3 of a graphical user interface included in and/or associated with a vehicle event data playback apparatus. The graphical user interface and/or the vehicle event data playback apparatus described herein may be configured to visually present a user with information related to operation of a vehicle. In some implementations, the user may review the information related to operation of the vehicle in real time, responsive to the vehicle being involved in a vehicle event, responsive to an active safety system alert, and/or at other times. The vehicle event may include an active safety system alert (e.g., events on the vehicle bus that indicate activations of active safety system), an accident, a specific driving maneuver, vehicle speed exceeding a set threshold or the posted speed limit, a narrowly avoided accident where a driver braked hard and/or swerved, trigger activations from external systems, a driver pressing a button (an/or gesturing via some other interface) designed to indicate an event, and/or other events. Specific driving maneuvers and/or active safety system alerts may be related to swerving, a U-turn, freewheeling, over-revving, short following (e.g. tailgating), a pedestrian detection and/or collision warning, driving in reverse, vehicle rollover, antilock braking, a forward collision warning, a lane departure warning, other events and/or maneuvers.

In some implementations, an active safety system may be installed in the vehicle at manufacture. In some implementations, the active safety system may be an aftermarket system installed in and/or coupled with the vehicle. The active safety system may be configured to detect and/or alert a vehicle driver to dangerous and/or other driving conditions. An active safety system alert may cause the driver to take action to prevent and/or lessen the severity of collisions, for example, and/or other undesired events. The active safety system may include one or more sensors, one or more processors, a user interface, and/or other components. The one or more active safety system sensors may be configured to generate output signals conveying information related to vehicle safety and/or other information. The one or more processors may be configured to detect unsafe conditions during operation of the vehicle based on the output signals from the active safety system sensors, and/or other sources of information. The one or more processors may be configured to generate one or more alerts responsive to detection of unsafe conditions. The active safety system may be manufactured by one or more manufacturers including, for example, Meritor-Wabco, Bendix, Mobileye and/or other active safety system manufacturers.

The vehicle event data playback apparatus visually presents information that is generated based on output signals from one or more sensors associated with the vehicle. The vehicle event data playback apparatus may synchronize the presented information such that information from individual sensors may be compared and/or viewed at the same time by the user. The information from individual sensors may be compared and/or viewed at the same time by the user at one or more time points before, during, and/or after a vehicle event, an active safety system alert, and/or at other times.

The sensors may be configured to generate output signals conveying information related to the operation and/or the context of the vehicle. The sensors may be configured to generate output signals conveying information related to one or more operating parameters of the vehicle. Information related to the operation of the vehicle, the context of the vehicle, and/or operating parameters of the vehicle may include vehicle safety information, vehicle event information, vehicle environment information, vehicle subsystem information, and/or other information. Information related to the operation of the vehicle, the context of the vehicle, and/or operating parameters of the vehicle may include feedback information from one or more of the mechanical systems of the vehicle, and/or other information. In some implementations, at least one of the sensors may be an active safety system sensor. In some implementations, at least one of the sensors may be a vehicle system sensor associated with an ECU system of the vehicle. Information related to the context of the vehicle may include information related to the environment in and/or around vehicle. In some implementations, the output signals conveying the information related to the context of the vehicle may be generated via non-standard aftermarket sensors installed in the vehicle as part of a vehicle event recorder, for example. The non-standard aftermarket sensors may include, for example, a video camera, an accelerometer, a gyroscope, a geolocation sensor (e.g., a GPS device), a radar detector, an audio sensor (e.g., a microphone), and/or other sensors.

The vehicle event playback apparatus may include and/or communicate with the vehicle event recorder. The vehicle event recorder may be configured to record data associated with vehicle events. The vehicle events may be triggered to be recorded based on the information generated by the active safety system (e.g., the active safety system sensors and/or the active safety system processor(s)), the information from the non-standard aftermarket sensors installed in the vehicle, the information from the vehicle system sensors associated with an ECU system of the vehicle, pre-determined vehicle event profiles, and/or other information. In some implementations, the vehicle event recorder may be triggered to record the vehicle events based on a trigger, for example. A trigger may include and/or result from a comparison of the information conveyed by the output signals from the sensors to criteria included in the pre-determined vehicle event profiles. A trigger may be one or more logical criteria that is determined to have been met by the system. The vehicle event recorder may be configured to obtain one or more pre-determined vehicle event profiles. The one or more pre-determined vehicle event profiles may include criteria sets associated with the vehicle events. A specific vehicle event may be triggered to be recorded based on the information conveyed by the output signals and the obtained pre-determined vehicle event profiles by comparing the information conveyed by the output signals (and/or information determined based on the output signals) to the criteria sets such that a specific vehicle event is triggered to be recorded responsive to the information in and/or determined from the output signals satisfying one or more individual criteria in a criteria set associated with the specific vehicle event. A vehicle event recorded by the vehicle event recorder may include audio information, video information, information from the ECU, information from the active safety system, and/or other information.

By way of a non-limiting example, the system may detect that a driver has exited his lane 3 times in a 90 second period, where the criteria for triggering the event included activation of a lane departure warning system (e.g., information may be communicated across the ECU to the present system) and time elapsed. As another example, an event may be triggered to be stored based on criteria that include following distance as measured by an active safety system and time (e.g., following distance of less than 1.4 seconds for 10 or more seconds), where the distance is measured and based on a formula inclusive of time.

The vehicle event playback apparatus and/or the vehicle event recorder may include one or more processors. The processors may be configured to synchronize (e.g., systematically align) the information included in the output signals from the one or more sensors associated with the vehicle (e.g., the active safety system sensors, the vehicle system sensors associated with the ECU system of the vehicle, the non-standard aftermarket sensors installed in the vehicle as part of the vehicle event recorder). The processors may synchronize the information even though the information from the individual sensors may not be generated and/or received by the vehicle event playback apparatus at the same time. Synchronization may include associating data from two or more individual output signals with a single event timeline for a given event. An event timeline may have a start-time, an end-time, and/or a continuous period of time therebetween, for example.

In some implementations, the output signals from the one or more sensors may include time information indicating a time the information was generated, the order in which the information was generated, a timing of the generated information (e.g., periodic and/or continuous), and/or other time information. In some implementations, the processors may associate information in the individual output signals with a corresponding time location in the event timeline based on the time information included in the output signals.

In some implementations, associating information in the individual output signals to a corresponding time location in the event timeline based on the time information included in the output signals may not produce a synchronized event timeline. For example, the timing information in a first output signal (e.g., information indicating the start of an event at 2:40:48 PM) may not coincide with the timing information in a second output signal (e.g., information indicating the start of the same event may be received at 2:41:02 PM) even though both output signals include information related to the same event. In such implementations, the one or more processors may analyze information in the individual output signals and associate corresponding information in the individual output signals with the same time location in the event timeline, regardless of any time information in the output signals.

Analyzing the information in the individual output signals may include searching for expected information in individual output signals and associating the expected information with one or more appropriate time locations in the event timeline. Information conveyed by a first output signal may cause the processors to expect information in one or more other output signals. Continuing with the example above, one or more processors may determine that the information in the first output signal indicates the start of an event and may associate the information with the start of an event timeline (e.g., at 2:40 PM). After determining that the information in the first output signal indicates the start of the event, the one or more processors may search the information conveyed by the second output signal for information that also indicates the start of the event, regardless of any time information included in the output signal. After finding information in the second output signal that also indicates the start of the event, the processors may associate the found information with start of the event in the event timeline.

In some implementations, associating information in the individual output signals to a corresponding time location in the event timeline may be performed by the vehicle event playback apparatus. In some implementations, associating information in the individual output signals to a corresponding time location in the event timeline may be performed by the vehicle event recorder. The various data streams recorded within an event may involve some delays in the data collection process. These delays may vary by the signal (e.g., acceleration data may be received “faster” than the OBD vehicle speed data and even “faster” than the GPS data). These delays may be related to how the underlying sensors collect data. For example, accelerometers may be MEMS sensors that provide sub-millisecond readings. OBD vehicle speed measurement may be derived from an optical wheel speed sensor that generates pulses where the pulse periods are proportional to vehicle speed. This means that the slower the vehicle, the larger the delay in speed measurement because two consecutive pulses are necessary to estimate the vehicle speed. These delays may be measured in tens of milliseconds, but nonetheless represent delays that may result in misalignment of data when compared to accelerometer signals, for example. GPS relies of triangulation of satellite signals to determine the vehicle location. Many GPS systems determine the vehicle location once every second. Two consecutive vehicle locations measurements are necessary to derive the speed (e.g., distance between the points divided by the time interval between the measurements). Therefore, speed measurements from GPS systems are typically delayed by one second. Another reason for the delays in input signals may be related to the various buffers employed by electronic devices as they transmit information. Many communication devices send data in packets, which involved waiting for enough data to become available before sending the packet of information. These wait times further contribute to the input signal delays.

In some implementations, by being aware of these fixed time delays, the system may be constructed to purposefully delay individual signals except for the most delayed signal such that the individual signals become aligned with respect to time (e.g., synchronized) with the most delayed signal. After synchronizing the signals, the vehicle event recorder may store the aligned signals in volatile memory and/or other electronic storage.

Responsive to triggering of a vehicle event, the event data may be saved from a volatile memory to a non-volatile memory. The vehicle event recorder may compensate for the known delay in individual signals such that the saved event may be properly centered around the trigger point. For example, the largest signal delay may be 1.75 seconds. In this example, the individual signals (other than the one with the largest delay) may be purposefully delayed by additional delays (1.75 seconds minus the delay for the particular signal), such that aligned data is written to volatile memory with a 1.75 second delay. At the time that a vehicle event is detected (e.g., at the time a detection trigger signal is received), the vehicle event recorder may record a 30 second event with 15 seconds before the detection and 15 seconds after the detection. Upon receiving the trigger signal, the vehicle event recorder may immediately select the last 16.75 seconds (15 plus 1.75) of event data to be written from volatile to non-volatile memory, and then save the remaining 13.25 seconds of data (15 minus 1.75).

An indication that the vehicle event was detected (e.g., a trigger signal) may also be delayed. Triggers such as external voltage signals, where the trigger is generated on the rising edge, may not involve delays. However, more complex triggers may rely on delayed signals and a triggering algorithm itself may introduce additional delays. For example, a simple accelerometer triggering algorithm that involves measuring braking force may be configured to trigger once deceleration of 0.5 g is maintained over 1.0 seconds. This means that the triggering signal may be generated once the 1.0 seconds elapses with deceleration exceeding 0.5 g. However, the trigger point would ideally be identified in the middle of this interval, or at 0.5 elapsed seconds after the deceleration first exceeds 0.5 g. Therefore, the triggering signal from this triggering algorithm may be considered to be delayed by 0.5 seconds. A vehicle event recorder may compensate for triggering delays when saving event data from the volatile memory to the non-volatile memory. In the above example where all aligned signals written to volatile memory are delayed by 1.75 seconds, upon receiving a trigger signal that is known to be delayed by 0.5 seconds, the vehicle event recorder may compensate for this additional delay by immediately selecting the last 16.25 seconds (15 plus 1.75 minus 0.5) of event data to written from volatile to non-volatile memory, then proceeding to save the remaining 13.75 seconds of data (15 minus 1.75 plus 0.5).

By way of another non-limiting example, the one or more sensors may include an accelerometer (e.g., associated with the vehicle event recorder), a camera (e.g., associated with the vehicle event recorder), a sensor associated with the vehicle ECU, and/or other sensors. The accelerometer may generate output signals indicating that the vehicle has started breaking. The accelerometer information (e.g., the braking start time) may agree with video images that show the vehicle beginning to slow down. Information from the vehicle ECU may not indicate that the vehicle is breaking until a short time later, when the vehicle ECU information is received by the one or more processors of the vehicle event data playback apparatus. In this example, the one or more processors of the vehicle event data playback apparatus may synchronize the information from the accelerometer, the camera, and the vehicle ECU system so the information may be presented to the user in a synchronized manner. The user may better understand the event because of the synchronized information. The user may better understand the cause of the event, the outcome of the event, road and/or other environmental conditions during the event, and/or other information related to the event by viewing the synchronized information.

As shown in FIG. 1A, view 3 includes a first (e.g., forward looking) video playback view 4, a second (e.g., vehicle interior driver video) playback view 5, an event timeline 7, a graphical representation 9 of information related to the operation and/or the context of the vehicle, one or more visual representations 11 of information related to the operation and/or the context of the vehicle, visual representations 13 of vehicle ECU information, a notation field 15, and/or other information. In some implementations, view 3 (as shown in FIG. 1B-1G) may not include all of the components of view 3 shown in FIG. 1A. The number of video playback views shown in the figures is not intended to be limiting. The number of video playback views does not necessarily correspond to a number of cameras included in the system. For example, even though in some of the figures only two video playback views are shown, there may be 1, 2, 3, 4, 5, or more cameras included in the system. The two views may playback information from any two of the cameras, and/or other components of the system.

In some implementations, view 3 may include additional components now shown in FIG. 1A (e.g., an additional playback view 5 as shown in FIGS. 1E, 1F, and 1G). In some implementations, the components may be presented with a different visual format. In some implementations, view 3 includes one or more active safety system fields 19 (shown in FIG. 1B, 1C, 1D, and FIG. 2). The one or more active safety system fields may present information related to detections and/or alerts by the active safety system. In some implementations, the active safety system fields may present a textual, numeric, graphical, and/or other representative indication of what the active safety system communicates to the driver of the vehicle. For example, the active safety system may alert a driver via a red/yellow/green color coded display that is part of the active safety system. The red/yellow/green color coded display may be reproduced in the one or more active safety system fields 19 of the graphical user interface of the vehicle event data playback apparatus. In some implementations, the active safety system fields in view 3 may reproduce the active safety system detections and/or alerts presented to the driver in real time and/or at another time.

The additional components of view 3 are described in more detail below. The information presented by the one or more active safety system fields and/or the other components of view 3 of the graphical user interface may be synchronized by the vehicle event recorder and/or the vehicle event data playback apparatus and displayed to the user at the same time. The synchronized display may be in real time (e.g., a remote user may view the synchronized information while the driver is operating the vehicle) and/or at a later time.

The graphical user interface may be specifically designed to function and/or cooperate with the vehicle event recorder, the active safety system, and/or the event dataset/alert information they produce. Such a graphical user interface may be interactive and/or responsive to user/operator inputs affected by a computer and/or other peripheral devices. For example, a ‘point-and-click’ action of a computer mouse, entering and/or selecting information via a touch screen, and/or other user/operator inputs. In some implementations the graphical user interface may be presented to a driver of the vehicle and/or other users (e.g. located at the vehicle and/or located remotely) via a user interface associated with the active safety system. In some implementations, the graphical user interface may be presented to a driver of the vehicle and/or other users via a user interface separate from the active safety system. In some implementations, the alerts and/or other information generated by the active safety system may be presented to a driver of the vehicle and/or other users via the graphical user interface of the vehicle event record playback system.

This system may include couplings to a vehicle event recorder and/or data captured via the vehicle event recorder, couplings to the active safety system and/or data captured via the active safety system, and/or other systems. This system may include one or more processors, one or more display devices, one or more graphical user interfaces, and/or other components. This system may receive as input vehicle event recorder datasets, information generated by the active safety system, and/or other information. Prescribed program code may be executed by the one or more processors to operate on datasets received from the vehicle event recorder, the active safety system, and/or other sources. Data from these datasets may be passed into control objects as parametric input to drive the various visual states of the control objects.

The graphical user interface of the vehicle event recorder dataset playback system presented herein may be comprised of a plurality of control objects. These control objects may include visual depictions of information which may change in response to user interaction and/or specific data values of a dataset under review. Some of these control objects may be interactive and adjusted responsive to ‘point-and-click’, ‘click and drag’, touch screen touches, and/or other user interactions. Control objects of these systems may be coupled to, and/or designed to cooperate with, data types and/or data objects unique to vehicle event recorder event record datasets, information from the active safety system, and/or information from other sources.

The control objects may have an appearance and/or data ranges that are correlated with information generated by vehicle event recorders during operation of vehicles. The control objects of these graphical user interfaces may have be configured to visually express data and/or information which is contained in a vehicle event record event dataset, information generated by the active safety system, and/or other information. Illustrative examples are included herefollowing.

Control objects may include and/or display information based on logic code. Logic code (not visible to a user) may facilitate receipt of input data, parsing that data, processing the data in accordance with prescribed algorithms (e.g., which are part of the logic code), providing outputs which may include adjustments to the visual states of the control objects, and/or other actions.

It should be noted that the series of control objects described below and shown in FIG. 1-FIG. 11 are not intended to be limiting. The system described herein may be configured to generate control objects related to any vehicle operation information that would be helpful for a user reviewing event data recorded before, during, and/or after a vehicle event.

A vehicle event recorder may be configured to capture compound datasets relating to vehicle operation and/or events during a finite period. Accordingly, one or more aspects of this vehicle event playback system may relate to a timeline control object and/or its relationship with the event dataset, and/or other controls of the graphical user interface. Timeline control objects may include an associated time range equivalent to the time range of the event period. The time range associated with a given event may vary in length from a fraction of a second to several tens of minutes or even hours. However, many events recorded by vehicle event recorders may be a few seconds or a few tens of seconds. Timeline control objects in accordance with these teachings may have one or more of a ‘start time’, an ‘end time’, a continuous time period therebetween, and/or other features. The ‘continuous’ time period between the start time and the end time may include a finite number of frame instants and/or frame stops, for example. These may be discrete moments in the event period for which an image may be captured. A timeline control object of these graphical user interfaces may be synchronized with the time span of events recorded by a vehicle event recorder when its start time and end time are set in accordance with specific related values in an event record dataset. In the example illustrated as FIG. 1, an event record includes a start time of 4:19:35 and an end time of 4:20:02 for a total event period of 27 seconds.

In some implementations, via the graphical user interface, a user may control the length of the event period, select (e.g., by clicking and/or touching a location) an individual time within the event period, continuously play the frame instants, rewind and/or fast forward the frame instants, and/or control the length of the even period in other ways.

For improved ease of operation of these timeline controls, when playing back vehicle event record data, it may be useful for a user to arrange the control to extend an appreciable length of interface regardless of the event period extent. That is, a timeline control which supports a twenty-seven second event may be the same size as a timeline control which supports a 10 minute event. Thus a timeline control may be ‘normalized’ to the event period without change of its physical size. In view of the data contained in an event record dataset, the start time and/or the end time may be associated with the timeline control extremities, and/or 100% of the timeline control length may be divided evenly and/or occupied by the entire event without dependence upon the actual extent of the event period. Accordingly, the timeline control object may cooperate with playback of the vehicle event recorder output in that the timeline may auto-adjust to normalize for events of any period length.

A timeline control object may be adapted to match the extent of a time period of a specific event under replay and/or thereafter may be set to agree with a specific event record. For example, as shown in FIG. 2, timeline pip markers 23, which may indicate noteworthy features of the event record, may be distributed about the timeline to provide access to information relating to those features. Timeline pip markers 23 may be pre-encoded elements made responsive to mouse clicks, touch screen touches, and/or other control indications entered and/or selected by a user via the graphical user interface and/or one or more peripheral devices. These pip markers may be associated with an instant in time and/or a time period which is a subset of the entire event period. Other control objects of the graphical user interface may be coupled to these pip marker objects and/or may be responsive thereto. They may be responsive to mouse clicks, touch-screen touches, and/or other control gestures associated with the pip markers. FIG. 2 also illustrates event timeline 22, image series players 21, 24, and a vehicle location 25.

The timeline object may include a timeline replay instant indicator control object. The instantaneous state of a timeline replay instant indicator may specify a replay instant in time, the replay instant to which other controls may be bound and responsive. A timeline replay instant indicator may mark the present instant of the playback system at a given time. The present instant may represent a moment in the event period between the event start time and the event end time. For controls bound to the timeline control object, the present instant time may be used to set the visual state of the control (individually, or as a coordinated group). The visual state of a control object may be different for the individual instants of the event period. When the timeline replay instant indicator is associated with another time instant of the event period, either manually or during a pre-play execution, individual ones of the controls bound to the timeline control may automatically update their visual state to represent data captured at that moment of the event period. Timeline replay instant indicators may be responsive to click-and-drag, touch, and/or other actions, and/or may be responsive to automated replay controls such as ‘play’ control, ‘fast forward’ control, ‘rewind’ control, etc.

Replay controls of these systems may include, for example, a ‘play’ control button, ‘fast forward’, rewind, ‘loop’, and/or other controls. Timeline control objects of these vehicle event playback systems may also include such controls.

Timeline control may be sensitive to keyboard input. For example, left and right arrow keys may be used to advance the timeline backward or forward, respectively, by one video frame (e.g. 0.25 seconds for 4 frames per second video) at a time. The space key may be used to start and/or pause the event data playback. Page Up and Page Down keys may be used to advance the timeline in larger fixed time intervals, such as 5.0 seconds forward or backward, respectively.

Modes of vehicle event playback systems presented here may include side-by-side playback of a plurality of videos (time series presentation of images). In two separate video control objects, a prescribed video may be played back synchronously. For example, a view from the front of a vehicle and a view from the back of the vehicle may be synchronized to represent separate images taken at the same point in time and played side by side. In some implementations, the synchronized video may be synchronized with vehicle ECU data, for example. This may be useful in gaining an understanding of a complex scene which may have more than one point of view, such as a driving incident. For example, in a driving incident it may be important to consider the precise timing of event in the view of the driver and/or in view of traffic ahead. When these two views are considered together, an expert reviewer may draw more precise conclusions with regard to the event details. Accordingly, the event playback systems may include side-by-side video playback controls for playing back videos of two or more unique viewpoints.

Vehicle event recorder systems may include a plurality of video cameras positioned at a plurality of video capture stations (locations). An individual video capture station may have a unique viewpoint relative to other video capture stations. In some vehicle event recorders, a ‘forward-looking’ camera may be arranged to capture a traffic view ahead of the vehicle and a second reward looking camera may be arranged to capture a view of a passenger/driver space.

There may be a time relationship between these two video views because actions taken by a vehicle operator may relate to the traffic and conditions ahead. One may understand a driver's response to things which may be discovered observable in the forward view. The vehicle event data playback systems may include a plurality of video players where individual views of the video players may be synchronized in time to the others and/or timeline control objects, and/or with respect to the timeline replay instant indicator. Where vehicle event recorders support more than two video cameras, a single timeline control may be used to synchronize those in a similar manner.

An operator/reviewer of these vehicle event data playback systems may ‘scroll’ through discrete video frames of the event timeline by adjusting the playback instant indicator, for example via ‘click-and-drag’, touch, keyboard key strokes, and/or other types of actions. Accordingly, the graphical user interface of this invention may include video playback controls coupled to the playback instant element of a timeline control object. In this way, two separate views about the vehicle environment may be considered simultaneously in a side-by-side arrangement where both views represent substantially the same instant in time as synchronized by the timeline control.

While some versions of these systems include side-by-side playback of forward facing and rear facing (for example) views taken from video cameras in a vehicle event recorder, (e.g., ‘real’ video), alternative versions may include at least one video playback display in which a ‘virtual video’ is presented. A virtual video may include a time series of images whereby the images are captured in a system which is not a vehicle event recorder. However, these virtual videos may relate directly to the events captured. For example, one type of virtual video in accordance with this teaching may present images having dependence upon the event timeline. Another example may have dependence upon the vehicle position as a function of time. Still another may have a position dependence with prescribed offset. Other examples are contemplated. More details of these types of virtual videos follow.

In some implementations, as shown in FIG. 3, an image series player 31 (video player) may be arranged to play an image series which was not recorded by the vehicle event recorder. Rather, a plurality of still images is recalled from a prepared database to form an image set which may be played in series. Those images when played together as a ‘video’ in the video player may constitute a ‘virtual video’ for purposes of this teaching. However, this replay may be well coordinated and having direct dependence with respect to the event record. For example, this virtual video timeline may be synchronized with the event timeline 32. The actual location of a vehicle as recorded by the vehicle event recorder may be coupled to the viewpoint from which the recalled images are made.

When the timeline control is set into a ‘play’ mode, the video of actual captured images may appear in a video player control adjacent to a second player which plays the virtual streetview video. This virtual video may include images from the same locations and/or viewpoints with respect to the event period, but these images may include enhancements and augmentations to bring a more complete understanding of the actual event. In one example, streetview images provided by a service like Google's StreetView system may be recalled in view of a plurality of position measurements taken by the vehicle event recorder. For this plurality of locations (vehicle's location as measured by a GPS for example) captured over the period of a given event, a streetview image from a corresponding location may be recalled from the image database. Individual recalled images may then be assigned a time instant corresponding to those times from which the playback timeline is comprised to assemble a virtual video of streetview images which directly corresponds to the video actually captured at the vehicle event recorder cameras. One advantage may lie at least partly in the clarity of the recalled images which may have been provided on a nice sunny day in comparison to images captured by the vehicle event recorder which may be of inferior clarity due, for example, to inclement weather including fog or sun glare, nighttime darkness, a large vehicle that obstructs the view of the event recorder, and/or other factors. Further advantages may be realized in view of the labels which may be may be incorporated with the prepared streetview images. For example, address label 28 may indicate a street address most closely associated with the image viewpoint. FIG. 3 also illustrates event timeline 32, image series player 34, vehicle location 35, a street address 38 corresponding to the vehicle location, a street view image 37 corresponding to the vehicle location, and/or other information.

Another benefit of this approach is that streetview may be further manipulated by the user to gain a better understanding of the environment in which the event was captured. For example, the user may pause the event playback, then rotate the streetview to show the image to the left, right or behind the vehicle. The system may be configured such that this user action would temporarily decouple the streetview from the event playback. Upon restarting the playback of the paused event playback, the system may be configured such that streetview would then realign and show the view in front of the vehicle again.

It may be inconvenient to view many videos simultaneously. To address this, these players may have two players side-by-side with the video type selectable in the interface by way of a tab strip control 33, which as shown in FIG. 3 for example, may include five tab selections (e.g., “BackCam”, “StreetView”, “MapView”, “BirdseyeView”, “Driver View”, and/or other tab selections). In agreement with which tab is selected and triggered, the video playback may be switched between the possible video presentation types. The player may present a synchronized playback of both real and virtual videos in agreement with the event period and/or the timeline control. The example of two side by side players is not intended to be limiting. The system described herein may have any number of players in the graphical user interface. The multiple players need not be side by side. They may be positioned anywhere in the graphical user interface that is helpful to the user. Additional views that correspond to additional cameras may be provided, for example. In some implementations, the cameras may include a forward road camera, an interior cab camera, a left side backward facing camera, a rear of the vehicle backward facing camera, and/or other cameras. The additional views/cameras may give greater perspective for a review analyst and/or a coach reviewing the information.

There may be another type of virtual video which may be supported by these playback systems which relates to moving maps. A map view virtual video is depicted in FIG. 4. In a fashion similar to that described in the streetview embodiment, a plurality of map images may be prepared and saved to form an image series consistent with an event record dataset. The vehicle location and/or the event timeline 41 may be considered in forming a series of map images suitable for playback in a mapview video player 42.

A separate map image 43 with appropriate scale, size and orientation may be presented in the viewer for every discrete instant of the event period as represented in the event timeline. When playback instant control 44 is moved to another time (constant) of the event timeline, the vehicle also moves (in most events). The new vehicle location 45 may cause a new map and/or a separate map image to be allocated and displayed for that time instant. The image in image series player 47 may correspond to vehicle location 45. The vehicle's position within the map may be indicated by an icon marker 46 to reflect the position of the vehicle as measured by the vehicle event recorder. In a an event replay, the series of map images may be played back synchronously alongside the actual forward view images (for example) captured at the vehicle event recorder camera.

A mapview player in accordance with this teaching may be useful when it is arranged to present maps with enhancements and/or markings which may be derived from information in the event record dataset. For example, when actual vehicle speeds are compared to local speed limits, a determination may be possible with regard to regions in which a speed infraction occurs. A graphical representation of the same may be included superimposed with the maps presented.

With reference to FIG. 5, timeline control object 51 agrees with the event record dataset from a vehicle event recorder. A playback instant control 52 marks an instantaneous moment of the event period. At the moment 4:19:38.50 p.m. indicated in a numeric label control 53, the vehicle GPS system determined the vehicle location at latitude 32.8295 and longitude −117.2733 as recorded by the vehicle event recorder's position determining system and position is reported numerically at position label control 55. In some implementations, the latitude and longitude information may be translated into a street address by the system and/or presented to the user in a field of the graphical user interface.

With the video player set into a map view mode 54 by a tab strip tool 56, a map image 57 suitable in scale and location may be presented that shows the vehicle's location at the corresponding instant in time. An icon image of a car 58 (for example) may mark the location in the map. In addition, a spatial highlight 59 for example (highlighting portions of Bonair and Draper streets in the image), may be superimposed on the map image to mark regions where speeding infractions have been identified.

Timeline marker 60 may designate a finite period of time within the event period when the speeding occurs. Thus, some timeline controls of these systems may include marker objects aligned and/or cooperative with elements appearing in virtual videos. This cooperation may be due to associations with time instants within the event period.

Both the streetview player and the mapview player may offer advantages in reaching a complete understanding of a scene. They may be augmented based on information collected by a vehicle event recorder.

Another virtual video player that may be included in these systems may be characterized as a bird's eye view virtual video player. Illustrated in FIG. 6, a bird's eye view video player of these systems may include images taken from an elevated viewpoint, for example images made from an airplane and/or satellite. In consideration of the vehicle's time-dependent position as measured and/or recorded by a vehicle event recorder, images may be selected from a prerecorded database of so described images, and/or obtained from other sources.

Timeline control 61 specifies an instant in time by a playback instant control 62. Video player display 63 may include an image 64 and a ‘car’ icon marker 65 to indicate the location of the vehicle at the capture time 4:19:38.50. When tab strip 66 is used to set the video player into a bird's eye view mode 67, an image series of perspective images made from altitude may be played back in time with respect to the event timeline. As the event player executes playback of event data, images displayed by image series player 68, bird's eye images 64, marker icon 65, and/or other components of the system, may be updated for instants of the timeline to reflect appropriate views of the event scene.

Because the image viewpoint may be selectable, it may be possible to specify that the viewpoint be constant at a virtual location, for example 100 meters behind the car and/or 50 meters above the car, and the view direction may point in the direction of travel with respect to the vehicle. In this way, the bird's eye view playback virtual video may be envisioned as if a helicopter video camera followed the car throughout the event to make the video images. A reviewer may gain a clearer understanding of the event environment as presented in a synchronized manner alongside with other event record information.

Accordingly, these vehicle event data playback systems may include one or more of a real view video player, a virtual video player, and/or other video players which may be arranged to interact with event data recorded in vehicle event recorders. Individual ones of these specialized video players may provide a time synchronized image series in conjunction with a visual presentation of other related event record data.

While the preceding description sets forth several video playback control objects, the following description is directed to other aspects of these vehicle event playback systems which may not be based upon data from image systems. Rather, the graphical user interface may control objects presented herefollowing to express non-image data from a plurality of sources. In some implementations, these sources may include onboard data systems. In some implementations, information sources may include those which are external with respect to a vehicle and/or its vehicle event recorders.

Control objects of graphical user interfaces of these playback systems bound to vehicle onboard systems may have two primary but non-limiting types including: 1) control objects bound to data from sensors deployed as part of a vehicle event recorder apparatus, for example, a vehicle event recorder may include accelerometers which measure G-force levels in two orthogonal spatial dimensions while a vehicle is being used; and 2) a control object bound to vehicle subsystems characterized as those installed by a vehicle manufacturer. Other types are contemplated. Data provided by way of a vehicle's ECU systems may be received, managed, parsed, and/or time-stamped at a vehicle event recorder which forms a dataset to which control objects of the graphical user interfaces may be bound. In some implementations, control objects of graphical user interfaces of these playback systems bound to vehicle onboard systems may include a third type. The third type of control object may be associated with the data generated by analysis modules of the vehicle even recorder, such as a wasted fuel estimate, triggers for dangerous driving maneuvers, calculated fuel consumption from mass air flow sensors, and/or other data generated by analysis module of the vehicle event recorder.

Some control objects of these systems may be bound to data sources not part of a vehicle event recorder system and/or external with regard to vehicle subsystems. These control objects may be nevertheless implicitly strongly coupled to event record datasets which are subject to playback in these devices and methods. In one illustrative example, a notation system which associates an expert reviewer's comments and notes with portions of an event record in a note field or notation control object may be coupled to receive data therefrom. Details of these types follow.

In one version, a control object and/or plurality of control objects may be coupled to the timeline control and thus the event period to affect time synchronization between these. Acceleration control objects may be arranged to visually show acceleration data collected during an event captured with a vehicle event recorder. With respect to acceleration data collected in a vehicle event recorder, it may be useful to present this type of information in two fashions. First, it may be useful to present instantaneous acceleration data associated with an instant of time during the event period. In a second fashion, acceleration data collected over the entire event period (or finite subset thereof) may be displayed in a graph of two axes. Force data may be presented in a form where the abscissa of which may be time, and the ordinate force.

For example, in FIG. 7, a view 72 of a first acceleration control object includes a line graph 71 representation of acceleration data into orthogonal directions. ‘G-force’ or acceleration is plotted versus time to form line representations 73 of acceleration data. A playback instant indicator 74 is synchronized with the playback instant indicator 75 of the timeline control object both spatially and with respect to displayed data.

Another related acceleration control object 76 which expresses acceleration data in an alternative visual form is additionally illustrated. This 2-D expression of instantaneous force includes a pointer 77 which gives a visual representation of force in both forward/aft and left/right directions 78 (also synchronized with playback instant indicators 74 and 75). The acceleration control objects may be bound to information in the event record dataset whereby synchronization is realized with respect to the timeline control and other control objects similarly bound including the video display control objects. Accordingly, control objects may be arranged to present instantaneous data as well as plots of data over time. In both cases, these controls may be bound and/or responsive to the playback timeline 79.

Vehicle event recorder systems may produce information-rich event record datasets. Event record datasets of vehicle event recorders may include data captured at vehicle subsystems, for example, by way of the onboard diagnostics and/or ECU systems. In vehicle event recorder systems so equipped, data captured at various vehicle subsystems may be time-stamped in a scheme coordinated with the event period and/or with video frame capture rates. Where such data is time-stamped, it may enhance synchronous replay via these vehicle event data playback systems.

Accordingly, these vehicle event data playback systems may be suited for playback of vehicle event records having been formed with time-stamped data from factory installed vehicle subsystems. Some vehicle event recorder systems may be arranged to capture engine speed data and/or to associate a time instant with speed data measurements. Engine speed in ‘revolutions per minute’ or RPM, may be read from an ECU via a coupling to which a vehicle event recorder may be connected. In some alternative versions, engine speed measurements may be made via ‘aftermarket’ installed sensors and/or vehicle subsystem detector which may obviate need to directly couple with the ECU. In either case, when event record datasets which are compatible with these playback systems are prepared and recorded, individual engine speed measurements may be associated with an instant in time and/or “time-stamped”. In systems common in the art where engine speed is recorded, engine speed is generally recorded without regard for simultaneous and/or synchronized playback. In those systems, engine speed data may not be provided with an association with time. Because it is a goal of these playback systems to playback data in a synchronized fashion, it may be necessary to time stamp data in this way. Where vehicle subsystems, for example an engine tachometer, do not provide measurement data at a rate equal or similar to video camera frame rates, data smoothing and or data averaging may be used to improve a dataset or portion thereof to make it better cooperate with the objectives of these playback systems, which may include a discrete number of time instances on an event timeline. Common vehicle tachometers may not face this restriction and may be otherwise free to take measurements at a convenient rate including those rates having unequal periods between data measurements. To facilitate time synchronization in these playback systems, the moment any measurement is made may be accounted for.

Many modern vehicle manufacturers may include advanced electronic systems with many of the vehicle's subsystems. For example, a mechanical accelerator pedal often includes a transducer from which pedal position may be readily measured. However, electronic data which may exist within a vehicle's proprietary electronic control schemes may not always readily available for third-party use. In some implementations, vehicle event recorders may be unable to record information relating to vehicle performance parameters such as the odometer on light duty vehicles, the fuel flow meter on light duty vehicles, and/or other parameters. This is not intended to be limiting. The system described herein should be understood to utilize any information made available by the vehicle's subsystems. If unavailable types of information are eventually made available by the vehicle manufactures, it should be considered within the scope of this disclosure to include a control object associated with the newly available information.

In implementations where a vehicle event recorder may be successfully coupled to vehicle electronic subsystems to receive the difficult to obtain and/or newly released data, the vehicle event recorder may be adapted to pass the data and/or manage a timestamp scheme in conjunction with the vehicle event recorder. For example, if a vehicle event recorder operates with a video frame rate of 30 frames per second, but the factory installed odometer and/or fuel flow data only update three times per second, a data recording scheme may be set to rectify timing issues between these independent data sources so that an event dataset may accurately reflect an ‘instantaneous’ time value for the data elements.

Vehicle event recorder systems which record time-stamped engine speed data are not known in the art. As such, synchronized playback of same has been impossible. As shown in FIG. 8, in systems disclosed herein, engine speed information may be expressed graphically in a visual presentation, in an alphanumeric expression, and/or in other forms as at least part of an engine speed control object 81. An engine speed control object of this example may be comprised of graphical portions and/or alphanumeric portions. An analog arc 82 may provide a range continuum upon which instantaneous magnitude may be displayed by a pointer 83. A digital numeric readout 84 may allow an instantaneous report of engine speed for an instant of the event timeline 85. Playback instant indicator 86 may be moved (e.g. via mouse type computer pointing peripheral device) to any point along the timeline. The engine speed control object, which is bound to the timeline, may be updated to indicate the engine speed recorded at that particular time. In a similar control object, vehicle speed control object 87, pointer 88 may yield an instantaneous value of ‘31 mph’ on an analog scale, while a digital numeric value 89 may also be provided at vehicle speed label.

Event records which may compatible with, and/or may be played by these vehicle event playback systems may include measurement data from the groups characterized as those including: throttle position data, engine load data, power takeoff system data, malfunction indicator light system data, brake system data, antilock brake system data, automatic traction control system data, electronic stability control system data, excess fuel consumption system data, and/or other data. FIG. 9 illustrates an event timeline 91 having playback instant indicator 92 set at time corresponding to 4:19:53.00 p.m. that is coupled to a plurality of control objects. Individual ones of the control objects may be arranged to express time-stamped data in a visual presentation which may include both graphical and/or alphanumeric representations.

Most vehicle event recorder systems are incapable of recording data related to throttle position. However, this may remain a factor in accident analysis. For example, it may be useful to understand how much time passes between the time a red light traffic signal appears (detectable via forward view video review—for example) and the time a vehicle operator disengages application of power (e.g. removes foot from an accelerator pedal). Because it may be difficult to arrange a custom sensor to detect accelerator position, many vehicle event recorders may include datasets devoid of this critical data.

While access to this information may be available on a vehicle's ECU system, to date, it has been prohibitively difficult to couple vehicle event recorders to the ECU of modern vehicles. Where that has been achieved, the dataset produced by such advanced vehicle event recorders may include time synchronization management schemes with respect to its video camera. These systems may include time-stamped data regarding throttle state where an event record dataset having such timestamp throttle state data are available. These vehicle event data playback systems may be cooperative and/or provide graphical user interface controls which are suitably responsive to such event record datasets.

A graphical user interface with a throttle state control object may provide graphical expressions of throttle states, and/or other information. The control object may be coupled to an event timeline and/or corresponding event timeline control whereby the instantaneous throttle state for a time in the event period may be selectively displayed by the control.

In some implementations, an instantaneous numeric value 93 (digital) and/or a graphical expression 94 (analog) of the throttle state may be given for individual instants of the timeline control. When the playback instant indicator is moved to another position of the event timeline, the throttle state control may be updated such that data represented there is updated in synchronization with other controls of the graphical user interface, including video.

The video of the video playback control object may indicate a red traffic signal has occurred at 4:19:44. A user/reviewer may be able to see by sliding the playback instant control to that moment of the timeline that the throttle state remained at 100% (recall FIG. 1). However, when the playback instant control is advanced further down the timeline to about 4:19:46, the throttle position may be at 0%, for example.

This may be verified by considering vehicle speed 710 (shown in FIG. 7) data presented in a continuum chart. In it, a user/reviewer may see that the vehicle begins to decelerate just before 4:19:45. While the acceleration data may be a good indicator of when the driver releases the throttle, it will remain imprecise because of mechanical delays inherent vehicle powertrain systems and the overall vehicle dynamics. Also, using acceleration data alone will result in misdetection of vehicle deceleration that may be due to vehicle travelling uphill and the force of gravity acting against the forward momentum of the vehicle. That is, the accelerator pedal may stay depressed at 25%, which may maintain vehicle speed while travelling 55 mph on a level road, but will result in vehicle deceleration should the road grade change to 5% uphill. To learn the precise time the throttle was released, the playback instant control may be manipulated to indicate a time prior to 4:19:45 and the throttle state control may be reviewed. This control may more accurately tell a fuller story as it relates to the question “when did the driver respond” rather than the question “when did the vehicle respond”. Accordingly, in some circumstances a throttle state control well synchronized with related data and/or bound to a timeline control unit may tell the full story which otherwise could not be known in the absence of throttle state information.

Similarly, a graphical user interface engine load control object 95 may be included in some versions of these vehicle event data playback systems. Engine load data may help a user reviewing an event via the graphical user interface to understand how heavily loaded the engine is at its given speed (RPM). This may inform the user if the vehicle was heavily loaded with cargo, whether the driver had excess power that he could have used for an evasive maneuver, etc. A sensor installed by a vehicle manufacturer may produce data to indicate engine load on a scale from 0% to 100%. Data from the sensor may be available via the ECU, for example. Data which indicates engine load may be periodically detected and/or captured, then added to the event record dataset produced by the vehicle event recorders. This step may be done with attention to time calibration and/or time synchronization. For example, data from the ECU may arrive at irregular intervals. Or the data may arrive with extreme time jitter between successive measurements. This data may be used in playback systems taught herein after being synchronized with other events encoded in event records. It may not be enough, with respect to the present vehicle event data playback systems, to merely collect multiple readings of engine load. Because of the particular playback characteristics suggested here, engine load data (among other data) may be time stamped before being included as part of these event records.

Another information source which may relate to vehicle performance and/or which may be replayed in these event data playback systems may relate to power distribution. It may be useful to know when engine power is needed by, and/or being delivered to, auxiliary systems. If a vehicle engine is delivering power to coupled power consuming subsystems (e.g. refrigeration, hydraulic actuator, et cetera) an event record may be configured to indicate the status of power takeoff systems.

In some implementations, a power takeoff indicator 96 may be embodied as a binary indicator. A power takeoff control object arranged to indicate binary states for every instant of an event timeline may include a graphical portion, an alpha numeric portion, and/or other portions, for example.

As shown in FIG. 9, other control objects may be provided to visually present data collected from vehicle subsystems. For example, these may include a brake indicator control object 98, an antilock braking system ABS control object 99, an automatic traction control (ATC) control object 910, an electronic stability control ESC control object 911, and/or other control objects. The graphical user interface control objects may be driven by data contained in a vehicle event recorder event dataset, and/or be responsive thereto. These controls may be bound to the timeline control object whereby visual presentation of data in the controls may be time synchronized. These controls may present data in graphical, alphanumeric, and/or other presentations.

In systems which produce data having a limited few states, it may be convenient to present the data states via prescribed and/or preconfigured icon symbols. For example a malfunction indicator light (MIL) system may operate to provide driver alerts with respect to five types of malfunction in addition to a binary ‘ON’-‘OFF’ value 912. A low tire pressure icon 913 may be displayed to indicate a malfunction. The control object, which may remain synchronized with event playback, may be showing its ‘ON’-‘OFF’ indication and/or malfunction type via the displayed icon, for example.

In some implementations, vehicle event recorders may include an excess fuel consumption indicator. When a driver takes an action which may be determined to be one associated with excess fuel consumption, an excess fuel consumption indicator may provide a driver with instant feedback to signal the condition. Where vehicles are equipped with such devices, they may also provide data to be included in a vehicle event recorder event record dataset. Accordingly, these vehicle event data playback apparatus may include an instant driver feedback control object 914. Because feedback may be provided as a three color scheme (for example) in vehicles in which they are deployed, the instant driver feedback control object may similarly express these feedback states. This control object may be an example which illustrates the value of a visual playback of vehicle event records. When an indicator is given to a driver to alert him that a maneuver has caused excess fuel consumption, it may be useful to understand in detail a driver's response in order to administer appropriate coaching. The system described herein may facilitate visualization and/or understanding of details associated with vehicle operation. For example, a reviewer/coach may decipher a driver response to instant feedback relating to fuel consumption maneuvers.

The description of data collected via vehicle event recorder systems is not intended to be limiting. There may be information sources that are not part of the vehicle event recorder which contribute data that may be played back the systems described herein.

In some implementations, vehicle event recorders may record data from vehicle systems, from event records, and/or from other sources, which details many aspects of vehicle and/or driver performance. In some of these systems, a human reviewer may study video captured in an event record and/or prepare notes in accordance with observations made. Some of these notes may be general to the entire event period (e.g. a ‘no seat belt’ infraction) and/or others may be particular to specific time portions of the event period and/or a single instant in the event period. Once prepared, the notations may become appended to, and/or part of an event record. When event records are played back in these data players, these notations may be displayed via the graphical user interface.

FIG. 10 shows one illustrated example of a timeline control 101 coupled to a notation field control 102. The notation field may include numeric review data such as a score value 103 and icon indicator 104 related to severity, a review date label 105 and a note list 106 containing therein a plurality of note entries (two notes entries are shown in the example).

A first note 107 relating to an occurrence of “aggressive accelerating” may be coupled to timeline marker pip 108 while “other task” notation 109 may be coupled to timeline marker pip 1010. Finally, note field 1011 may contain text which describes the reviewer's observations as those relate to the ‘other task’ note. Since these notes relate to specific parts of the event period, event playback may be improved when note text is appropriately displayed with respect to the note time association during the event data playback.

Access to several of these notes may also be affected by pointing and clicking, touching, etc., on the timeline pip marker elements. For example, if a user were to click on/touch the diamond shaped marker pip, the “other task” note closes and the “aggressive accelerating” note opens to reveal text of that note. In this way, access to appended notes may be readily available via timeline cues.

The notation control object described in the foregoing graphs may be illustrative of a first kind of information whose source may not be from onboard a vehicle but may be related to a vehicle event playback. There may be other sources which provide information related to a vehicle event but are not part of the vehicle, any of the vehicle's subsystems, or a vehicle event recorder.

Another information source external from the vehicle may include one which reports on environmental conditions related to the time and location of the recorded event. In one type of environment control object 1012, the state of the weather may be reported as recorded in remote weather reporting stations. For example, if an event recorder produces an event record with location and time information, a weather station may report ambient temperature via text label 1013. It may report whether or not the roadways were dry or wet at that time via icon display 1014. It may indicate whether or not the roads were icy or snowy. These systems may indicate whether or not the event occurred during a windy day. In view of sidereal time, and in view of a vehicle's direction of travel (as recorded by a vehicle event recorder), this control object may indicate the level of sun glare 1015 which may have hindered a driver's visibility during an event. A weather reporting station accessed via the internet (for example) before, during, or after an event record is made, may provide such data.

Other information sources external from the vehicle that may be helpful to a user reviewing an event via the graphical user interface may include databases containing speed limit information and/or road type information (e.g., freeway, major boulevard, two-lane road), sources of information about a traffic level, and/or other information.

FIG. 11 illustrates additional features. For reference, event timeline control 111 is illustrated in this drawing. A ‘now playing’ event label 112 may identify an event which is currently being addressed by, and/or is a subject of the event data player. A driver 113 drop-down type selection box 114 may permit operators of these playback systems to select other fleet drivers to which these playback systems may be pointed to view events associated with that particular driver. Another selection box 115 may enable an administrator to mark an event with additional notations to indicate a training status.

Although the system(s) and/or method(s) of this disclosure have been described in detail for the purpose of illustration based on what is currently considered to be the most practical and preferred implementations, it is to be understood that such detail is solely for that purpose and that the disclosure is not limited to the disclosed implementations, but, on the contrary, is intended to cover modifications and equivalent arrangements that are within the spirit and scope of the appended claims. For example, it is to be understood that the present disclosure contemplates that, to the extent possible, one or more features of any implementation can be combined with one or more features of any other implementation.

Claims

1. A method to synchronize vehicle operation information comprising: receiving the vehicle operation information via output signals generated by a plurality of sensors, wherein the plurality of sensors is coupled with a vehicle, wherein the plurality of sensors includes a first sensor that generates a first output signal conveying a first operating parameter and a second sensor that generates a second output signal conveying a second operating parameter, wherein the first output signal is associated with first timing information, and wherein the second output signal is associated with second timing information;detecting a first vehicle event based on the first output signal and the second output signal, the first vehicle event having a start time and an end time; andresponsive to detecting the first vehicle event, associating information from the first output signal and the second output signal generated during the first vehicle event to create a first event record, wherein the first timing information includes a first time-stamp that indicates the start time of the first vehicle event, wherein the second timing information includes a second time-stamp that indicates the start time of the first vehicle event, and wherein the first time-stamp does not coincide with the second time-stamp; andsynchronizing the information from the first output signal and the second output signal generated during the first vehicle event based on analysis of the first time-stamp and the second time-stamp, wherein synchronizing includes identifying and correlating corresponding information associated with one or more physical phenomena in the first output signal and the second output signal during the first vehicle event.
2. The method of claim 1, wherein synchronizing the information includes searching for expected phenomena in the second output signal that indicate the start time of the first vehicle event, wherein the first timing information indicates one or more of a time of day the information was generated, an order in which the information was generated, a periodic timing of the generated information, and a continuous timing of the generated information.
3. The method of claim 1, wherein the vehicle operation information includes one or more of vehicle safety information, vehicle event information, vehicle environment information, or vehicle subsystem information.
4. The method of claim 1, wherein the first operating parameter is related to vehicle safety, and wherein vehicle safety information is received via output signals generated by one or more sensors included in an active safety system coupled with the vehicle.
5. The method of claim 4, wherein the first vehicle event is detected based on the vehicle safety information received from the active safety system.
6. The method of claim 1, further comprising causing a graphical user interface to present the synchronized vehicle operation information to a user.
7. The method of claim 6, wherein a view of the graphical user interface includes one or more active safety system fields, the method further comprising causing the one or more active safety system fields to present information related to at least one of detections and alerts by an active safety system coupled with the vehicle, the active safety system fields presenting one or more of a textual, a numeric, or a graphical indication of information the active safety system communicates to a driver of the vehicle.
8. The method of claim 1, wherein the first operating parameter is related to at least one of vehicle event information and vehicle environment information, the method further comprising receiving the at least one of vehicle event information and the vehicle environment information via output signals generated by one or more sensors included in a vehicle event recorder coupled with the vehicle.
9. The method of claim 1, further comprising causing a graphical user interface to present the synchronized vehicle operation information to a user in real time during operation of the vehicle.
10. The method of claim 1, wherein receiving includes receiving information from one or more sensors included in an active safety system sensor, a video camera, an accelerometer, a gyroscope, a geolocation sensor, a radar detector, an engine RPM sensor, a vehicle speedometer, a tilt sensor, or a braking sensor.
11. A vehicle event data playback apparatus for synchronizing vehicle operation information comprising: one or more processors configured to synchronize vehicle operation information, wherein synchronizing the vehicle operation information comprises: receive the vehicle operation information via output signals generated by a plurality of sensors, wherein the plurality of sensors is coupled with a vehicle, wherein the plurality of sensors includes a first sensor that generates a first output signal conveying a first operating parameter and a second sensor that generates a second output signal conveying a second operating parameter, wherein the first output signal is associated with first timing information, and wherein the second output signal is associated with second timing information;detect a first vehicle event based on the first output signal and the second output signal, the first vehicle event having a start time and an end time; andresponsive to detecting the first vehicle event, associate information from the first output signal and the second output signal generated during the first vehicle event to create a first event record, wherein the first timing information includes a first time-stamp that indicates the start time of the first vehicle event, wherein the second timing information includes a second time-stamp that indicates the start time of the first vehicle event, and wherein the first time-stamp does not coincide with the second time-stamp; andsynchronize the information from the first output signal and the second output signal generated during the first vehicle event based on analysis of the first time-stamp and the second time-stamp, wherein synchronizing includes identifying and correlating corresponding information associated with one or more physical phenomena in the first output signal and the second output signal during the first vehicle event.
12. The vehicle event data playback apparatus of claim 11, wherein the one or more processors are configured such that synchronizing includes searching for expected phenomena in the second output signal that indicate the start time of the first vehicle event, wherein the first timing information indicates one or more of a time of day the information was generated, an order in which the information was generated, a periodic timing of the generated information, and a continuous timing of the generated information.
13. The vehicle data playback apparatus of claim 11, wherein the one or more processors are configured such that the vehicle operation information includes one or more of vehicle safety information, vehicle event information, vehicle environment information, or vehicle subsystem information.
14. The vehicle event data playback apparatus of claim 11, wherein the processors are configured such that the first operating parameter is related to vehicle safety, and wherein the processors receive vehicle safety information via output signals generated by one or more sensors included in an active safety system coupled with the vehicle.
15. The vehicle event data playback apparatus of claim 14, wherein the processors are configured such that the first vehicle event is detected based on the vehicle safety information received from the active safety system.
16. The vehicle event data playback apparatus of claim 11, further comprising a graphical user interface configured to present the synchronized vehicle operation information to a user, wherein the processors cause the graphical user interface to present the synchronized vehicle operation information to the user.
17. The vehicle data playback apparatus of claim 16, wherein a view of the graphical user interface includes one or more active safety system fields, the one or more active safety system fields configured to present information related to at least one of detections and alerts by an active safety system coupled with the vehicle, the active safety system fields presenting one or more of a textual, a numeric, or a graphical indication of information the active safety system communicates to a driver of the vehicle.
18. The vehicle data playback apparatus of claim 11, wherein the processors are configured such that the first operating parameter is related to at least one of vehicle event information and vehicle environment information, and wherein the processors receive the at least one of vehicle event information and vehicle environment information via output signals generated by one or more sensors associated with an engine control module installed in the vehicle at manufacture.
19. The vehicle event data playback apparatus of claim 11, further comprising a graphical user interface configured to present the synchronized vehicle operation information to a user in real time during operation of the vehicle.
20. The vehicle event data playback apparatus of claim 11, wherein the processors receive information from one or more sensors included in an active safety system sensor, a video camera, an accelerometer, a gyroscope, a geolocation sensor, a radar detector, an engine RPM sensor, a vehicle speedometer, a tilt sensor, or a braking sensor.

US Referenced Citations (902)

Number	Name	Date	Kind
2943141	Knight	Jun 1960	A
3634866	Meyer	Jan 1972	A
3781824	Caiati	Dec 1973	A
3812287	Lemelson	May 1974	A
3885090	Rosenbaum	May 1975	A
3992656	Joy	Nov 1976	A
4054752	Dennis, Jr.	Oct 1977	A
4072850	McGlynn	Feb 1978	A
4258421	Juhasz	Mar 1981	A
4271358	Schwarz	Jun 1981	A
4276609	Patel	Jun 1981	A
4280151	Tsunekawa	Jul 1981	A
4281354	Conte	Jul 1981	A
4401976	Stadelmayr	Aug 1983	A
4409670	Herndon	Oct 1983	A
4420773	Toyoda	Dec 1983	A
4425097	Owens	Jan 1984	A
4456931	Toyoda	Jun 1984	A
4489351	dAlayer de Costemore dArc	Dec 1984	A
4496995	Colles	Jan 1985	A
4500868	Tokitsu	Feb 1985	A
4528547	Rodney	Jul 1985	A
4533962	Decker	Aug 1985	A
4558379	Hu/tter	Dec 1985	A
4588267	Pastore	May 1986	A
4593313	Nagasaki	Jun 1986	A
4621335	Bluish	Nov 1986	A
4625210	Sagl	Nov 1986	A
4630110	Cotton	Dec 1986	A
4632348	Keesling	Dec 1986	A
4638289	Zottnik	Jan 1987	A
4646241	Ratchford	Feb 1987	A
4651143	Yamanaka	Mar 1987	A
4671111	Lemelson	Jun 1987	A
4718685	Kawabe	Jan 1988	A
4754255	Sanders	Jun 1988	A
4758888	Lapidot	Jul 1988	A
4763745	Eto	Aug 1988	A
4785474	Bernstein	Nov 1988	A
4789904	Peterson	Dec 1988	A
4794566	Richards	Dec 1988	A
4804937	Barbiaux	Feb 1989	A
4806931	Nelson	Feb 1989	A
4807096	Skogler	Feb 1989	A
4814896	Heitzman	Mar 1989	A
4837628	Sasaki	Jun 1989	A
4839631	Tsuji	Jun 1989	A
4843463	Michetti	Jun 1989	A
4843578	Wade	Jun 1989	A
4853856	Hanway	Aug 1989	A
4853859	Morita	Aug 1989	A
4866616	Takeuchi	Sep 1989	A
4876597	Roy	Oct 1989	A
4883349	Mittelhauser	Nov 1989	A
4896855	Furnish	Jan 1990	A
4926331	Windle	May 1990	A
4930742	Schofield	Jun 1990	A
4936533	Adams	Jun 1990	A
4939652	Steiner	Jul 1990	A
4942464	Milatz	Jul 1990	A
4945244	Castleman	Jul 1990	A
4949186	Peterson	Aug 1990	A
4980913	Skret	Dec 1990	A
4987541	Levente	Jan 1991	A
4992943	McCracken	Feb 1991	A
4993068	Piosenka	Feb 1991	A
4995086	Lilley	Feb 1991	A
5012335	Cohodar	Apr 1991	A
5027104	Reid	Jun 1991	A
5046007	McCrery	Sep 1991	A
5050166	Cantoni	Sep 1991	A
5056056	Gustin	Oct 1991	A
5057820	Markson	Oct 1991	A
5096287	Kakinami	Mar 1992	A
5100095	Haan	Mar 1992	A
5111289	Lucas	May 1992	A
5140434	Van Blessinger	Aug 1992	A
5140436	Blessinger	Aug 1992	A
5140438	Kurahashi	Aug 1992	A
5144661	Shamosh	Sep 1992	A
5178448	Adams	Jan 1993	A
5185700	Bezos	Feb 1993	A
5196938	Blessinger	Mar 1993	A
5223844	Mansell	Jun 1993	A
5224211	Roe	Jun 1993	A
5262813	Scharton	Nov 1993	A
5283433	Tsien	Feb 1994	A
5294978	Katayama	Mar 1994	A
5305214	Komatsu	Apr 1994	A
5305216	Okura	Apr 1994	A
5308247	Dyrdek	May 1994	A
5309485	Chao	May 1994	A
5311197	Borden	May 1994	A
5321753	Gritton	Jun 1994	A
5327288	Wellington	Jul 1994	A
5330149	Haan	Jul 1994	A
5333759	Deering	Aug 1994	A
5343527	Moore	Aug 1994	A
5353023	Mitsugi	Oct 1994	A
5361326	Aparicio, IV	Nov 1994	A
5387926	Bellan	Feb 1995	A
5388045	Kamiya	Feb 1995	A
5388208	Weingartner	Feb 1995	A
5404330	Lee	Apr 1995	A
5408330	Squicciarini	Apr 1995	A
5422543	Weinberg	Jun 1995	A
5430431	Nelson	Jul 1995	A
5430432	Camhi	Jul 1995	A
5435184	Pineroli	Jul 1995	A
5445024	Riley, Jr.	Aug 1995	A
5445027	Zorner	Aug 1995	A
5446659	Yamawaki	Aug 1995	A
5455625	Englander	Oct 1995	A
5455716	Suman	Oct 1995	A
5465079	Bouchard	Nov 1995	A
5473729	Bryant	Dec 1995	A
5477141	Mather	Dec 1995	A
5495242	Kick	Feb 1996	A
5495243	McKenna	Feb 1996	A
5497419	Hill	Mar 1996	A
5499182	Ousborne	Mar 1996	A
5504482	Schreder	Apr 1996	A
5513011	Matsumoto	Apr 1996	A
5515285	Garrett, Sr.	May 1996	A
5519260	Washington	May 1996	A
5521633	Nakajima	May 1996	A
5523811	Wada	Jun 1996	A
5526269	Ishibashi	Jun 1996	A
5530420	Tsuchiya	Jun 1996	A
5532678	Kin	Jul 1996	A
5537156	Katayama	Jul 1996	A
5539454	Williams	Jul 1996	A
5541590	Nishio	Jul 1996	A
5544060	Fujii	Aug 1996	A
5546191	Hibi	Aug 1996	A
5546305	Kondo	Aug 1996	A
5548273	Nicol	Aug 1996	A
5552990	Ihara	Sep 1996	A
5559496	Dubats	Sep 1996	A
5568211	Bamford	Oct 1996	A
5570087	Lemelson	Oct 1996	A
5570127	Schmidt	Oct 1996	A
5574424	Nguyen	Nov 1996	A
5574443	Hsieh	Nov 1996	A
D376571	Kokat	Dec 1996	S
5581464	Woll	Dec 1996	A
5586130	Doyle	Dec 1996	A
5590948	Moreno	Jan 1997	A
5596382	Bamford	Jan 1997	A
5596647	Wakai	Jan 1997	A
5600775	King	Feb 1997	A
5608272	Tanguay	Mar 1997	A
5610580	Lai	Mar 1997	A
5612686	Takano	Mar 1997	A
5631638	Kaspar	May 1997	A
5638273	Coiner	Jun 1997	A
5642106	Hancock	Jun 1997	A
5646856	Kaesser	Jul 1997	A
5652706	Morimoto	Jul 1997	A
RE35590	Bezos	Aug 1997	E
5654892	Fujii	Aug 1997	A
5659355	Barron	Aug 1997	A
5666120	Kline	Sep 1997	A
5667176	Zamarripa	Sep 1997	A
5669698	Veldman	Sep 1997	A
5671451	Takahashi	Sep 1997	A
5677979	Squicciarini	Oct 1997	A
5680117	Arai	Oct 1997	A
5680123	Lee	Oct 1997	A
5686765	Washington	Nov 1997	A
5686889	Hillis	Nov 1997	A
5689442	Swanson	Nov 1997	A
5696705	Zykan	Dec 1997	A
5706362	Yabe	Jan 1998	A
5706909	Bevins	Jan 1998	A
5712679	Coles	Jan 1998	A
5717456	Rudt	Feb 1998	A
5719554	Gagnon	Feb 1998	A
5758299	Sandborg	May 1998	A
5781101	Stephen	Jul 1998	A
5781145	Williams	Jul 1998	A
5784007	Pepper	Jul 1998	A
5784021	Oliva	Jul 1998	A
5784521	Nakatani	Jul 1998	A
5790403	Nakayama	Aug 1998	A
5790973	Blaker	Aug 1998	A
5793308	Rosinski	Aug 1998	A
5793420	Schmidt	Aug 1998	A
5793739	Tanaka	Aug 1998	A
5793985	Natarajan	Aug 1998	A
5794165	Minowa	Aug 1998	A
5797134	McMillan	Aug 1998	A
5798458	Monroe	Aug 1998	A
5800040	Santo	Sep 1998	A
5802545	Coverdill	Sep 1998	A
5802727	Blank	Sep 1998	A
5805079	Lemelson	Sep 1998	A
5813745	Fant, Jr.	Sep 1998	A
5815071	Doyle	Sep 1998	A
5815093	Kikinis	Sep 1998	A
5819198	Peretz	Oct 1998	A
5825284	Dunwoody	Oct 1998	A
5825412	Hobson	Oct 1998	A
5844505	Van Ryzin	Dec 1998	A
5845733	Wolfsen	Dec 1998	A
5867802	Borza	Feb 1999	A
5877897	Schofield	Mar 1999	A
5896167	Omae	Apr 1999	A
5897602	Mizuta	Apr 1999	A
5897606	Miura	Apr 1999	A
5899956	Chan	May 1999	A
5901806	Takahashi	May 1999	A
5914748	Parulski	Jun 1999	A
5919239	Fraker	Jul 1999	A
5926210	Hackett	Jul 1999	A
5928291	Jenkins	Jul 1999	A
5938321	Bos	Aug 1999	A
5946404	Bakshi	Aug 1999	A
5948038	Daly	Sep 1999	A
5959367	OFarrell	Sep 1999	A
5978017	Tino	Nov 1999	A
6002326	Turner	Dec 1999	A
6006148	Strong	Dec 1999	A
6008723	Yassan	Dec 1999	A
6008841	Charlson	Dec 1999	A
6009370	Minowa	Dec 1999	A
6011492	Garesche	Jan 2000	A
6028528	Lorenzetti	Feb 2000	A
6037860	Zander	Mar 2000	A
6037977	Peterson	Mar 2000	A
6041410	Hsu	Mar 2000	A
6049079	Noordam	Apr 2000	A
6057754	Kinoshita	May 2000	A
6060989	Gehlot	May 2000	A
6064792	Fox	May 2000	A
6067488	Tano	May 2000	A
6076026	Jambhekar	Jun 2000	A
6084870	Wooten	Jul 2000	A
6088635	Cox	Jul 2000	A
6092008	Bateman	Jul 2000	A
6092021	Ehlbeck	Jul 2000	A
6092193	Loomis	Jul 2000	A
6100811	Hsu	Aug 2000	A
6111254	Eden	Aug 2000	A
6118768	Bhatia	Sep 2000	A
6122738	Millard	Sep 2000	A
6141611	Mackey	Oct 2000	A
6144296	Ishida	Nov 2000	A
6147598	Murphy	Nov 2000	A
6151065	Steed	Nov 2000	A
6163338	Johnson	Dec 2000	A
6163749	McDonough	Dec 2000	A
6167186	Kawasaki	Dec 2000	A
6170742	Yacoob	Jan 2001	B1
6181373	Coles	Jan 2001	B1
6182010	Berstis	Jan 2001	B1
6185490	Ferguson	Feb 2001	B1
6195605	Tabler	Feb 2001	B1
6200139	Clapper	Mar 2001	B1
6208919	Barkesseh	Mar 2001	B1
6211907	Seaman	Apr 2001	B1
6218960	Ishikawa	Apr 2001	B1
6246933	Bague	Jun 2001	B1
6246934	Otake	Jun 2001	B1
6252544	Hoffberg	Jun 2001	B1
6253129	Jenkins	Jun 2001	B1
6259475	Ramachandran	Jul 2001	B1
6263265	Fera	Jul 2001	B1
6266588	McClellan	Jul 2001	B1
6298290	Abe	Oct 2001	B1
6300875	Schafer	Oct 2001	B1
6324450	Iwama	Nov 2001	B1
6333759	Mazzilli	Dec 2001	B1
6337622	Sugano	Jan 2002	B1
6349250	Hart	Feb 2002	B1
6353734	Wright	Mar 2002	B1
6356823	Iannotti	Mar 2002	B1
6360147	Lee	Mar 2002	B1
6366207	Murphy	Apr 2002	B1
6389339	Just	May 2002	B1
6389340	Rayner	May 2002	B1
6400835	Lemelson	Jun 2002	B1
6405112	Rayner	Jun 2002	B1
6405132	Breed	Jun 2002	B1
6408232	Cannon	Jun 2002	B1
6411874	Morgan	Jun 2002	B2
6421080	Lambert	Jul 2002	B1
6434510	Callaghan	Aug 2002	B1
6449540	Rayner	Sep 2002	B1
6456321	Ito	Sep 2002	B1
6459988	Fan	Oct 2002	B1
6470241	Yoshikawa	Oct 2002	B2
6472771	Frese	Oct 2002	B1
6490513	Fish	Dec 2002	B1
6493650	Rodgers	Dec 2002	B1
6505106	Lawrence	Jan 2003	B1
6507838	Syeda-Mahmood	Jan 2003	B1
6508400	Ishifuji	Jan 2003	B1
6516256	Hartmann	Feb 2003	B1
6518881	Monroe	Feb 2003	B2
6525672	Chainer	Feb 2003	B2
6529159	Fan	Mar 2003	B1
6535804	Chun	Mar 2003	B1
6552682	Fan	Apr 2003	B1
6556905	Mittelsteadt	Apr 2003	B1
6559769	Anthony	May 2003	B2
6574538	Sasaki	Jun 2003	B2
6575902	Burton	Jun 2003	B1
6580373	Ohashi	Jun 2003	B1
6580983	Laguer-Diaz	Jun 2003	B2
6593848	Atkins, III	Jul 2003	B1
6594576	Fan	Jul 2003	B2
6611740	Lowrey	Aug 2003	B2
6611755	Coffee	Aug 2003	B1
6624611	Kirmuss	Sep 2003	B2
6629029	Giles	Sep 2003	B1
6629030	Klausner	Sep 2003	B2
6636791	Okada	Oct 2003	B2
6664922	Fan	Dec 2003	B1
6665613	Duvall	Dec 2003	B2
6679702	Rau	Jan 2004	B1
6684137	Takagi	Jan 2004	B2
6694483	Nagata	Feb 2004	B1
6701234	Vogelsang	Mar 2004	B1
6714894	Tobey	Mar 2004	B1
6718239	Rayner	Apr 2004	B2
6721640	Glenn	Apr 2004	B2
6721652	Sanqunetti	Apr 2004	B1
6728612	Carver	Apr 2004	B1
6732031	Lightner	May 2004	B1
6732032	Banet	May 2004	B1
6735503	Ames	May 2004	B2
6737954	Chainer	May 2004	B2
6738697	Breed	May 2004	B2
6739078	Morley	May 2004	B2
6741168	Webb	May 2004	B2
6745153	White	Jun 2004	B2
6747692	Patel	Jun 2004	B2
6748305	Klausner	Jun 2004	B1
6760757	Lundberg	Jul 2004	B1
6762513	Landgraf	Jul 2004	B2
6779716	Grow	Aug 2004	B1
6795017	Puranik	Sep 2004	B1
6795111	Mazzilli	Sep 2004	B1
6795759	Doyle	Sep 2004	B2
6798743	Ma	Sep 2004	B1
6804590	Sato	Oct 2004	B2
6810362	Adachi	Oct 2004	B2
6812831	Ikeda	Nov 2004	B2
6819989	Maeda	Nov 2004	B2
6831556	Boykin	Dec 2004	B1
6832140	Fan	Dec 2004	B2
6832141	Skeen	Dec 2004	B2
6836712	Nishina	Dec 2004	B2
6842762	Raithel	Jan 2005	B2
6847873	Li	Jan 2005	B1
6850823	Eun	Feb 2005	B2
6859695	Klausner	Feb 2005	B2
6859705	Rao	Feb 2005	B2
6862524	Nagda	Mar 2005	B1
6865457	Mittelsteadt	Mar 2005	B1
6867733	Sandhu	Mar 2005	B2
6873261	Anthony	Mar 2005	B2
6882313	Fan	Apr 2005	B1
6882912	DiLodovico	Apr 2005	B2
6894606	Forbes	May 2005	B2
6895248	Akyol	May 2005	B1
6898492	deLeon	May 2005	B2
6898493	Ehrman	May 2005	B2
6919823	Lock	Jul 2005	B1
6922566	Puranik	Jul 2005	B2
6928348	Lightner	Aug 2005	B1
6931309	Phelan	Aug 2005	B2
6947817	Diem	Sep 2005	B2
6950122	Mirabile	Sep 2005	B1
6954223	Miyazawa	Oct 2005	B2
6988034	Marlatt	Jan 2006	B1
7003289	Kolls	Feb 2006	B1
7012632	Freeman	Mar 2006	B2
7020548	Saito	Mar 2006	B2
7023333	Blanco	Apr 2006	B2
7039510	Gumpinger	May 2006	B2
7076348	Bucher	Jul 2006	B2
7079927	Tano	Jul 2006	B1
7082359	Breed	Jul 2006	B2
7082382	Rose, Jr.	Jul 2006	B1
7088387	Freeman	Aug 2006	B1
7095782	Cohen	Aug 2006	B1
7098812	Hirota	Aug 2006	B2
7100190	Johnson	Aug 2006	B2
7113853	Hecklinger	Sep 2006	B2
7117075	Larschan	Oct 2006	B1
7119832	Blanco	Oct 2006	B2
7138904	Dutu	Nov 2006	B1
7155321	Bromley	Dec 2006	B2
7177738	Diaz	Feb 2007	B2
7209833	Isaji	Apr 2007	B2
7239252	Kato	Jul 2007	B2
7254482	Kawasaki	Aug 2007	B2
7265663	Steele	Sep 2007	B2
7266507	Simon	Sep 2007	B2
7272179	Siemens	Sep 2007	B2
7308341	Schofield	Dec 2007	B2
7317974	Luskin	Jan 2008	B2
7343306	Bates	Mar 2008	B1
7348895	Lagassey	Mar 2008	B2
7349027	Endo	Mar 2008	B2
7370261	Winarski	May 2008	B2
7382933	Dorai	Jun 2008	B2
7386376	Basir	Jun 2008	B2
7389178	Raz	Jun 2008	B2
7398140	Kemwein	Jul 2008	B2
7457693	Olsen	Nov 2008	B2
7471189	Vastad	Dec 2008	B2
7471192	Hara	Dec 2008	B2
7536457	Miller	May 2009	B2
7548586	Mimar	Jun 2009	B1
7561054	Raz	Jul 2009	B2
7584033	Mittelsteadt	Sep 2009	B2
7623754	McKain	Nov 2009	B1
7659827	Gunderson	Feb 2010	B2
7659835	Jung	Feb 2010	B2
7667731	Kreiner	Feb 2010	B2
7689001	Kim	Mar 2010	B2
7702442	Takenaka	Apr 2010	B2
7725216	Kim	May 2010	B2
7768548	Silvernail	Aug 2010	B2
7769499	McQuade	Aug 2010	B2
7783956	Ko	Aug 2010	B2
7804426	Etcheson	Sep 2010	B2
7821421	Tamir	Oct 2010	B2
7845560	Emanuel	Dec 2010	B2
7853376	Peng	Dec 2010	B2
7868912	Venetianer	Jan 2011	B2
7893958	DAgostino	Feb 2011	B1
7940250	Forstall	May 2011	B2
7941258	Mittelsteadt	May 2011	B1
7974748	Goerick	Jul 2011	B2
8054168	McCormick	Nov 2011	B2
8068979	Breed	Nov 2011	B2
8090598	Bauer	Jan 2012	B2
8113844	Huang	Feb 2012	B2
8139820	Plante	Mar 2012	B2
8140265	Grush	Mar 2012	B2
8140358	Ling	Mar 2012	B1
8152198	Breed	Apr 2012	B2
8239092	Plante	Aug 2012	B2
8269617	Cook	Sep 2012	B2
8311858	Everett	Nov 2012	B2
8314708	Gunderson	Nov 2012	B2
8321066	Becker	Nov 2012	B2
8373567	Denson	Feb 2013	B2
8417562	Siemens	Apr 2013	B1
8442690	Goldstein	May 2013	B2
8471701	Yariv	Jun 2013	B2
8508353	Cook	Aug 2013	B2
8538696	Cassanova	Sep 2013	B1
8538785	Coleman	Sep 2013	B2
8564426	Cook	Oct 2013	B2
8564446	Gunderson	Oct 2013	B2
8571755	Plante	Oct 2013	B2
8577703	McClellan	Nov 2013	B2
8606492	Botnen	Dec 2013	B1
8634958	Chiappetta	Jan 2014	B1
8635557	Geise	Jan 2014	B2
8676428	Richardson	Mar 2014	B2
8744642	Nemat-Nasser	Jun 2014	B2
8775067	Cho	Jul 2014	B2
8781292	Ross	Jul 2014	B1
8803695	Denson	Aug 2014	B2
8805110	Rhoads	Aug 2014	B2
8849501	Cook	Sep 2014	B2
8855847	Uehara	Oct 2014	B2
8862395	Richardson	Oct 2014	B2
8868288	Plante	Oct 2014	B2
8880279	Plante	Nov 2014	B2
8892310	Palmer	Nov 2014	B1
8989959	Plante	Mar 2015	B2
8996234	Tamari	Mar 2015	B1
8996240	Plante	Mar 2015	B2
9047721	Botnen	Jun 2015	B1
9085362	Kilian	Jul 2015	B1
9183679	Plante	Nov 2015	B2
9201842	Plante	Dec 2015	B2
9208129	Plante	Dec 2015	B2
9226004	Plante	Dec 2015	B1
9240079	Lambert	Jan 2016	B2
9607526	Hsu-Hoffman	Mar 2017	B1
20010005217	Hamilton	Jun 2001	A1
20010005804	Rayner	Jun 2001	A1
20010018628	Jenkins	Aug 2001	A1
20010020204	Runyon	Sep 2001	A1
20010052730	Baur	Dec 2001	A1
20020019689	Harrison	Feb 2002	A1
20020027502	Mayor	Mar 2002	A1
20020029109	Wong	Mar 2002	A1
20020035422	Sasaki	Mar 2002	A1
20020044225	Rakib	Apr 2002	A1
20020059453	Eriksson	May 2002	A1
20020061758	Zarlengo	May 2002	A1
20020067076	Talbot	Jun 2002	A1
20020087240	Raithel	Jul 2002	A1
20020091473	Gardner	Jul 2002	A1
20020105438	Forbes	Aug 2002	A1
20020107619	Klausner	Aug 2002	A1
20020111725	Burge	Aug 2002	A1
20020111756	Modgil	Aug 2002	A1
20020118206	Knittel	Aug 2002	A1
20020120374	Douros	Aug 2002	A1
20020135679	Scaman	Sep 2002	A1
20020138587	Koehler	Sep 2002	A1
20020163532	Thomas	Nov 2002	A1
20020169529	Kim	Nov 2002	A1
20020169530	Laguer-Diaz	Nov 2002	A1
20020183905	Maeda	Dec 2002	A1
20030016753	Kim	Jan 2003	A1
20030028298	Macky	Feb 2003	A1
20030053433	Chun	Mar 2003	A1
20030055557	Dutta	Mar 2003	A1
20030065805	Barnes	Apr 2003	A1
20030067541	Joao	Apr 2003	A1
20030079041	Parrella	Apr 2003	A1
20030080713	Kirmuss	May 2003	A1
20030080878	Kirmuss	May 2003	A1
20030081121	Kirmuss	May 2003	A1
20030081122	Kirmuss	May 2003	A1
20030081127	Kirmuss	May 2003	A1
20030081128	Kirmuss	May 2003	A1
20030081934	Kirmuss	May 2003	A1
20030081935	Kirmuss	May 2003	A1
20030095688	Kirmuss	May 2003	A1
20030112133	Webb	Jun 2003	A1
20030125854	Kawasaki	Jul 2003	A1
20030144775	Klausner	Jul 2003	A1
20030152145	Kawakita	Aug 2003	A1
20030154009	Basir	Aug 2003	A1
20030158638	Yakes	Aug 2003	A1
20030177187	Levine	Sep 2003	A1
20030187704	Hashiguchi	Oct 2003	A1
20030191568	Breed	Oct 2003	A1
20030195678	Betters	Oct 2003	A1
20030214585	Bakewell	Nov 2003	A1
20030220835	Barnes	Nov 2003	A1
20030222880	Waterman	Dec 2003	A1
20040008255	Lewellen	Jan 2004	A1
20040033058	Reich	Feb 2004	A1
20040039503	Doyle	Feb 2004	A1
20040039504	Coffee	Feb 2004	A1
20040044452	Bauer	Mar 2004	A1
20040044592	Ubik	Mar 2004	A1
20040054444	Abeska	Mar 2004	A1
20040054513	Laird	Mar 2004	A1
20040054689	Salmonsen	Mar 2004	A1
20040064245	Knockeart	Apr 2004	A1
20040070926	Boykin	Apr 2004	A1
20040083041	Skeen	Apr 2004	A1
20040088090	Wee	May 2004	A1
20040103008	Wahlbin	May 2004	A1
20040103010	Wahlbin	May 2004	A1
20040104842	Drury	Jun 2004	A1
20040111189	Miyazawa	Jun 2004	A1
20040117638	Monroe	Jun 2004	A1
20040135979	Hazelton	Jul 2004	A1
20040138794	Saito	Jul 2004	A1
20040145457	Schofield	Jul 2004	A1
20040153244	Kellum	Aug 2004	A1
20040153362	Bauer	Aug 2004	A1
20040167689	Bromley	Aug 2004	A1
20040179600	Wells	Sep 2004	A1
20040181326	Adams	Sep 2004	A1
20040184548	Kerbiriou	Sep 2004	A1
20040203903	Wilson	Oct 2004	A1
20040209594	Naboulsi	Oct 2004	A1
20040210353	Rice	Oct 2004	A1
20040230345	Tzamaloukas	Nov 2004	A1
20040230370	Tzamaloukas	Nov 2004	A1
20040230373	Tzamaloukas	Nov 2004	A1
20040230374	Tzamaloukas	Nov 2004	A1
20040233284	Lesesky	Nov 2004	A1
20040236474	Chowdhary	Nov 2004	A1
20040243285	Gounder	Dec 2004	A1
20040243308	Irish	Dec 2004	A1
20040243668	Harjanto	Dec 2004	A1
20040254689	Blazic	Dec 2004	A1
20040254698	Hubbard	Dec 2004	A1
20040267419	Jeng	Dec 2004	A1
20050021199	Zimmerman	Jan 2005	A1
20050043869	Funkhouser	Feb 2005	A1
20050060070	Kapolka	Mar 2005	A1
20050060071	Winner	Mar 2005	A1
20050065682	Kapadia	Mar 2005	A1
20050065716	Timko	Mar 2005	A1
20050073585	Ettinger	Apr 2005	A1
20050078423	Kim	Apr 2005	A1
20050088291	Blanco	Apr 2005	A1
20050099498	Lao	May 2005	A1
20050100329	Lao	May 2005	A1
20050102074	Kolls	May 2005	A1
20050125117	Breed	Jun 2005	A1
20050131585	Luskin	Jun 2005	A1
20050131595	Luskin	Jun 2005	A1
20050131597	Raz	Jun 2005	A1
20050136949	Barnes	Jun 2005	A1
20050137757	Phelan	Jun 2005	A1
20050137796	Gumpinger	Jun 2005	A1
20050146458	Carmichael	Jul 2005	A1
20050149238	Stefani	Jul 2005	A1
20050149259	Cherveny	Jul 2005	A1
20050152353	Couturier	Jul 2005	A1
20050159964	Sonnenrein	Jul 2005	A1
20050166258	Vasilevsky	Jul 2005	A1
20050168258	Poskatcheev	Aug 2005	A1
20050171692	Hamblen	Aug 2005	A1
20050174217	Basir	Aug 2005	A1
20050182538	Phelan	Aug 2005	A1
20050182824	Cotte	Aug 2005	A1
20050185052	Raisinghani	Aug 2005	A1
20050185936	Lao	Aug 2005	A9
20050192749	Flann	Sep 2005	A1
20050197748	Holst	Sep 2005	A1
20050200714	Marchese	Sep 2005	A1
20050203683	Olsen	Sep 2005	A1
20050206741	Raber	Sep 2005	A1
20050209776	Ogino	Sep 2005	A1
20050212920	Evans	Sep 2005	A1
20050216144	Baldassa	Sep 2005	A1
20050228560	Doherty	Oct 2005	A1
20050233805	Okajima	Oct 2005	A1
20050251304	Cancellara	Nov 2005	A1
20050256681	Brinton	Nov 2005	A1
20050258942	Manasseh	Nov 2005	A1
20050264691	Endo	Dec 2005	A1
20050283284	Grenier	Dec 2005	A1
20060001671	Kamijo	Jan 2006	A1
20060007151	Ram	Jan 2006	A1
20060011399	Brockway	Jan 2006	A1
20060015233	Olsen	Jan 2006	A1
20060022842	Zoladek	Feb 2006	A1
20060025897	Shostak	Feb 2006	A1
20060030986	Peng	Feb 2006	A1
20060040239	Cummins	Feb 2006	A1
20060047380	Welch	Mar 2006	A1
20060053038	Warren	Mar 2006	A1
20060055521	Blanco	Mar 2006	A1
20060057543	Roald	Mar 2006	A1
20060058950	Kato	Mar 2006	A1
20060072792	Toda	Apr 2006	A1
20060078853	Lanktree	Apr 2006	A1
20060082438	Bazakos	Apr 2006	A1
20060092043	Lagassey	May 2006	A1
20060095175	deWaal	May 2006	A1
20060095199	Lagassey	May 2006	A1
20060095349	Morgan	May 2006	A1
20060103127	Lie	May 2006	A1
20060106514	Liebl	May 2006	A1
20060111817	Phelan	May 2006	A1
20060122749	Phelan	Jun 2006	A1
20060129578	Kim	Jun 2006	A1
20060142913	Coffee	Jun 2006	A1
20060143435	Kwon	Jun 2006	A1
20060147187	Takemoto	Jul 2006	A1
20060161960	Benoit	Jul 2006	A1
20060168271	Pabari	Jul 2006	A1
20060178793	Hecklinger	Aug 2006	A1
20060180647	Hansen	Aug 2006	A1
20060184295	Hawkins	Aug 2006	A1
20060192658	Yamamura	Aug 2006	A1
20060200008	Moore-Ede	Sep 2006	A1
20060200305	Sheha	Sep 2006	A1
20060204059	Ido	Sep 2006	A1
20060209090	Kelly	Sep 2006	A1
20060209840	Paatela	Sep 2006	A1
20060212195	Veith	Sep 2006	A1
20060215884	Ota	Sep 2006	A1
20060226344	Werth	Oct 2006	A1
20060229780	Underdahl	Oct 2006	A1
20060242680	Johnson	Oct 2006	A1
20060247833	Malhotra	Nov 2006	A1
20060253307	Warren	Nov 2006	A1
20060259218	Wu	Nov 2006	A1
20060261931	Cheng	Nov 2006	A1
20070001831	Raz	Jan 2007	A1
20070005404	Raz	Jan 2007	A1
20070027583	Tamir	Feb 2007	A1
20070027726	Warren	Feb 2007	A1
20070035632	Silvernail	Feb 2007	A1
20070043487	Krzystofczyk	Feb 2007	A1
20070100509	Piekarz	May 2007	A1
20070120948	Fujioka	May 2007	A1
20070124332	Ballesty	May 2007	A1
20070127833	Singh	Jun 2007	A1
20070132773	Plante	Jun 2007	A1
20070135979	Plante	Jun 2007	A1
20070135980	Plante	Jun 2007	A1
20070136078	Plante	Jun 2007	A1
20070142986	Alaous	Jun 2007	A1
20070143499	Chang	Jun 2007	A1
20070150138	Plante	Jun 2007	A1
20070150140	Seymour	Jun 2007	A1
20070173994	Kubo	Jul 2007	A1
20070179691	Grenn	Aug 2007	A1
20070183635	Weidhaas	Aug 2007	A1
20070208494	Chapman	Sep 2007	A1
20070213920	Igarashi	Sep 2007	A1
20070216521	Guensler	Sep 2007	A1
20070217670	Bar-Am	Sep 2007	A1
20070219685	Plante	Sep 2007	A1
20070219686	Plante	Sep 2007	A1
20070236474	Ramstein	Oct 2007	A1
20070241874	Okpysh	Oct 2007	A1
20070244614	Nathanson	Oct 2007	A1
20070253307	Mashimo	Nov 2007	A1
20070257781	Denson	Nov 2007	A1
20070257782	Etcheson	Nov 2007	A1
20070257804	Gunderson	Nov 2007	A1
20070257815	Gunderson	Nov 2007	A1
20070260677	DeMarco	Nov 2007	A1
20070262855	Zuta	Nov 2007	A1
20070263984	Sterner	Nov 2007	A1
20070268158	Gunderson	Nov 2007	A1
20070271105	Gunderson	Nov 2007	A1
20070273480	Burkman	Nov 2007	A1
20070279214	Buehler	Dec 2007	A1
20070280677	Drake	Dec 2007	A1
20070299612	Kimura	Dec 2007	A1
20080035108	Ancimer	Feb 2008	A1
20080059019	Delia	Mar 2008	A1
20080071827	Hengel	Mar 2008	A1
20080111666	Plante	May 2008	A1
20080122603	Plante	May 2008	A1
20080137912	Kim	Jun 2008	A1
20080143834	Comeau	Jun 2008	A1
20080147267	Plante	Jun 2008	A1
20080157510	Breed	Jul 2008	A1
20080167775	Kuttenberger	Jul 2008	A1
20080169914	Albertson	Jul 2008	A1
20080177436	Fortson	Jul 2008	A1
20080195261	Breed	Aug 2008	A1
20080204556	de Miranda	Aug 2008	A1
20080211779	Pryor	Sep 2008	A1
20080234920	Nurminen	Sep 2008	A1
20080243389	Inoue	Oct 2008	A1
20080252412	Larsson	Oct 2008	A1
20080252485	Lagassey	Oct 2008	A1
20080252487	McClellan	Oct 2008	A1
20080269978	Shirole	Oct 2008	A1
20080281485	Plante	Nov 2008	A1
20080309762	Howard	Dec 2008	A1
20080319604	Follmer	Dec 2008	A1
20090009321	McClellan	Jan 2009	A1
20090043500	Satoh	Feb 2009	A1
20090043971	Kim	Feb 2009	A1
20090051510	Follmer	Feb 2009	A1
20090138191	Engelhard	May 2009	A1
20090157255	Plante	Jun 2009	A1
20090216775	Ratliff	Aug 2009	A1
20090224869	Baker	Sep 2009	A1
20090290848	Brown	Nov 2009	A1
20090299622	Denaro	Dec 2009	A1
20090312998	Berckmans	Dec 2009	A1
20090326796	Prokhorov	Dec 2009	A1
20090327856	Mouilleseaux	Dec 2009	A1
20100030423	Nathanson	Feb 2010	A1
20100045451	Periwal	Feb 2010	A1
20100047756	Schneider	Feb 2010	A1
20100049516	Talwar	Feb 2010	A1
20100054709	Misawa	Mar 2010	A1
20100057342	Muramatsu	Mar 2010	A1
20100063672	Anderson	Mar 2010	A1
20100063680	Tolstedt	Mar 2010	A1
20100063850	Daniel	Mar 2010	A1
20100070175	Soulchin	Mar 2010	A1
20100076621	Kubotani	Mar 2010	A1
20100085193	Boss	Apr 2010	A1
20100085430	Kreiner	Apr 2010	A1
20100087984	Joseph	Apr 2010	A1
20100100315	Davidson	Apr 2010	A1
20100103165	Lee	Apr 2010	A1
20100104199	Zhang	Apr 2010	A1
20100149418	Freed	Jun 2010	A1
20100153146	Angell	Jun 2010	A1
20100157061	Katsman	Jun 2010	A1
20100191411	Cook	Jul 2010	A1
20100201875	Rood	Aug 2010	A1
20100220892	Kawakubo	Sep 2010	A1
20100250020	Lee	Sep 2010	A1
20100250022	Hines	Sep 2010	A1
20100250060	Maeda	Sep 2010	A1
20100250116	Yamaguchi	Sep 2010	A1
20100253918	Seder	Oct 2010	A1
20100268415	Ishikawa	Oct 2010	A1
20100283633	Becker	Nov 2010	A1
20100312464	Fitzgerald	Dec 2010	A1
20110035139	Konlditslotis	Feb 2011	A1
20110043624	Haug	Feb 2011	A1
20110060496	Nielsen	Mar 2011	A1
20110077028	Wilkes	Mar 2011	A1
20110091079	Yu-Song	Apr 2011	A1
20110093159	Boling	Apr 2011	A1
20110112995	Chang	May 2011	A1
20110121960	Tsai	May 2011	A1
20110125365	Larschan	May 2011	A1
20110130916	Mayer	Jun 2011	A1
20110140884	Santiago	Jun 2011	A1
20110145042	Green	Jun 2011	A1
20110153367	Amigo	Jun 2011	A1
20110161116	Peak	Jun 2011	A1
20110166773	Raz	Jul 2011	A1
20110172864	Syed	Jul 2011	A1
20110173015	Chapman	Jul 2011	A1
20110208428	Matsubara	Aug 2011	A1
20110212717	Rhoads	Sep 2011	A1
20110213628	Peak	Sep 2011	A1
20110224891	Iwuchukwu	Sep 2011	A1
20110251752	DeLarocheliere	Oct 2011	A1
20110251782	Perkins	Oct 2011	A1
20110254676	Marumoto	Oct 2011	A1
20110257882	McBurney	Oct 2011	A1
20110273568	Lagassey	Nov 2011	A1
20110282542	Nielsen	Nov 2011	A9
20110283223	Vaittinen	Nov 2011	A1
20110304446	Basson	Dec 2011	A1
20120021386	Anderson	Jan 2012	A1
20120035788	Trepagnier	Feb 2012	A1
20120041675	Juliver	Feb 2012	A1
20120046803	Inou	Feb 2012	A1
20120071140	Oesterling	Mar 2012	A1
20120078063	Moore-Ede	Mar 2012	A1
20120081567	Cote	Apr 2012	A1
20120100509	Gunderson	Apr 2012	A1
20120109447	Yousefi	May 2012	A1
20120123806	Schumann	May 2012	A1
20120134547	Jung	May 2012	A1
20120150436	Rossano	Jun 2012	A1
20120176234	Taneyhill	Jul 2012	A1
20120190001	Knight	Jul 2012	A1
20120198317	Eppolito	Aug 2012	A1
20120203402	Jape	Aug 2012	A1
20120210252	Fedoseyeva	Aug 2012	A1
20120269383	Bobbitt	Oct 2012	A1
20120277950	Plante	Nov 2012	A1
20120280835	Raz	Nov 2012	A1
20120283895	Noda	Nov 2012	A1
20120330528	Schwindt	Dec 2012	A1
20130004138	Kilar	Jan 2013	A1
20130006469	Green	Jan 2013	A1
20130018534	Hilleary	Jan 2013	A1
20130021148	Cook	Jan 2013	A1
20130028320	Gardner	Jan 2013	A1
20130030660	Fujimoto	Jan 2013	A1
20130048795	Cross	Feb 2013	A1
20130073112	Phelan	Mar 2013	A1
20130073114	Nemat-Nasser	Mar 2013	A1
20130096731	Tamari	Apr 2013	A1
20130127980	Haddick	May 2013	A1
20130145269	Latulipe	Jun 2013	A1
20130151980	Lee	Jun 2013	A1
20130170762	Marti	Jul 2013	A1
20130197774	Denson	Aug 2013	A1
20130209968	Miller	Aug 2013	A1
20130274950	Richardson	Oct 2013	A1
20130278631	Border	Oct 2013	A1
20130317711	Plante	Nov 2013	A1
20130332004	Gompert	Dec 2013	A1
20130345927	Cook	Dec 2013	A1
20130345929	Bowden	Dec 2013	A1
20140025225	Armitage	Jan 2014	A1
20140025254	Plante	Jan 2014	A1
20140032062	Baer	Jan 2014	A1
20140046550	Palmer	Feb 2014	A1
20140047371	Palmer	Feb 2014	A1
20140058583	Kesavan	Feb 2014	A1
20140089504	Scholz	Mar 2014	A1
20140094992	Lambert	Apr 2014	A1
20140098228	Plante	Apr 2014	A1
20140152828	Plante	Jun 2014	A1
20140226010	Molin	Aug 2014	A1
20140232863	Paliga	Aug 2014	A1
20140279707	Joshua	Sep 2014	A1
20140280204	Avery	Sep 2014	A1
20140300739	Mimar	Oct 2014	A1
20140309849	Ricci	Oct 2014	A1
20140335902	Guba	Nov 2014	A1
20140336916	Yun	Nov 2014	A1
20140339374	Mian	Nov 2014	A1
20150000415	Kelley	Jan 2015	A1
20150035665	Plante	Feb 2015	A1
20150057836	Plante	Feb 2015	A1
20150105934	Palmer	Apr 2015	A1
20150112542	Fuglewicz	Apr 2015	A1
20150112545	Binion	Apr 2015	A1
20150134226	Palmer	May 2015	A1
20150135240	Shibuya	May 2015	A1
20150156174	Fahey	Jun 2015	A1
20150170428	Harter	Jun 2015	A1
20150189042	Sun	Jul 2015	A1
20150203116	Fairgrieve	Jul 2015	A1
20150022449	Salinger	Aug 2015	A1
20150222449	Salinger	Aug 2015	A1
20150317846	Plante	Nov 2015	A1
20150371462	Ramesh	Dec 2015	A1
20160054733	Hollida	Feb 2016	A1

Foreign Referenced Citations (70)

Number	Date	Country
2469728	Dec 2005	CA
2469728	Dec 2005	CA
2692415	Aug 2011	CA
2692415	Aug 2011	CA
4416991	Nov 1995	DE
20311262	Sep 2003	DE
202005008238	Aug 2005	DE
102004004669	Dec 2005	DE
102004004669	Dec 2005	DE
0708427	Apr 1996	EP
0840270	May 1998	EP
0848270	Jun 1998	EP
1170697	Jan 2002	EP
1324274	Jul 2003	EP
1355278	Oct 2003	EP
1427165	Jun 2004	EP
1818873	Aug 2007	EP
2104075	Sep 2009	EP
2320387	May 2011	EP
2407943	Jan 2012	EP
2268608	Jan 1994	GB
2402530	Dec 2004	GB
2402530	Dec 2004	GB
244694	Aug 2008	GB
2451485	Feb 2009	GB
24471848	Jun 2011	GB
58085110	May 1983	JP
S5885110	May 1983	JP
62091092	Apr 1987	JP
S6291092	Apr 1987	JP
S62166135	Jul 1987	JP
02056197	Feb 1990	JP
H0256197	Feb 1990	JP
H04257189	Sep 1992	JP
H05137144	Jun 1993	JP
H08124069	May 1996	JP
H09163357	Jun 1997	JP
H09272399	Oct 1997	JP
10076880	Mar 1998	JP
H1076880	Mar 1998	JP
2002191017	Jul 2002	JP
2002191017	Jul 2002	JP
5294188	Sep 2013	JP
1000588169	Jun 2006	KR
8809023	Nov 1988	WO
9005076	May 1990	WO
9427844	Dec 1994	WO
9600957	Jan 1996	WO
9701246	Jan 1997	WO
9726750	Jul 1997	WO
9937503	Jul 1999	WO
9940545	Sep 1999	WO
9962741	Dec 1999	WO
0007150	Feb 2000	WO
0048033	Feb 2000	WO
0077620	Feb 2000	WO
0028410	May 2000	WO
0123214	Apr 2001	WO
0125054	Apr 2001	WO
0146710	Jun 2001	WO
03045514	Jun 2003	WO
2005095175	Oct 2005	WO
2006022824	Mar 2006	WO
2006022824	Mar 2006	WO
2007067767	Jan 2007	WO
2007109091	Sep 2007	WO
2009081234	Jul 2009	WO
2011055743	May 2011	WO
2013072939	May 2013	WO
2013159853	Oct 2013	WO

Non-Patent Literature Citations (196)

Entry
World News Tonight, CBC Television New Program discussing teen drivers using the DriveCam Program and DriveCam Technology, Oct. 10, 2005, on PC formatted CD-R, World News Tonight.wmv, 7.02 MB, Created Jan. 12, 2011. (1 pg.).
Written Opinion issued in PCT/US07/68328 on Oct. 15, 2007 (3 pgs.).
Written Opinion of the International Searching Authority for PCT/US2006/47042. dated Feb. 25, 2008 (5 pages).
Written Opinion of the International Searching Authority for PCT/US2006/47055, dated Mar. 20, 2008 (5 pages).
“DriveCam, Inc's Disclosure of Proposed Constructions and Extrinsic Evidence Pursuant to Patent L.R. 4.1.a & 4.1.b” Disclosure and Extrinsic Evidence in DriveCam, Inc. v. SmartDrive Systems, Inc., Case No. 3:11-CV-00997-H-RBB, for the Southern District of California. Nov. 8, 2011, 68 pages.
“DriveCam Driving Feedback System”, DriveCam brochure, Jun. 12, 2001, Document #6600128, 2 pages.
“DriveCam Driving Feedback System” DriveCam brochure, Mar. 15, 2004, 4 pages.
“DriveCam Passenger Transportation Module”, DriveCam brochure, Oct. 26, 2001, 2 pages.
“DriveCam Video Event Data Recorder”, DriveCam brochure, Nov. 6, 2002, Document #6600127, 2 pages.
“Responsive Claim Construction and Identification of Extrinsic Evidence of Defendani/Counterclaimant SmartDrive Systems, Inc.” Claim Construction and and Extrinsic Evidence in DriveCam, Inc. v. SmartDrive Systems, Inc., Case No. 3:11-CV-00997-H (RBB), for the Southern District of California. Nov. 15, 2011, 20 pages.
“Sonic MyDVD 4.0: Tutorial: Trimming video segments”. Tutorial for software bundled with Adaptec VideoOh! DVD USB 2.0 Edition, 2003, 13 pages.
“User's Manual for DriveCam Video Systems' HindSight 20/20 Software Version 4.0” DriveCam Manual, San Diego, 2003, Document #6600141-1, 54 pages.
Canadian Office Action issued in Application No. 2,632,685 dated Jan. 30, 2015; 5 pages.
Dan Maher, “DriveCam Taking Risk Out of Driving”, DriveCam brochure folder, Jun. 6, 2005, 6 pages.
Del Lisk, “DriveCam Training Seminar” Handout, 2004, 16 pages.
European Examination Report issued in EP 07772812.9 dated Jan. 22, 2015; 5 pages.
Jean (DriveCam vendor) “DriveCam Driving Feedback System”, DriveCam brochure, Nov. 6, 2002, Document #6600128-1, 2 pages.
Notice of Allowance Application for U.S. Appl. No. 11/566,424, dated Feb. 26, 2010, 6 pages.
Notice of Allowance for U.S. Appl. No. 11/377,164, dated Dec. 3, 2014, 5 pages.
Notice of Allowance for U.S. Appl. No. 11/377,164, dated Feb. 13, 2015, 2 pages.
Notice of Allowance for U.S. Appl. No. 11/377,164, dated Feb. 25, 2014, 2 pages.
Notice of Allowance for U.S. Appl. No. 11/377,164, dated Nov. 18, 2013, 7 pages.
Notice of Allowance for U.S. Appl. No. 11/377,167, dated Apr. 1, 2015, 7 pages.
Notice of Allowance for U.S. Appl. No. 11/800,876, dated Apr. 19, 2012, 8 pages.
Notice of Allowance for U.S. Appl. No. 13/957,810, dated Jun. 8, 2015, 10 pages.
USPTO Final Office Action for U.S. Appl. No. 11/296,906, dated Aug. 8, 2012, 15 pages.
USPTO Final Office Action for U.S. Appl. No. 12/096,591, dated Dec. 5, 2014, 23 pages.
USPTO Final Office Action for U.S. Appl. No. 12/096,591, dated Jul. 18, 2012, 15 pages.
USPTO Final Office Action for U.S. Appl. No. 12/096,591, dated Nov. 7, 2013, 14 pages.
USPTO Final Office Action for U.S. Appl. No. 13/957,810, dated Jun. 27, 2014, 22 pages.
USPTO Final Office Action for U.S. Appl. No. 14/036,299, dated Feb. 24, 2015, 9 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/296,906, dated Apr. 8, 2014, 19 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/296,906, dated Jun. 12, 2012, 13 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/377,164, dated Apr. 7, 2014, 7 pages.
USpto Non-Final Office Action for U.S. Appl. No. 11/377,164, dated Aug. 18, 2014, 5 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/377,164, dated Sep. 10, 2012, 10 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/377,167, dated Jun. 27, 2013, 11 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 12/096,591, dated Jun. 14, 2011, 8 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 12/096,591, dated Mar. 27, 2013, 16 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 13/957,810, dated Apr. 17, 2015, 6 pages.
USPTO Non-final Office Action for U.S. Appl. No. 13/957,810, dated Nov. 27, 2013, 18 pages.
Inovate Motorsports, OT-1 16 Channel 0BD-II Interface User Manual, Version 1.0, Nov. 28, 2007, pp. 3, 4, 21 and 27.
PCT International Search Report and Written Opinion for PCT/US15/60721 dated Feb. 26, 2016, 11 pages.
Trivinci Systems, LLC, “Race-Keeper System User Guide”, V1 .1.02, Jan. 2011, p. 21.
USPTO Non-Final Office Action dated Jan. 4, 2016 in U.S. Appl. No. 14/529,134, filed Oct. 30, 2014 (65 pgs).
PCT International Search Report and Written Opinion for PCT/IB16/51863, dated Sep. 16, 2016 (18 pgs.).
PCT International Search Report and Written Opinion for PCT/US2016/012757 dated Mar. 18, 2016 (11 pgs.).
Edwin Olson, A Passive Solution to the Sensor Synchronization Problem, the 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems, Oct. 18-22, 2010, 6 pages.
Patent Abstracts of Japan vol. 011, No. 292 (E-543), Sep. 19, 1987 (Sep. 19, 1987) & JP 62 091092 A (OK ENG:KK), Apr. 25, 1987 (Apr. 25, 1987) (1 pg.).
Patent Abstracts of Japan vol. 012, No. 001 (M-656), Jan. 6, 1988 (Jan. 6, 1988) & JP 62 166135 A (Fuji Electric Co Ltd), Jul. 22, 1987 (Jul. 22, 1987) (1 pg.).
Patent Abstracts of Japan vol. 014, No. 222 (E-0926), May 10, 1990 (May 10, 1990) & JP 02 056197 A (Sanyo Electric Co Ltd), Feb. 26, 1990 (Feb. 26, 1990) (1 pg.).
Patent Abstracts of Japan vol. 017, No. 039 (E-1311), Jan. 25, 1993 (Jan. 25, 1993) & JP 04 257189 A (Sony Corp), Sep. 11, 1992 (Sep. 11, 1992) (1 pg.).
Patent Abstracts of Japan vol. 017, No. 521 (E-1435), Sep. 20, 1993 (Sep. 20, 1993) & JP 05 137144 A (Kyocera Corp), Jun. 1 1993 (Jun. 1, 1993) (7 pgs.).
Patent Abstracts of Japan vol. 1996, No. 09, Sep. 30, 1996 (Sep. 30, 1996) & JP 08 124069 A (Toyota Motor Corp), May 17, 1996 (May 17, 1996) (15 pgs.).
Patent Abstracts of Japan vol. 1997, No. 10, Oct. 31, 1997 (Oct. 31, 1997) & JP 09 163357 A (Nippon Soken Inc), Jun. 20, 1997 (Jun. 20, 1997) (1 pg.).
Patent Abstracts of Japan vol. 1998, No. 02, Jan. 30, 1998 (Jan. 30, 1998) & JP 09 272399 A (Nippon Soken Inc), Oct. 21, 1997 (Oct. 21, 1997) (1 pg.).
Patent Abstracts of Japan vol. 1998, No. 8, Jun. 30, 1998 (Jun. 30, 1998) & Jp 10 076880 A (Muakami Corp), Mar. 24, 1998 (Mar. 24, 1998) (1 pg.).
PCT/US2010/022012, Invitation to Pay Additional Fees with Communication of Partial International Search, dated Jul. 21, 2010. (4 pgs.).
Peter G. Thurlow, Letter (including exhibits) Regarding Patent Owner's Response to Initial Office Action in Ex Parte Reexamination, dated Mar. 27, 2012. (78 pgs.).
Preliminary Claim Construction and Identification of Extrinsic Evidence of Defendant/Counterclaimant SmartDriveSystems, Inc.' in DriveCam, Inc. v. SmartDrive Systems, Inc., Case No. 3:11-CV-00997-H (RBB), for the Southern District of California. Nov. 8, 2011. (13 pgs.).
Quinn Maughan, ‘DriveCam Enterprise Services’, Jan. 5, 2006 (2 pgs.).
Quinn Maughan, ‘DriveCam Managed Services’, Jan. 5, 2006. (2 pgs.).
Quinn Maughan, ‘DriveCam Standard Edition’, Jan. 5, 2006. (2 pgs.).
Quinn Maughan, ‘DriveCam Unit Installation’, Jul. 21, 2005 (9 pgs.).
Quinn Maughan, ‘Enterprise Services’, Apr. 17, 2006 (2 pgs.).
Quinn Maughan, ‘Enterprise Services’, Apr. 7, 2006 (2 pgs.).
Quinn Maughan, ‘Hindsight Installation Guide’, Sep. 29, 2005 (55 pgs.).
Quinn Maughan, ‘Hindsight Users Guide’, Jun. 7, 2005 (80 pgs.).
Ronnie Rittenberry, ‘Eyes on the Road’, Jul. 2004 (2 pgs.).
SmartDrives Systems, Inc's Production, SO14246-S014255, Nov. 16, 2011 (1 pg.).
Supplement to DriveCam's Disclosure of Asserted Claims and Preliminary Infringement Contentions' in DriveCam, Inc. v. SmartDrive Systems, Inc., Case No. 3:11-CV-00997-H-RBB, for the Southern District of California. Oct. 14, 2011. (7 pgs.).
The DriveCam, Nov. 6, 2002. (2 pgs.).
The DriveCam, Nov. 8, 2002 (2 pgs.).
Traqmate GPS Data Acquisition's Traqmate Data Acquisition with Video Overlay system was used to create a video of a driving event on Oct. 2, 2005 (available at http://www.trackvision.net/phpBB2/viewtopic.php?t=51&sid=1184fbbcbe3be5c87ffa0f2ee6e2da76), printed from site on Jan. 11, 2012. (4 pgs.).
Trivinci Systems, LLC, Race-Keeper Systems User Guide, Jan. 2011, v1, 1.02, pp. 34 and 39.
U.S. Appl. No. 12/691,639, entitled ‘Driver Risk Assessment System and Method Employing Selectively Automatic Event Scoring’, filed Jan. 21, 2010. (52 pgs.).
U.S. Appl. No. 11/377,167, Final Office Action dated Nov. 8, 2013. (19 pgs.).
U.S. Appl. No. 11/377,157, filed Mar. 16, 2006 entitled, “Vehicle Event Recorder Systems and Networks Having Parallel Communications Links” (28 pgs.).
U.S. Appl. No. 11/377,167, filed Mar. 16, 2006 entitled, “Vehicle Event Recorder Systems And Networks Having Integrated Cellular Wireless Communications Systems” (29 pgs.).
USPTO Final Office Action for U.S. Appl. No. 11/297,669, dated Nov. 7, 2011, 15 pages.
USPTO Final Office Action for U.S. Appl. No. 13/957,810, dated Jun. 27, 2014, 24 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/296,906, dated Apr. 2, 2009, 7 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/296,906, dated Nov. 6, 2009, 9 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/297,669, dated Apr. 28, 2011, 11 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/299,028, dated Apr. 24, 2008, 9 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/377,164, dated Nov. 19, 2007, 7 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/377,164, dated Nov. 25, 2011, 9 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/377,164, dated Sep. 11, 2008, 8 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/377,167, dated Jun. 5, 2008, 11 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/800,876, dated Dec. 1, 2010, 12 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 11/800,876, dated Dec. 20, 2011, 8 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 12/096,591, dated May 20, 2014, 19 pages.
USPTO Non-Final Office Action for U.S. Appl. No. 14/036,299, dated Aug. 12, 2014. (14 pgs.).
USPTO Non-Final Office Action for U.S. Appl. No. 11/296,907, dated Mar. 22, 2007 (17 pages).
USPTO Non-final Office Action dated Aug. 27, 2009 during prosecution of U.S. Appl. No. 11/566,424 (25 pgs.).
USPTO Non-Final Office Action dated Nov. 27, 2013 in U.S. Appl. No. 13/957,810, filed Aug. 2, 2013. (19 pgs.).
Veeraraghavan, H., et al., Computer Vision Algorithms for Intersection Monitoring, IEEE Transactions on Intelligent Transportation Systems, 4(2):78-89 (2003); Digital Object Identifier 10.1109/TITS.2003.821212.
Wijesoma, W.S., et al., Road Curb Tracking in an Urban Environment, Proceedings of the Sixth International Conference of Information Fusion, 1:261-268 (2003).
GE published its VCR User's Guide for Model VG4255 in 1995. (44 pgs.).
Glenn Oster, ‘Hindsight 20/20 v4.0 Software Installation’, 1 of 2, Jun. 20, 2003 (2 pgs.).
Glenn Oster, ‘HindSight 20/20 v4.0 Software Installation’, 2 of 2, Jun. 20, 2003. (54 pgs.).
Glenn Oster, ‘Illuminator Installation’, Oct. 3, 2004 (2 pgs.).
Hans Fantel, Video; Search Methods Make a Difference in Picking VCR's, NY Times, Aug. 13, 1989. (1 pg.).
I/O Port Racing Supplies' website discloses using Traqmate's Data Acquisition with Video Overlay system in conjunction with professional driver coaching sessions (available at http://www.ioportracing.com/Merchant2/merchant.mvc?Screen=CTGY&Categorys-ub.-Code=coaching)., printed from site on Jan. 11, 2012. (4 pgs.).
Inovate Motorsports, OT-1 16 Channel OBD-II Interface User Manual, Version 1.0, Nov. 28, 2007, pp. 3, 4, 21 & 27.
Interior Camera Data Sheet', Oct. 26, 2001 (2 pgs.).
International Search Report and Written Opinion issued in PCT/US07/68325 dated Feb. 27, 2008. (10 pgs.).
International Search Report and Written Opinion issued in PCT/US07/68325 dated Feb. 27, 2008. (2 pgs.).
International Search Report and Written Opinion issued in PCT/US07/68328 dated Oct. 15, 2007. (2 pgs.).
International Search Report and Written Opinion issued in PCT/US07/68329 dated Mar. 3, 2008. (10 pgs.).
International Search Report and Written Opinion issued in PCT/US07/68332 dated Mar. 3, 2008. (8 pgs.).
International Search Report and Written Opinion issued in PCT/US07/68334 dated Mar. 5, 2008. (11 pgs.).
International Search Report for PCTUS2006/47055, dated Mar. 20, 2008 (2 pages).
International Search Report issued in PCT/US2006/47042 dated Feb. 25, 2008 (3 pgs.).
J. Gallagher, ‘Lancer Recommends Tech Tool’, Insurance and Technology Magazine, Feb. 2002 (1 pg.).
Jean (DriveCam vendor), ‘DC Data Sheet’, Nov. 6, 2002. (1 pg.).
Jean (DriveCam vendor), ‘DriveCam brochure’, Nov. 6, 2002. (4 pgs.).
Jean (DriveCam vendor), ‘Feedback Data Sheet’, Nov. 6, 2002. (1 pg.).
Jean (DriveCam vendor), ‘Hindsight 20-20 Data Sheet’, Nov. 4, 2002 (2 pg.).
Jessyca Wallace, ‘Analyzing and Processing DriveCam Recorded Events’, Oct. 6, 2003 (8 pgs.).
Jessyca Wallace, ‘Overview of the DriveCam Program’, Dec. 15, 2005 (13 pgs.).
Jessyca Wallace, ‘The DriveCam Driver Feedback System’, Apr. 6, 2004 (21 pgs.).
Joint Claim Construction Chart, U.S. Pat. No. 6,389,340, ‘Vehicle Data Recorder’ for Case No. 3:11-CV-00997-H-RBB, Document 43-1, filed Dec. 1, 2011, pp. 1-33.
Joint Claim Construction Chart in DriveCam, Inc. v. SmartDrive Systems, Inc., Case No. 11-CV-0997-H (RBB), for the Southern District of California, Document 43, filed Dec. 1, 2011, pp. 1-2.
Joint Claim Construction Worksheet, U.S. Pat. No. 6,389,340, ‘Vehicle Data Reporter’ for Case No. 3:11-CV-00997-H-RBB, Document 44-1, filed Dec. 1, 2011, pp. 1-10.
Joint Claim Construction Worksheet, U.S. Pat. No. 6,389,340, “Vehicle Data Reporter” for Case No. 3:11-CV-00997-H-RBB, Document 44-1, filed Dec. 1, 2011, pp. 1-10.
Joint Claim Construction Worksheet in DriveCam, Inc. v. SmartDrive Systems, Inc., Case No. 3:11-CV-00997 H (RBB), for the Southern District of California, Document 44, filed Dec. 1, 2011, pp. 1-2.
Joint Motion for Leave to Supplement Disclosure of Asserted Claims and Preliminary Infringement Contentions in DriveCam, Inc. v. SmartDrive Systems, Inc., Case No. 3:11-cv-00997-H-RBB, Document 29, filed Oct. 12, 2011, pp. 1-7.
Julie Stevens, ‘DriveCam Services’, Nov. 15, 2004 (9 pgs.).
Julie Stevens, ‘Program Support Roll-Out & Monitoring’, Jul. 13, 2004 (2 pgs.).
Jung, Sang-Hack, et al., Egomotion Estimation in Monocular Infra-red Image Sequence for Night Vision Applications, IEEE Workshop on Applications of Computer Vision (WACV '07), Feb. 2007, 6 pgs.; Digital Object Identifier 10.1109/WACV.2007.20.
JVC Company of America, JVC Video Cassette Recorder HR-IP820U Instructions (1996). (44 pgs.).
Kamijo, S., et al., A Real-Time Traffic Monitoring System by Stochastic Model Combination, IEEE International Conference on Systems, Man and Cybernetics, 4:3275-3281 (2003).
Kamijo, S., et al., An Incident Detection System Based on Semantic Hierarchy, Proceedings of the 7th International IEEE Intelligent Transportation Systems Conference, Oct. 3-6, 2004, pp. 853-858; Digital Object Identifier 10.1109/ITSC.2004.1399015.
Karen, ‘Downloading Options to HindSight 20120’, Aug. 6, 2002. (15 pgs.).
Karen, ‘Managers Guide to the DriveCam Driving Feedback System’, Jul. 30, 2002 (11 pgs.).
Kathy Latus (Latus Design), ‘Case Study—Cloud 9 Shuttle’, Sep. 23, 2005 (1 pg.).
Kathy Latus (Latus Design), ‘Case Study—Lloyd Pest Control’, Jul. 19, 2005 (1 pg.).
Kathy Latus (Latus Design), ‘Case Study—Time Warner Cable’, Sep. 23, 2005. (1 pg.).
Ki, Yong-Kul, et al., A Traffic Accident Detection Model using Metadata Registry, Proceedings of the Fourth International Conference on Software Engineering Research, Management and Applications; Aug. 9-11, 2006 pp. 255-259 Digital Object Identifier 10.1109/SERA.2006.8.
Kitchin, Charles. “Understanding accelerometer scale factor and offset adjustments.” Analog Devices (1995). (4 pgs.).
Lin, Chin-Teng et al., EEG-based drowsiness estimation for safety driving using independent component analysis; IEEE Transactions on Circuits and Systems-I: Regular Papers, 52(12):2726-2738 (2005); Digital Object Identifier 10.1109/TCSI.2005.857555.
Lisa Mckenna, ‘A Fly on the Windshield?’, Pest Control Technology Magazine, Apr. 2003 (2 pgs.).
Miller, D.P., Evaluation of Vision Systems for Teleoperated Land Vehicles. Control Systems Magazine, IEEE, 8(3):37-41 (1988); Digital Identifier 10.1109/37.475.
Munder, S., et al., Pedestrian Detection and Tracking Using a Mixture of View-Based Shape-Texture Models, IEEE Transactions on Intelligent Transportation Systems, 9(2):333-343 (2008); Digital Identifier 10.1109/TITS.2008.922943.
Panasonic Corporation, Video Cassette Recorder (VCR) Operating Instructions for Models No. PV-V4020/PV-V4520. (40 pgs.).
Passenger Transportation Mode Brochure, May 2, 2005. (2 pgs.).
Patent Abstracts of Japan vol. 007, No. 180 (P-215), Aug. 9, 1983 (Aug. 9, 1983) & JP 58 085110 A (Mitsuhisa Ichikawa), May 21, 1983 (May 21, 1983) (1 pg.).
Adaptec published and sold its VideoOh! DVD software USB 2.0 Edition in at least Jan. 24, 2003. (1 pg.).
Ambulance Companies Use Video Technology to Improve Driving Behavior, Ambulance Industry Journal, Spring 2003 (2 pgs.).
Amended Complaint for Patent Infringement, Trade Secret Misappropriation, Unfair Competition and Conversion in DriveCam, Inc. v. SmartDrive Systems, Inc., Case No. 3:11-CV-00997-H-RBB, for the Southern District of California, Document 34, filed Oct. 20, 2011, pp. 1-15.
Amended Complaint for Patent Infringement, Trade Secret Misappropriation, Unfair Competition and Conversion in DriveCam, Inc. v. SmartDrive Systems, Inc., Case No. 3:11-CV-00997-H-RBB, for the Southern District of California, Document 34, filed Oct 20, 2011, pp. 1-15.
Amendment filed Dec. 23, 2009 during prosecution of U.S. Appl. No. 11/566,424 (13 pgs.).
Answer to Amended Complaint; Counterclaims; and Demand for Jury Trial in DriveCam Inc. v. SmartDrive Systems, Inc., Case No. 3:11-CV-00997 H (RBB), for the Southern District of California, Document 47, filed Dec. 13, 2011, pp. 1-15.
Answer to Amended Complaint; Counterclaims; and Demand for Jury Trial in DriveCam, Inc. v. SmartDrive Systems, Inc., Case No. 3:11-CV-00997 H (RBB), for the Southern District of California, Document 47, filed Dec. 13, 2011, pp 1-15.
U.S. Appl. No. 11/296,906, filed Dec. 8, 2005, File History (279 pgs.).
U.S. Appl. No. 11/297,669, filed Dec. 8, 2005, File History (183 pgs.).
U.S. Appl. No. 11/297,889, filed Dec. 8, 2005, File History (77 pgs.).
U.S. Appl. No. 11/298,069, filed Dec. 9, 2005, File History (105 pgs.).
U.S. Appl. No. 11/299,028, filed Dec. 9, 2005, File History (181 pgs.).
U.S. Appl. No. 11/593,659, filed Nov. 7, 2006, File History (296 pgs.).
U.S. Appl. No. 11/593,682, filed Nov. 7, 2006, File History (173 pgs.).
U.S. Appl. No. 11/593,882, filed Nov. 7, 2006, File History (94 pgs.).
U.S. Appl. No. 11/595,015, filed Nov. 9, 2006, File History (171 pgs.).
U.S. Appl. No. 11/637,754, filed Dec. 13, 2006, File History (241 pgs.).
U.S. Appl. No. 11/637,755, filed Dec. 13, 2006, File History (171 pgs.).
Bill, ‘DriveCam—FAQ’, Dec. 12, 2003 (3 pgs.).
Bill Siuru, ‘DriveCam Could Save You Big Bucks’, Land Line Magazine, May-Jun. 2000 (1 pg.).
Chris Woodyard, ‘Shuttles save with DriveCam’, Dec. 9, 2003 (1 pg.).
Dan Carr, Flash Video Template: Video Presentation with Navigation, Jan. 16, 2006, http://www.adobe.com/devnet/fiash/articles/vidtemplate_mediapreso_flash8.html (13 pgs.).
David Cullen, ‘Getting a real eyeful’, Fleet Owner Magazine, Feb. 2002 (1 pg.).
David Maher, ‘DriveCam Brochure Folder’, Jun. 6, 2005 (2 pg.).
David Maher, “DriveCam Brochure Folder”, Jun. 8, 2005 (2 pg.).
David Vogeleer et al., Macromedia Flash Professional 8UNLEASHED (Sams Oct. 12, 2005). (4 pgs.).
Del Lisk, ‘DriveCam Training Handout Ver4’, Feb. 3, 2005 (16 pgs.).
Drivecam, Inc., User's Manual for Drivecam Video Systems' Hindsight 20/20 Software Version 4.0 (2003). (54 pgs.).
DriveCam, Inc.'s Infringement Contentions Exhibit A, U.S. Pat. No. 6,389,340, Document 34.1, Oct. 20, 2011. (10 pgs.)
DriveCam, Inc.'s Infringement Contentions Exhibit B, U.S. Pat. No. 7,659,827. Aug. 19, 2011. (29 pgs.).
DriveCam, Inc.'s Infringement Contentions Exhibit B, U.S. Pat. No. 7,804,426, Document 34.2, Oct. 20, 2011. (20 pg.).
DriveCam, Inc.'s Infringement Contentions Exhibit C, U.S. Pat. No. 7,659,827, Document 34.3, Oct. 20, 2011. (17 pgs.).
DriveCam, Inc.'s Infringement Contentions Exhibit C, U.S. Pat. No. 7,804,426. Aug. 19, 2011 (47 pgs.).
DriveCam, Inc.'s Infringement Contentions Exhibit D, Document 34.4, Oct. 20, 2011 (2 pgs.).
DriveCam—Illuminator Data Sheet, Oct. 2, 2004. (1 pg.).
Drivecam.com as retrieved by the Internet Wayback Machine as of Mar. 5, 2005(1 pg.).
DriveCam's Disclosure of Asserted Claims and Preliminary Infringement Contentions in DriveCam, Inc. v. SmartDrive Systems, Inc., Case No. 3:11-CV-00997-H-RBB, for the Southern District of California. Aug. 19, 2011. (6 pgs.).
DriveCam Driving Feedback System, Mar. 15, 2004 (12 pgs.).
DriveCam Extrinsic Evidence with Patent LR 4.1 .a Disclosures, Nov. 3, 2011. (1 pg.).
DriveCam Extrinsic Evidence with Patent LR 4.1 .a Disclosures, Nov. 8, 2011. (1 pg.).
Driver Feedback System, Jun. 12, 2001 (1 pg.).
First Amended Answer to Amended Complaint and First Amended Counterclaims; and Demand for Jury Trial in DriveCam, Inc. v. SmartDrive Systems, Inc., Case No. 3:11-CV-00997 H (RBB), for Southern District of California, Document 53, filed Dec. 20, 2011, pp. 1-48.
First Amended Answer to Amended Complaint and First Amended Counterclaims; and Demand for Jury Trial in DriveCam, Inc. v. SmartDrive Systems, Inc., Case No. 3:11-CV-00997 H (RBB), for Southern District of California, Document 55, filed Jan. 1, 2012, pp. 86-103.
First Amended Answer to Amended Complaint and First Amended Counterclaims; and Demand for Jury Trial in DriveCam, Inc. v. SmartDrive Systems, Inc., Case No. 3:11-CV-00997 H (RBB), for Southern District of California, Document 55, filed Jan. 3, 2012, pp. 49-103.
Franke, U., et al., Autonomous Driving Goes Downtown, IEEE Intelligent Systems, 13(6):40-48 (1988); Digital Object Identifier 10.1109/5254.736001.
Gallagher, B., et al., Wireless Communications for Vehicle Safety: Radio Link Performance and Wireless Connectivity Methods, Vehicular Technology Magazine, IEEE, 1(4):4-24 (2006); Digital Object Identifier 10.1109/MVT.2006.343641.
Gandhi, T., et al., Pedestrian Protection Systems: Issues, Survey, and Challenges, IEEE Transactions on Intelligent Transportation Systems, 8(3):413-430 (2007); Digital Object Identifier 10.1109/TITS.2007.903444.
Gary and Sophia Rayner, Final Report for Innovations Deserving Exploratory Analysis (IDEA) Intelligent Transportation Systems (ITS) Programs' Project 84, I-Witness Black Box Recorder, San Diego, CA. Nov. 2001. (19 pgs.).

Related Publications (1)

	Number	Date	Country
	20170069147 A1	Mar 2017	US

Continuations (1)

	Number	Date	Country
Parent	14055833	Oct 2013	US
Child	15357897		US

Vehicle event playback apparatus and methods

Information

Patent Number

Date Filed

Date Issued

Inventors

Original Assignees

Examiners

Agents

CPC

Field of Search

CPC

International Classifications

Disclaimer