Patent Application
20200314736
Not Applicable.
Not Applicable.
The present invention relates in general to event data recording in road vehicles for accident or other event reconstruction, and, more specifically, to monitoring a wireless environment for inclusion with recorded video as part of the recorded event data.
Event data recording systems have been deployed in some vehicles in order to preserve information about the vehicle and its surroundings around the time of a significant event, such as a collision. One type of system is a “Dash Camera” system which captures video information about the environment either continuously or beginning with the detection of a trigger event. Known systems have typically included a single forward-looking camera or multiple cameras in order to provide multiple views or even a full 360° view. However, these systems have limitations due to inherent properties of the cameras (e.g., limited field of view or range, resolution, light sensitivity, frame rate, etc.) and of the environment (light blockage, weather, etc.). In certain cases, every possible scrap of information could play an important role in ascertaining the causes of an event. The captured images might not provide a whole picture of the influences upon, participants in, or witnesses to an event. Furthermore, the captured images may provide insufficient details to reliably identify a vehicle or person fleeing an event.
In preferred embodiments of the invention, a vehicle imaging system is augmented to capture information about the wireless activities within range of the vehicle, and the captured wireless information is stored in a memory (e.g., in the vehicle or sent wirelessly from the vehicle to a remote location such as cloud storage) at or near the time of an event in order to be preserved for later reconstruction of the event. Metadata relating to various types of wireless information can be utilized.
In one example, WiFi transmissions in 2.4 GHz and 5 GHz frequency bands can be passively monitored. The WiFi signals may be emitted from a smart phone or other portable device to a wireless router (e.g., in a unit located near a roadway or in a vehicle). The metadata being logged can include the SSID, MAC address, and a measured signal strength from each WiFi access point or WiFi device nearby. To provide a thorough snapshot of the wireless environment, all potential WiFi channels can be scanned. Reception via multiple WiFi antennas on the vehicle can be used in order to triangulate the position of the device based on relative signal strengths at the different antennas.
In another example, active monitoring is performed for nearby Bluetooth devices (e.g., cellphones and wearable devices) by sending a presence request. For detected devices, the signal strength is logged and the metadata may provide a list of devices with the signal strengths. Again, multiple WiFi antennas can be deployed on the vehicle to triangulate the position of the device. Bluetooth emissions can also be passively monitored.
The wireless environment to be monitored can also include RF signals using the Dedicated Short Range Communications (DSRC) protocol within an Intelligent Transportation System (ITS). The monitored signals may include vehicle-to-vehicle (V2V) or vehicle-to-infrastructure (V2I) messages, wherein the stored metadata can include contents of such messages (e.g., location, vehicle states, and traffic signals) as well as RF signal strength and the number/identity of sources.
In another example, transmissions from nearby cellphones or other portable devices to a cell tower are monitored. Triangulation of location can be supported by recording the signal strengths for each particular device at the multiple antennas.
Other example devices include drones or drone remote controllers (for both data and control signals) and wireless security cameras which often transmit in 2.4 GHz, 5 GHz, or 5.8 GHz bands.
For the above examples of established radio services using licensed or unlicensed RF access, the vehicle may preferably include an appropriate receiver adapted to detect the corresponding RF signals. Other aspects of the RF environment can also be monitored such as electromagnetic interference (EMI) from nearby unintentional emitters (e.g., microwave ovens or electric motors) or intentional transmitters using an RF service for which a dedicated receiver is not present (e.g., aircraft). The EMI can be received using a wideband receiver and the signal characterized using a spectrum analyzer, for example.
In one aspect of the invention, vehicular apparatus comprises a plurality of wireless receivers detecting respective radio frequency (RF) signals from external sources. As used herein, “external source” includes portable devices (i.e., sources not fixed to the vehicle) carried by persons either inside or outside the vehicle. A central controller compiles wireless metadata based on the detected RF signals. A buffer memory is provided for queueing newly compiled metadata together with respective timestamps identifying times of collection of the metadata. A non-volatile memory provides for longer term storage of respective metadata from the buffer memory upon occurrence of a trigger event. The wireless metadata may preferably include network identifiers of active WiFi devices, active cellular telephone channels, contents or source identifiers of detected DSRC messages (i.e., V2V or V2I communications), spectrum data of detected EMI sources, and a listing of available Bluetooth devices. The compiled metadata may preferably include a measured signal strength for an RF signal detected by a receiver.
Augmenting an event data recording system to monitor and preserve metadata relating to a wireless RF environment provides significant new advantages in event reconstruction, law enforcement, and personal security. For event reconstruction, wireless information logged near the time of the incident can be used to reconstruct various details, including estimating the number of occupants in a vehicle, identifying the vehicle occupants based on detected devices, or estimating the number of pedestrians or other vehicles that were near the incident. In connection with a hit and run event, the captured wireless environment information obtained before the offending vehicle drives away can include wireless signatures that can be analyzed along with the video data to help identify the vehicle. The preserved metadata can be used by law enforcement to detect wireless signals and isolate certain signatures in the data to match against various databases. In regard to personal security, a user can review the wireless environment to check whether anyone carrying a portable wireless device is located near the vehicle before exiting the vehicle in a remote area, for example.
Many vehicles would already be provided with sensors (i.e., receivers) that detect a variety of wireless signals (e.g., WiFi, Bluetooth, BLE, and Cellular). Additional receivers can be added to cover additional bands or to enhance detection performance. During times that the vehicle is operating normally, each of the receivers will monitor their corresponding wireless signals and store relevant metadata information (e.g., device IDs, signal strength, and location) into a memory buffer such as a ring buffer with a capacity to store all the metadata collected for at least a desired duration (e.g., 30 seconds, 60 seconds, or 90 seconds). The ring buffer size preferably configured to be sufficient to hold twice the data or more that is collected during a desired snapshot duration, in order to ensure definite availability of snapshot data of the configured duration whenever an event occurs. The central module ring buffer preferably stores the metadata from all the receivers with respective timestamps without any data loss. For example, if one receiver/sensor reports samples every 100 ms and another every 1 ms, each of them will be stored at their native frequency.
Preferably, the vehicle uses several different onboard antennas working at different respective frequency bands to produce a robust wireless map of the vehicle wireless surroundings. Known types of receivers or transceivers are used to detect, demodulate, and/or evaluate RF signals for various known radio services (e.g., WiFi, Bluetooth, DSRC, and cellular). For unintentional radiators or transmitters for other radio services, an on-chip spectrum analyzer can be used to detect and quantify electromagnetic interference, e.g., in the 2.4 GHz, 5.8 GHz, and 5.9 GHz bands. A known type of WiFi chip can be dual used for this purpose.
When a trigger event (e.g., an acceleration or deceleration event, or a user-triggered event button) is detected, the RF metadata information from the vehicle receivers compiled by a central storage module (e.g., a Dash Camera module, a Body Control Module, or other module in vehicle) is transferred to a non-volatile memory which may be located in the vehicle or offboard. The receivers can alternatively send the wireless environmental data for compilation and long term storage after the trigger event occurs.
Referring to
Also within field of view 21 are a pedestrian 25 having a portable wireless device 26 and a pedestrian 27 having a portable wireless device 28. Devices 26 and 28 may be comprised of smartphones, for example. RF signals 29 are emitted which can be comprised of cellular signals, Bluetooth signals, Wi-Fi signals, or any other form of wireless communication. In the case of cellular signals, devices 26 and 28 are in communication with a cellular tower 30 interconnected with a carrier network 31. Receivers 22 in host vehicle 20 can be configured to monitor cellular phone signals 29 from nearby devices 26 and 28 and/or cellular signals from tower 30. Not within field of view 21, a pedestrian 32 carries a portable wireless device 33. The corresponding RF signals of wireless device 33 may be monitored and detected as respective components in the wireless environment to be included in the recorded data for an event. In addition, portable devices (not shown) can be detected and monitored by host vehicle 20 which have been carried into the passenger compartment of vehicle 20 (i.e., they are external sources in the sense that they are not hard-wired to vehicle 20).
The invention can monitor other fixed sources within the wireless environment. For example, a roadside node 36 of an intelligent transportation system (ITS) is within the range of host vehicle 20 which transmits DSRC signals 37 as part of V2I communications. The DSRC signals which can be detected, demodulated, and compiled into the stored data for recording during a trigger event. The metadata compiled from DSRC signals in a preferred embodiment may include the number of sources and addresses or other identifiers of the sources. The metadata can further include detected and/or demodulated signals representing ITS information such as the state of traffic control signals (traffic lights), vehicle IDs, and vehicle location, heading, and speed.
Another fixed external source of RF signals may include a building 40 having an RF transmitter 41 such as a WiFi router which emits RF signals 42. Building 40 or other facilities may also include electrical equipment such as a device 43 (e.g., a motor) which unintentionally or passively radiates RF signals 44 as electromagnetic interference. Signals 44 contribute to the monitored wireless environment and can be recorded to indicate various types of activity that may be in progress during a trigger event.
When a trigger event is detected in step 53, then the contents currently queued with the buffer memory are written in step 55 into a nonvolatile memory in the vehicle or some other location such as an offboard server in a cloud. The nonvolatile memory provides longer-term storage to facilitate accident reconstruction or other analysis after an event has occurred. In step 56, an event counter is increased so that a subsequent trigger event results in additional ring buffer data being written to a different portion of the nonvolatile memory.
One preferred vehicle or apparatus of the invention as shown in
A user interface 71 is connected to central module 60 adapted for use in configuring the system and for providing local user access to the compiled and/or stored information from nonvolatile memory 70 and/or from the buffer memory. A remote interface 72 can be provided for sharing of recorded event data with third parties, e.g., wirelessly sharing to a call center or law enforcement. The vehicle apparatus includes a trigger detector 73 which may utilize a signal generated in a vehicle powertrain controller or safety restraints controller (not shown) or may include a user control button within user interface 71.
Data collection and compilation of wireless metadata is shown in greater detail in
The invention may perform active collection of RF signals from external sources using an interrogating transmitter 83 to send presence requests or other interrogating signals to a two-way RF device 77, such as a Bluetooth transceiver. The presence request elicits the emission of responsive RF signals from two-way device 77 which are picked up by a detector/demodulator 84 which provides the same or similar types of wireless metadata 81 to be compiled within central control module 60.
Within central module 60, the compiled wireless metadata 81 is combined with timestamps and video signals in a combiner 85, and a resulting data stream is queued in a buffer memory 86. Preferably, buffer memory 86 is comprised of a ring buffer which queues newly compiled metadata according to a continuously updated data pointer so that the oldest metadata within buffer 86 is continuously overwritten by newly compiled metadata. Preferably, the full contents of ring buffer 86 are recorded as a snapshot by transferring to the nonvolatile memory whenever a trigger event occurs.
