Patent Grant
7427924
This invention relates to a driver monitoring device, system and method for determining and responding to fatigue and/or inattention of the driver.
The N.T.S.B. has reported statistics which indicate that 31% of all fatal-to-the-truckdriver accidents occur due to fatigue/inattention and 58% of all single-vehicle large truck crashes were also fatigue related. If these numbers can be reduced, so can the number of serious and fatal accidents.
Commercial drivers drive long hours each day and up to 70 hours per week. Such driving occurs throughout all hours of a day: daylight, dusk and night. Fatigue has long been a problem for commercial drivers. Fatigue is cumulative; without proper rest periods, or “off-duty” cycles, fatigue will “accumulate” in the human body and eventually create moments when the driver has to struggle to stay awake. Once this occurs, the overall performance of the driver is greatly compromised.
The Department of Transportation, the National Transportation Safety Board as well as N.H.T.S.A. have collectively invested millions of dollars in researching driver fatigue. Statistics show that over two decades, 30% to 40% of all serious accidents involving heavy trucks are in some way related to fatigue or inattention. Since driver fatigue can only be controlled to an extent, one object of this invention is to detect when the driver is falling asleep or becomes inattentive.
A driver fatigue and inattention monitoring system as described herein monitors whether a driver is sleeping or inattentive. This integrated system monitors the steering input behavior of the driver during a specified period of time. If the number of steering inputs is below the expected predetermined threshold, the system activates an alarm, such as an audible alarm and/or light in the cab, waking and/or stimulating the driver. Conversely, according to one preferred embodiment of this invention, if the steering input exceeds an expected predetermined threshold, the system activates the alarm. According to one preferred embodiment of this invention, this system can deactivate cruise control and/or activate various other truck systems/components connected with the system to further aid in the control of the truck and to alert nearby motorists.
Research indicates that as drivers begin to become inattentive or fall asleep, the amount of physical steering they do diminishes. This leads to “drifting” or lane changing that would not occur if the driver were awake and/or paying attention. During alert driving, drivers “micro-steer.” Micro-steering as used in the specification and claims herein is defined as a series of small steering movements which correct the course of the vehicle. If a driver stops micro-steering, the vehicle drifts or changes lanes. Micro-steering only occurs while the driver is awake. Therefore, there is a direct correlation between micro-steering behavior and driver wakefulness.
Further research concludes that, as a driver begins to fall asleep, they may “jerk” the steering wheel, called “jerk steer,” thereby exceeding an expected steering input. Such a sudden motion may be described as a large angular rotation of the steering wheel in a small time increment. According to one preferred embodiment of this invention, this correlation between sudden, large steering inputs and driver wakefulness may further be incorporated into the disclosed system.
A Driver Fatigue Monitoring System as described in the specification and claims is designed to monitor whether a driver is sleeping and/or inattentive. This integrated system monitors the steering input behavior of the driver during a specified period of time. If the number and/or degree of steering inputs is below or above the expected predetermined threshold, the system activates an audible alarm and light in the vehicle cab, waking the driver. In other words, if the driver is not reaching an expected level of micro-steering and/or if the driver exceeds an expected angular threshold with sudden jerk-steering, the system will provide an alert to the driver.
Furthermore, this system can deactivate cruise control and/or activate various other preprogrammed truck systems/components, like brake lights or hazard lights, to further aid in the control of the vehicle and to alert nearby motorists. A proof of concept system was assembled and installed on a 1996 Peterbilt 379 tractor.
One important advantage of the system is that driver sleep and/or fatigue is detected within seconds of onset, thereby awakening the driver. This saves lives, community property and cargo.
The subject invention provides positive results for each of several factors including: practicality, cost, engineering issues, maintainability, market acceptance, prevention of accident(s) and reduction/elimination of injury. This system has been evaluated on all accounts listed. Affirmative results were obtained for each of the factors.
The subject invention is intended to be a practical for both pre- and post-production application to vehicles. Standard off-the-shelf automotive components may be used for motion detection and off-the-shelf electronics (programmable logic controllers) were used. This makes the design cost effective and easy to maintain as no special parts must be developed or fabricated. Low cost and easy maintenance and effectiveness usually lead to market acceptance. As for the safety criteria, waking a sleeping driver will prevent accidents or allow the driver opportunity to mitigate them.
These and other objects and features of this invention will be better understood from the following descriptions taken in conjunction with the drawings wherein:
As shown schematically in
Steering input may be measured by calculating a steering input activity level measured over a time duration. The steering input activity level may be measured as a number of steering inputs, a degree rotation of a single steering input, or both.
For instance, in a preferred embodiment of this invention that alerts against micro-steering, steering input activity level may be measured to determine whether the number and/or degree of steering inputs is below the expected predetermined threshold. The invention may further or alternatively alert against jerk-steering and measure steering input activity to determine whether the number and/or degree of steering inputs is above the expected predetermined threshold. Thus, if the driver is not reaching an expected level of micro-steering and/or if the driver exceeds an expected angular threshold with sudden jerk-steering, the system will provide an alert to the driver.
Controller 40 is preferably connected with respect to sensor 30. Sensor 30 preferably provides a steering input count signal to controller 40 for processing as described in more detail below. Controller 40 preferably processes and identifies a condition that requires activation of alarm 50 connected with respect to controller 40. Alarm 50 is preferably positioned and/or activatable within operator and/or driver compartment 15 of vehicle 10. Alarm 50 is preferably activated when the steering input count signal drops below a minimum level and/or when the steering input rotation exceeds a maximum level.
Alarm 50 may comprise an audible alarm, such as a buzzer, horn, siren or similar such audible alarm. Additionally, or in the alternative, alarm 50 may comprise a light, such as a warning light, a strobe, a rotating light or similar visual cue. According to a preferred embodiment of this invention, whenever alarm 50 is activated, the driver or operator should manually reset alarm 50 to resume normal operation of vehicle 10.
According to one preferred embodiment of this invention shown schematically in
According to one variation of this embodiment, switch 75 is connected to speed control device 70 and alarm 50 wherein switch 75 is operable to reset alarm 50. As a result, if a driver trips alarm 50 resulting in interruption of speed control device 70, a manual operation such as resetting switch 75 is required to resume operation of speed control device 70, such as cruise control. In addition or alternatively, an engine brake may be activated following activation of alarm 50.
According to one preferred embodiment of this invention, also shown schematically in
Controller 40 may further include limiter 85 connected with respect to alarm 50 so that, following a predetermined number of alarm activations, limiter 85 reduces the performance of vehicle 10, such as by shutting down cylinders in engine 25, adjusting parameters within the engine management software, and/or other operation which results in a reduction of performance, particularly speed and acceleration capabilities, of vehicle 10. In addition, limiter 85 may work with engine management software to sequentially reduce vehicle 10 power and/or performance as alarm 50 is activated, with or without use of counter 80. For example, following activation of alarm 50 a first time, engine performance may be reduced by 25% following which a second activation of alarm 50 may reduce engine performance by 50% and so on until, theoretically, vehicle 10 may be completely immobilized upon X number of alarm activations, wherein X may be set by the owner/custodian or operator/driver of vehicle 10.
According to one particular arrangement of the invention, best shown in
Sensor 30 may comprise any number of sensors known to those having ordinary skill in the art, however, specifically, sensor 30 may comprise an ABS sensor or an industrial proximity sensor which are positioned relative to steering control 20 such as steering shaft 22 in vehicle 10 having traditional steering control or to an electronic control for certain contemporary drive-by-wire steering systems.
Controller 40 may comprise a programmable logic controller connected with sensor 30. The programmable logic controller preferably uses ladder logic to evaluate steering input to interpret a pulse signal generated by sensor 30.
According to an additional embodiment of the subject invention, recording device 90 is connected with respect to alarm 50. Recording device 90 preferably logs one or more parameters upon activation of alarm 50. Such parameters may include a time, a date and/or a speed at activation of alarm 50. Such logged data may then be analyzed by the owner/custodian of vehicle 10 or applicable authorities to determine the specific conditions that led to activation of alarm 50. In addition, global positioning system 95 may be connected with respect to recording device, the recording device logging at least one of a longitude, latitude and speed at activation of the alarm.
According to a preferred embodiment of this invention, the system has a self-check function which initiates at vehicle start-up. If system faults are detected at startup or during driving, a “check steering” light illuminates notifying the driver that the system has malfunctioned.
In addition to the failure signal light, the system reverts to “typical” speed control functions in the event of a monitoring system failure. This “fail to safety” philosophy is preferably applied to all other pre-programmed vehicle system options. That is, all other pre-programmed vehicle system options preferably go back to their normal way of operating as if the new monitoring system was not in place.
One important advantage of the system is that driver sleep and/or fatigue is detected within seconds of onset, thereby awakening the driver. This saves lives, community property and cargo.
Accordingly, a method of monitoring driver fatigue and inattention in vehicle 10 having steering control 20 includes connecting sensor 30 with respect to steering control 20 and controller 40 with respect to sensor 30. In addition, alarm 50 is connected with respect to controller 40. A steering input count signal and/or a steering wheel rotation signal is provided from sensor 30 to controller 40 and alarm 50 is activated when the steering input count signal drops below a minimum level or the steering wheel rotation signal exceeds a maximum level.
The steering input count signal may provide information that the driver's input count signal has dropped below a minimum level of expected micro-steer. The steering input count signal and/or the steering wheel rotation signal may further provide information that the driver's input count signal is exceeding a steering angle or rotation in a large time increment or a short time increment thus suggesting jerk-steer. Each of these conditions preferably results in activation of alarm 50. In addition, alarm 50 may be activated if there occurs a change from micro-steering to large amplitude steering thereby suggesting sleep, inattention and/or impairment.
As described in more detail above, one or more additional operations may be tied into alarm 50 including speed control device 70, engine brake, brake lights, flashers, counter 80, limiter 85 and/or any other vehicle operation that will assist in the continued safe operation and/or non-operation and/or reduced operation of vehicle 10. Several particular applications and methods of use of the subject invention are described in the following examples.
As described above, the driver fatigue monitoring system may comprise the following components: ABS tone-ring 35, ABS sensor, wiring and a controller 40 consisting of a networked I/O module. The networked I/O module may feature universal input-output ranges and a micro-controller to provide monitoring and control capabilities. This networked I/O module monitors discreet levels of various devices and/or provides on/off control capabilities. The networked I/O module function capability preferably includes control of: on/off; high/low; open/close switching; along with activation of audible and visual alarms.
Tone-ring 35, such as shown in
According to one preferred embodiment of this invention, as long as the networked I/O module detects a signal greater than a predetermined parameter, such as approximately 40 millivolts in approximately 5.4 seconds, no action is taken by the networked I/O module. If the signal drops below the predetermined parameter, for example, 40 millivolts in 5.4 seconds, a predetermined set of contacts close in the networked I/O module. Once these contacts close, the system preferably begins to activate. A desired configuration of alarm 50 preferably: (1) disables the cruise control (normally closed contacts), and (2) activates alarm 50 within driver compartment 15 such as an audible alarm and light (normally open contacts), such as shown in the wiring diagram in
This system may be comprised of components as described above, or alternatively, by non-automotive components with the exception of tone ring 35. An industrial proximity sensor may be used in place of the ABS sensor. The industrial proximity sensor has the ability to actually detect a discreet number of tooth counts from the tone ring, such as shown in
The tone ring 35 and proximity sensor preferably operate in a similar manner as the components described in Example 1. However, the signal generated from the proximity sensor is preferably fed into a programmable logic controller (PLC). The PLC preferably uses ladder logic to evaluate the steering input and the subsequent output actions. The PLC is programmed originally through an interface, such as a laptop computer, but can be reprogrammed through a touch pad on its face. As long as the PLC detects a signal (such as 3 pulses in 6 seconds), no action is taken.
If the signal drops below 3 pulses in 6 seconds, the output logic begins to function and deactivates the cruise control and activates alarm 50, such as an audible alarm and light within driver compartment 15. Once this activation has occurred, the system will remain “on” disallowing future cruise mode and continuing the alarm and light. Preferably, this activation may only be deactivated by cycling switch 75, such as the cruise control on/off switch. Once the on/off switch is cycled, the system can be reactivated.
Alternatively, if the signal exceeds a defined steering angle or rotation in a minimum time period, the output logic may likewise function and deactivate cruise control and activate alarm 50. When such defined steering angles and/or rotations are exceeded, the system according to this invention preferably deactivates cruise control thereby eliminating the need for the driver to step on the brake pedal for manual deactivation.
The cruise control is preferably deactivated the same way for both Examples 1 and 2. Specifically, the networked I/O module of Example 1 or the PLC of Example 2 eliminate the electrical ground source at the cruise on/off dash switch location. Once the switch ground has been interrupted, the cruise control drops out and no longer will function until the system re-initiates.
The driver fatigue/inattention monitoring system and method as described has numerous applications including automobiles, SUVs, buses and a variety of trucks. A principle function of the system is to detect sleeping or inattentive drivers. Expanding the system to control other vehicle system controls can be customized based on the vehicle, driver, community and vehicle operational profile.
As described above, the system may include recording device 90 to record an “event” if sleep is detected, that can be logged on the PLC for later extraction and analysis. The event data may include time, date, speed(s), etc. If the vehicle utilizes satellite navigation, an “event” can be time stamped with: longitude, latitude, speed and can automatically send an emergency message. In addition, a cruise control termination strategy could be developed which would eventually preclude the use of cruise control after repeated activations of the sleep detection system. Only after a prescribed “lockout” time has been met would the cruise be reactivated. Along with this to force the driver to stop driving and rest, an engine de-rating schedule similar to engine protection logic may be utilized. This would begin to reduce engine performance sequentially (i.e., 75%, 50%, 25% power) after multiple triggers, if the driver refused to stop driving and rest.
Additional hardware and/or software may be included which eliminate the possibility of false steering inputs caused by road vibrations. For example, a triaxial accelerometer may be incorporated to record road vibrations and then have them subtracted out of the count. Alternatively, or in addition, tone-ring 35 and/or sensor 30 may be mounted to a portion of steering shaft 22 within driver compartment 15.
According to one preferred embodiment of this invention, the system may be adjustable for time duration, steering input counts and/or steering angles/rotation. Depending upon the vehicle, industry, environment, etc., adjustments may be made to parameters such as time duration and steering input counts based upon parameters that are important to the application. For example, steering inputs counts may be increased for use on winding mountain roads versus use on long stretches of interstate highways. In addition, adjustments may be made in steering angles/rotation depending on the “lock-to-lock” steering response ratio of the particular vehicle. Protections may be incorporated so that only authorized maintenance personnel with personal identification numbers (PINS) can make such adjustments. Time stamping the adjustment along with other critical chronology data will assist with accountability.
According to one preferred embodiment of this invention, the system may use adaptive learning similar to fuel trim control or shift scheduling in some automatic transmissions. This approach will allow for specific driving habits to be recognized and learned by the system. Once accomplished, the system may adjust for a driver's normal or expected behavior and can more accurately distinguish sleeping, inattentiveness and/or fatigue. Such adaptive learning may allow the system to: identify driver behavior patterns and variations therefrom; detect and compare unusual driving patterns with expected driving patterns; learn road conditions; adjust for steering system wear; and/or recognize a maintenance failure such as loose or worn suspension linkages, bearings, tires, steering boxes and/or similar components.
The system may further accommodate for “bump steer” due to jouncing; “road steer” due to roadway crown and other grooves/bumps; and other environmental factors. Additional steering input sensors may be incorporated into vehicle 10 to detect if the road or the driver has initiated the most recent steering motion.
Additionally, mechanisms to recognize steering box wear, with respect to age and mileage, that create more active steering by the driver to maintain guidance of the vehicle may be incorporated. Such customization may further include determination of a steering “center” to account for vehicle turning as with a gradual lane change or perhaps misalignment.
According to one preferred embodiment of this invention, the system may remain in an off or “sleep” mode until the vehicle reaches a predetermined speed such as 45 mph. In addition, the system of this invention may be used in connection with other methods of sleep/inattention/impairment detection such as driver EEG, EKG, eyelid blink, facial droop/changes, pupil reactions, etc.
While in the foregoing specification this invention has been described in relation to certain preferred embodiments thereof, and many details have been set forth for purpose of illustration, it will be apparent to those skilled in the art that the invention is susceptible to additional embodiments and that certain of the details described herein can be varied considerably without departing from the basic principles of the invention.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4017843 | Yanagishima | Apr 1977 | A |
| 4234051 | Morris, Jr. | Nov 1980 | A |
| 4278969 | Woods | Jul 1981 | A |
| 4450438 | Seko et al. | May 1984 | A |
| 4463347 | Seko et al. | Jul 1984 | A |
| 4496938 | Seko et al. | Jan 1985 | A |
| 4581607 | Seko et al. | Apr 1986 | A |
| 4611199 | Seko et al. | Sep 1986 | A |
| 4725824 | Yoshioka | Feb 1988 | A |
| 6198397 | Angert et al. | Mar 2001 | B1 |
| 6366207 | Murphy | Apr 2002 | B1 |
| 7138923 | Ferrone et al. | Nov 2006 | B2 |
| Number | Date | Country | |
|---|---|---|---|
| 20060180379 A1 | Aug 2006 | US |
