The present invention relates generally to the automated control of onboard vehicle systems, and in particular to an apparatus and method for identifying an authorized driver of a vehicle using onboard system settings and then controlling an onboard vehicle system in accordance with a modeled profile of the authorized driver.
Modem vehicle design strives to achieve a seamless interaction between the architecture of various onboard vehicle systems and an operator or driver of the vehicle. Generally, interaction between the vehicle systems and a driver can be divided into three levels or classifications: access, accommodation, and dynamic control. With respect to access, the vehicle system can be configured such that only certain authorized drivers can operate the vehicle. With respect to accommodation, the vehicle's interior and/or exterior systems can be adjusted in conjunction with known preferences of the driver. With respect to dynamic control, the vehicle's dynamic characteristics can be uniquely tailored to the known preferences of its present driver.
In particular, access can be controlled by granting a potential driver access to a vehicle only if that driver has a portable device such as a key fob, a radio frequency identification (RFID) device or tag, etc. However, possession of the portable device may allow some unauthorized drivers access the vehicle. To enhance overall vehicle security, a popular trend is to employ driver identification methodologies to further verify the authority of a potential driver with respect to the vehicle. Some exemplary state-of-the-art driver identification methodologies and security measures include identifying the unique biometric characteristics of the driver, e.g., the driver's fingerprints, finger veins, iris patterns, retinal patterns, handprints, voice recognition, facial recognition, speech recognition, etc. Once affirmatively identified in this manner, the driver is considered to be authorized, and the vehicle can be accessed by that driver. However, biometric sensors and processing algorithms can add considerable cost and complexity to a vehicle.
Regarding accommodation and dynamic control, some vehicles allow each operator or driver of the vehicle to record his or her preferred vehicle system settings, driving preferences, and/or driving style within an individual user profile, with each driver selecting from among the stored user profiles upon entering the vehicle. Once a desired profile is selected, an electronic control unit or controller retrieves the corresponding setting information for various vehicle systems and adjusts the associated control settings accordingly. As with the access methods described above, preset profiles can require the affirmative selection of a profile, with the profiles being static values. However, despite the many technical advances in the levels or classifications of access, accommodation, and dynamic control as described above, existing vehicle systems and control methods remain less than optimal, particularly as they relate to the automatic and seamless customization of vehicle systems settings for a given driver over a variety of driving conditions.
Accordingly, a method and apparatus provide adaptive driver recognition based on a driver's present vehicle settings and automatic control of an onboard vehicle system using that driver's identity. That is, the method and apparatus can statistically-model certain highly descriptive or sensitive vehicle settings along with discrete vehicle settings to generate a historical vehicle system setting profile unique to that particular driver, with this profile referred to hereinafter as the historical driver profile (HDP) for simplicity.
More specifically, adaptive in-vehicle “learning” of an authorized driver's preferred vehicle system settings is provided by continuously monitoring the driver's vehicle system settings over time and over a range of driving conditions, and then statistically modeling sensitive vehicle settings as described below to generate the HDP for that particular driver. Along with the modeled settings, the HDP can also include discrete vehicle settings, such as relatively consistent settings, on/off settings, etc. An authorized driver is then affirmatively recognized using the currently selected VSS, i.e., those settings that the driver chooses or selects upon entering the vehicle, with the HDP being updated using the currently selected VSS and any modifications thereto. Over time, such as during a number of future trips taken by the same authorized driver over different driving conditions, additional information regarding the VSS can be correlated to the HDP for that driver to further optimize the accuracy of the HDP. Once the driver is identified, various autonomous or automatic control actions can be taken, such as automatically adjusting or customizing certain other vehicle system settings using the HDP for that driver.
In particular, a vehicle includes a plurality of vehicle systems each having a set of driver-selectable or driver-adjustable vehicle system settings (VSS), and a control system operable for determining an identity of one of a plurality of authorized drivers of the vehicle using the VSS. The control system automatically executes a vehicle control action, such as automatically updating one or more VSS during the course of a trip or over several trips, using the identity of the driver. The control system can statistically model a predetermined set of the most sensitive of the VSS for each authorized driver over time to thereby produce the HDP for each driver. The predetermined set of the most sensitive of the VSS can include without being limited to: seat position, mirror position, pedal position, steering wheel position, suspension settings, climate control settings, etc. The HDP can be further optimized by including a set of discrete VSS in the HDP, such as radio or other entertainment system settings, seat warmer on/off status, moon roof open/closed status, etc., and the mean and variance of such VSS where appropriate, as described below.
The control system has a driver recognition algorithm which includes each of a feature extraction subprocess, a feature selection subprocess, and a feature classification subprocess. In one exemplary embodiment, the feature extraction subprocess is a Linear Discriminant Analysis (LDA) subprocess, and the feature classification process is a Gaussian Mixture Model (GMM) subprocess, although other subprocesses capable of uniquely identifying the driver by comparing a set of VSS to a modeled HDP for that driver are also usable within the scope of the invention.
A method for automatically controlling a vehicle system includes collecting the set of driver-selectable VSS, processing predetermined sensitive settings of the VSS through a statistical modeling algorithm to determine an identity of a driver of the vehicle, and executing a vehicle control action corresponding to that identity. Collecting the set of VSS can detect a driver-selectable or driver-adjustable VSS of one or more vehicle systems, with the term “selectable” referring to such discrete settings as radio stations and “adjustable” referring to variable setting such as mirror positions. VSS can include by way of example: mirrors, seats, pedals, steering wheel, radio, HVAC systems, etc., with a predetermined set of the more sensitive of the settings used in the statistical model. Processing the set of VSS includes consolidating the set of VSS to form an original feature vector collectively describing the VSS, transforming the original feature vector using a feature extraction subprocess to thereby generate a new feature vector, and processing the new feature vector through a feature selection subprocess to thereby generate a final feature vector. The final feature vector can be processed through a classification subprocess to thereby determine the identity of the driver.
The above features and advantages, and other features and advantages of the present invention are readily apparent from the following detailed description of the best modes for carrying out the invention when taken in connection with the accompanying drawings.
With reference to the Figures, wherein like reference numerals refer to like or similar components throughout the several figures, and beginning with
The vehicle 10 includes various systems or devices, each of which is at least partially adjustable or repositionable by an authorized driver 12 of the vehicle 10 in order to provide a driving experience that is uniquely tailored to that particular driver. For example, the vehicle 10 can include adjustable side mirrors 26S, a rear-view mirror 26R, an input panel or human-vehicle interface (HVI) 50, control pedals 17, the steering wheel 20, etc. For dynamic control of the vehicle 10, the pedals 17 can include a throttle or accelerator pedal and a brake pedal, and could optionally include a clutch pedal when the vehicle 10 is configured with a manual transmission. Although not shown in
The HVI 50 itself can be adapted to house or include various control switches, knobs, buttons, touch-screen interfaces, voice-recognition interfaces, or other suitably configured input devices allowing the manual selection of preferred settings for each of the various vehicle systems. In addition to the vehicle systems listed above, additional exemplary vehicle systems can include, without being limited to, heating, ventilation, and air conditioning (HVAC) controls, radio station and/or volume controls, compact disc (CD)/digital video disc (DVD)/MP3 controls, interior/exterior lighting controls, four-wheel/two-wheel drive mode setting controls, etc. For simplicity, the HVI 50 is shown in
The vehicle 10 also includes an automatic driver recognition and control system (DRCS) 30 that is adapted to identify or recognize an authorized driver 12 of the vehicle 10 based on a set of vehicle system settings or VSS as described below with reference to
Referring to
After the BCM 34 collects a set of local signals 35 from each LCM 32, the BCM 34 generates a collective set of vehicle system setting or VSS information 52. The VSS information 52 is relayed or transmitted to a setting-based driver identification module (IDSM) 54 of the controller 53. In addition to the VSS information 52, the controller 53 also received the remote signals 22 from the remote device 13 of
The controller 53 recognizes the identity of the driver 12 of
The controller 53 can be configured as a general purpose digital computer generally comprising a microprocessor or central processing unit, read only memory (ROM), random access memory (RAM), electrically-programmable read only memory (EPROM), high speed clock, analog to digital (A/D) and digital to analog (D/A) circuitry, and input/output circuitry and devices (I/O), as well as appropriate signal conditioning and buffer circuitry. Each set of algorithms resident in the controller 53 or accessible thereby, such as the algorithm 100 of
Within the scope of the invention, if the optional BIDM 38 shown in phantom is included within the DRCS 30, such a device or devices can use the biometric sensors 36 (also shown in phantom) to gather a set of unique biometric characteristics of a driver 12, such as the driver's fingerprints, finger veins, iris patterns, retinal patterns, handprints, voice recognition, facial recognition, speech recognition, etc., and relay this information as the biometric signals 37. The optional BIDM 38 can further optimize the performance of the DRCS 30 as noted above. Whether or not a BIDM 38 is used, the DRCS 30 first performs a vehicle setting-based driver recognition function using the collective set of VSS information, i.e., the local signals 35, and then performs a decision fusion function within the DFM 56 that ultimately transforms or processes the initial driver recognition results in a particular manner, as will now be set forth in detail with reference to
Referring to
At step or logic block 106 a set of final features (arrow 74) is determined, with logic block 106 selecting an optimal subset of the original features (arrow 70) to further reduce a dimension of the final features (arrow 74). The final features (arrow 74) are then input to a classifier (CL) at step or logic block 108. The classifier (CL) determines the identity of a driver such as driver 12 of
Still referring to
At step or logic block 104, i.e., the feature extraction (FE) step or logic block, the algorithm 100 conducts a transformation function on the original feature vector oi (arrow 70) output from the step or logic block 102 to thereby generate a new feature vector qi=f(oi) as the transformed or new features (arrow 72). Various feature extraction techniques or methods can be used within the scope of the invention, e.g., Principle Component Analysis (PCA), Linear Discriminant Analysis (LDA), Kernel PCA, Generalized Discriminant Analysis (GDA), etc. For exemplary purposes, LDA can be used to show a linear transformation: qi=UToi, where oi is an no-by-1 vector, U is an no-by-nq _l matrix and qi is an nq-by-1 (nq≦no) vector with each row representing the value of the new features. The matrix U is determined off-line during a design phase, which will be described later hereinbelow.
At step or logic block 106, i.e., the feature selection (FS) step or logic block, the transformed or new features (arrow 72) are further processed to select an optimal subset of the new features, i.e., the final features (arrow 74). Various feature selection techniques can be used within the scope of the invention, e.g., Exhaustive Search, Branch-and-Bound Search, Sequential Forward/Backward Selection, and Sequential Forward/Backward Floating Search, can be used within the scope of the invention. The subset that yields the best or optimal performance is chosen as the final features (arrow 74) to be used for final driver classification.
For example, the resulting subset describing the final features (arrow 74) may consist of n features corresponding to the {l1 l2 . . . ln}(1≦l1≦l2≦ . . . ≦ln≦nq) row of the feature vector qi. The matrix U can be written or described as U=└u1 u2 . . . unq┘, with each vector being an no-by-1 vector. The algorithm 100 selects only those vectors corresponding to the best or optimal subset, and therefore W=[ul1 u12 . . . uln], an no-by-n matrix. Combining the feature extraction and feature selection, the final features (arrow 74) corresponding to the original feature vector oi can be derived as xi=WToi. Within the scope of the invention, since the dimension of the extracted features (i.e., nq) is relatively small, Exhaustive Search is used in one embodiment to evaluate the classification performance of each possible combination of the extracted features, which will be explained in detail hereinbelow.
At step or logic block 108, i.e., the classification (CL) step or logic block, the final features (arrow 74) are classified or compared to a population of modeled HDP to determine the identity of the driver 12 of
Additionally, typical pattern recognition problems usually have training patterns for the classifier design, and the classifier itself is fixed once the design process is completed. For in-vehicle driver recognition, the classifier includes a “learning” capability to provide the ability to update itself with the new patterns, i.e., new sets of vehicle settings or the VSS information (arrow 52). That is, the classifier (CL) of
The present invention addresses the unique requirements of an in-vehicle driver recognition problem by employing a design based on Gaussian Mixture Models. The term “mixture model” as used herein refers to a model in which independent variables are fractions of a total value. Such a mixture model can be suitable for situations where an observation belongs to one of a number of different sources or categories, but when a source or category to which the observation belongs cannot be measured. In this form of mixture, each of the sources is described by a component probability density function, and its mixture weight is the probability that an observation comes from this component.
A GMM in particular is a specific type of mixture model where all the component probability density functions are Gaussian. Once the number of component models and the corresponding parameters for each component model are known, the source or category, i.e., the class as represented by the component distribution, that a specific observation belongs to can be identified. Since a vehicle is likely to have more than one driver and the vehicle settings of each individual driver are approximately of joint Gaussian distribution, GMMs are suitable for representing the density distribution of the VSS information (arrow 52) of the vehicle 10 shown in
Therefore, within the scope of the invention GMMs can be used to estimate the density distribution of the VSS information (arrow 52) describing the various vehicle settings, and to identify the current driver based on his/her settings. The GMM-based driver recognition starts when a driver, such as the driver 12 of
On the other hand, if N>0, the DRCS 30 detects whether there is setting adjustment within a certain period of time after the driver 12 enters the vehicle 10. If the driver 12 adjusts the vehicle settings, the algorithm 100 can pause or wait until the adjustment has been completed, e.g., until the vehicle settings have not been changed for T seconds. The algorithm can then conduct feature extraction (FE) and feature selection (FS) using the new setting measurements oi or original features (arrow 70) to generate a new feature setting vector xi=WToi as the new features (arrow 72). The algorithm 100 then determines the identity of the driver 12 by classifying it into the (N+2 ) classes based on the current GMM with the parameters pki−1, μji−1, and Σk i−1,
If P(c|xi)>Pth for any 1≦k≦N, where Pth is a pre-determined threshold, the driver has been identified as an existing driver (driver k). The algorithm 100 adds the new feature vector xi (arrow 72) into a data sample set, and updates the GMM model accordingly. The update of the GMM model can be carried out in various ways. For example, equivalent mixing probability pc=1/N can be assumed and the mixing probability gets updated only when a new driver appears. For each driver j, the algorithm 100 stores the most recent Nj (e.g., Nj≦10) feature sets: Xj. As the new feature vector xi (arrow 72) belongs to driver k, only the parameter associated with driver k needs to be updated.
Combining the new feature vector (arrow 72) with the existing feature vectors of driver k results in {tilde over (X)}c={Xc, xi}, μci is updated as the mean of {tilde over (X)}c and Σi c as the variance of {tilde over (X)}c. After the update, the oldest feature set in {tilde over (X)}c is removed if necessary so as to limit the number of feature vectors in Xc. The parameters associated with other drivers remain the same: μij=μi−1j and Σij=Σi−1j for j≠c (1≦j≦N).
If P(c|xi)≦Pth, the driver 12 of
In accordance with the invention, the process of frequent driver recognition is optimized via a low-cost, relatively precise apparatus and method as set forth above. The identity of a driver such as driver 12 of
The solution provided herein is relatively non-intrusive, as unlike various biometric scanning and user profile-based selections, the driver 12 is not required to take any additional affirmative steps that the driver 12 would not ordinarily take upon entering the vehicle 10. That is, certain predetermined VSS are disproportionately descriptive or sensitive relative to other VSS. These predetermined VSS can be used to model the driver's HDP over time, with the HDP modified as needed by certain other VSS that are more discrete and less variable, such as on/off settings, open/closed settings, discrete position settings, etc.
Over time, the DRCS 30 adapts itself to the driver 12 and various vehicle driving conditions, thus facilitating automatic customization or adjustment of vehicle system settings. For example, once the driver's identity has been established using the vehicle settings or VSS information (arrow 55) as described above, that is, after comparing the driver's most recently entered VSS to various HDP and selecting that driver's HDP, certain control actions can be automatically and seamlessly executed in accordance with that drivers HDP to thereby customize the overall driving experience. Exemplary control actions can include, without being limited to, automatically adjusting or repositioning the mirrors 26S, 26R, the driver seat 24D, the pedals 17, the steering wheel 20, etc. Likewise, the settings for the radio 29R and/or the HVAC 29E of
While the best modes for carrying out the invention have been described in detail, those familiar with the art to which this invention relates will recognize various alternative designs and embodiments for practicing the invention within the scope of the appended claims.