Patent Application
20060149426
2. Description of the Related Art
Detection of the position of a vehicle occupant is very useful in various industries. One industry that uses such information is the automotive industry where the position of a vehicle occupant is detected with respect to an airbag deployment region to prevent an injury occurring when an airbag deploys due to an automobile crash or other incident. Generally, current solutions rely on a combination of sensors including seat sensors, which detect the pressure or weight of an occupant to determine whether the seat is occupied. However, because this system does not provide a distinction between tall and short occupants, for example, or occupants who are out of position during a collision, an injury may still result from the explosive impact of the airbag into out-of-position occupants. Further, the airbag may be erroneously deployed upon sudden deceleration when using the weight sensors to detect the position of the vehicle occupant.
Other solutions provide capacitive sensors in the roof of a vehicle to determine a position of the vehicle occupant. However, similar to the weight or pressure sensors, the capacitive sensors do not provide accurate positioning information of small occupants, such as children. The capacitive sensors also require a large area in the roof of the vehicle for implementation and are not easily capable of being implemented in existing vehicles.
Various embodiments of the present invention provide a method including (a) detecting a location of an eye of a user using an automated detection process, and (b) automatically determining a position of a head of the user with respect to an object based on the detected location of the eye.
Various embodiments of the present invention provide a method including (a) detecting a location of an eye of a user using an automated detection process, and (b) automatically determining at least one of height and orientation information of the user with respect to an object based on the detected location of the eye.
Moreover, various embodiments of the present invention provide a method including (a) detecting a location of an eye of a user inside a vehicle using an infrared reflectivity of the eye or a differential angle illumination of the eye, (b) automatically determining at least one of height and orientation information of the user based on the detected location of the eye, and (c) controlling a mechanical device inside the vehicle in accordance with the determined information of the user.
Various embodiments of the present invention provide a method including (a) detecting a location of an eye of a user inside a vehicle using an infrared reflectivity of the eye or a differential angle illumination of the eye, (b) automatically determining a position of a head of the user based on the detected location of the eye, and (c) controlling a mechanical device inside the vehicle in accordance with the determined position of the head.
Various embodiments of the present invention provide a method including (a) detecting a location of an eye of a user using an automated detection process, (b) determining a position of the user based on the detected location of the eye, and (c) automatically implementing a pre-crash and/or a post-crash action in accordance with the determined position.
Various embodiments of the present invention further provide a method including (a) detecting an eye blinking pattern of a user using an infrared reflectivity of an eye of the user, and (b) transmitting messages from the user in accordance with the detected eye blinking pattern of the user.
These and/or other aspects and advantages of the invention will become apparent and more readily appreciated from the following description of the embodiments, taken in conjunction with the accompanying drawings of which:
Reference will now be made in detail to the present embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The embodiments are described below to explain the present invention by referring to the figures.
The term “automated” indicates that the detection process is performed in an automated manner by a machine, as opposed to detection by humans. The machine might include, for example, a computer processor and sensors. Similarly, various processes may be described herein as being performed “automatically”, thereby indicating that the processes are performed in an automated manner by a machine, as opposed to performance by humans.
The automated detection process to detect the location of an eye(s) could be, for example, a differential angle illumination process such as that disclosed in U.S. application Ser. No. 10/377,687, U.S. patent Publication No. 20040170304, entitled “APPARATUS AND METHOD FOR DETECTING PUPILS”, filed on Feb. 28, 2003, by inventors Richard E. Haven, David J. Anvar, Julie E. Fouquet and John S. Wenstrand, attorney docket number 10030010-1, which is incorporated herein by reference. In this differential angle illumination process, generally, the locations of eyes are detected by detecting pupils based on a difference between reflected lights of different angles of illumination. More specifically, lights are emitted at different angles and the pupils are detected using the difference between reflected lights as a result of the different angles of illumination. Moreover, in this process, two images of an eye that are separated in time or by wavelength of light may be captured and differentiated by a sensor(s) to detect a location of the eye based on a difference resulting between the two images.
Alternatively, the automated detection process to detect the location of an eye(s) could be, for example, a process such as that disclosed in U.S. application Ser. No. 10/843,517, entitled “METHOD AND SYSTEM FOR WAVELENGTH-DEPENDENT IMAGING AND DETECTION USING A HYBRID FILTER”, filed on May 10, 2004, by inventors Julie E. Fouquet, Richard E. Haven, and Scott W. Corzine, attorney docket number 10040052-1, and U.S. application Ser. No. 10/739,831, entitled “METHOD AND SYSTEM FOR WAVELENGTH-DEPENDENT IMAGING AND DETECTION USING A HYBRID FILTER”, filed on Dec. 18, 2003, attorney docket number 10031131-1, which are incorporated herein by reference. In this process, generally, at least two images of a face and/or eyes of a subject are taken, where one image is taken, for example, at or on an axis of a detector and the other images is taken, for example, at a larger angle away from the axis of the detector. Accordingly, when eyes of the subject are open, the difference between the two images highlights the pupils of the eyes, which can be used to infer that the subject's eyes are closed when the pupils are not detectable in the differential image.
Further, as described in the above-referenced U.S. applications titled “METHOD AND SYSTEM FOR WAVELENGTH-DEPENDENT IMAGING AND DETECTION USING A HYBRID FILTER”, a wavelength-dependent illumination process can be implemented in which, generally, a hybrid filter having filter layers is provided for passing a light at or near a first wavelength and at or near a second wavelength while blocking all other wavelengths for detecting amounts of light received at or near the first and second wavelengths. Accordingly, generally, a wavelength-dependent imaging process is implemented to detect whether the subject's eyes are closed or open.
The general descriptions herein of the above-described automated detection processes are only intended as general descriptions. The present invention is not limited to the general descriptions of these automated detection processes. Moreover, the above-referenced automated detection processes are only intended as examples of automated detection processes to detect the location of an eye(s). The present invention is not limited to any particular process.
Referring to
For example, in operation 12 of
More specifically, as an example, with stereo eye detection systems, each eye detection system would provide eye location information in operation 10. Then, in operation 12, a triangulation method would be used between the eye detection systems to provide more detailed three-dimensional head position information. In addition, the triangulation method could be implemented in operation 12 to provide, for example, gaze angle information.
To improve accuracy, timing of imaging between the stereo eye detection systems could be well controlled. There are a number of manners to accomplish such control. For example, such control can be accomplished by using a buffer memory in each eye detection system to temporarily store images taken simultaneously by the eye detection systems. The memory of a respective eye detection system might be, for example, a separate memory storage block downloaded, for example, from a pixel sensor array of the respective eye detection system. Alternatively, image data may be temporarily stored, for example, in the pixel array itself. The images from the different eye detection systems could then, for example, be sequentially processed to extract eye location information from each image.
As another example of the use of stereo eye detection systems, the cost of a buffer memory or pixel complexity may be reduced, for example, by eliminating the memory component. For example, eye detection systems could include, for example, CMOS image sensors which are continuously recording sequential images. The readout of each image sensor can then be scanned on a line-by-line basis. Effectively, simultaneous images may be extracted by reading a line from a first sensor and then reading the same line from a second sensor. The readout from the two images can then be interleaved. Subsequent lines could be alternatively read out from alternating image sensors. Information on the eye location can then be extracted from each of the composite images made up of the alternate lines of the image data as it is read, to thereby provide information indicating a three-dimensional position of the head.
The above-described examples of the operation of stereo eye detection systems are only intended as examples. The present invention is not limited to any particular manner of operating stereo eye detection systems.
Instead of using a triangulation method, in operation 12, an algorithm can be used to determine the position of a head of the user with respect to an object based on the detected location of at least one eye in operation 10. An example of an algorithm might be, for example, to estimate a boundary of a head by incorporating average distances of facial structures from a detected location of an eye. Since the location of the object is known, the position of the head with respect to the object can be determined from the estimated boundary of the head. Of course, this is only an example of an algorithm, and the present invention is not limited to any particular algorithm.
Further, as an additional example, in operation 12 of
Therefore, in operation 12 of
In various embodiments of the present invention, as will be discussed in more detail further below, a mechanical device of the vehicle can be appropriately controlled, or appropriate corrective action can be taken, in accordance with the determined position of the head of a user, or simply in accordance with a determined position of the user.
For example, the object in the vehicle might be a dashboard, so that the position of the head with respect to the dashboard is determined. Then, a mechanical device of the vehicle can be controlled based on the determined position. For example, in various embodiments of the present invention, appropriate control can be automatically performed to adjust a seat or a mirror (such as, for example, a rear view mirror or a side view mirror). Of course, the present invention is not limited to the object being the dashboard, or to the controlled mechanical device being a seat or a mirror.
Alternatively, in various embodiments of the present invention, appropriate control can be automatically performed to implement a pre-crash corrective action. Such pre-crash corrective action could include, for example, activating a seat belt, performing appropriate braking action, performing appropriate speed control, performing appropriate vehicle stability control, etc. These are only intended as examples of pre-crash corrective action, and the present invention is not limited to these examples.
In addition, in various embodiments of the present invention, appropriate control can be automatically performed to implement a post-crash corrective action. Such post-crash corrective action could include, for example, automatically telephoning for assistance, automatically shutting off the engine, etc. These are only intended as examples of post-crash crash corrective actions, and the present invention is not limited to these examples.
Therefore, it should be understood that “pre-crash” corrective actions are actions that are taken before the impending occurrence of an expected event, such as a crash. “Post-crash” corrective actions are actions that are taken after the occurrence of the expected event, such as a crash. However, it should be understood that an expected event might not actually occur. For example, pre-crash actions might be automatically implemented which prevent the crash from actually occurring.
While determining a position of a head of the user with respect to an object is described in relation to a user inside a vehicle, the present invention is not limited to determining a position of a head of the user in a vehicle. For example, the present invention can be implemented to detect the location of an eye of the user with respect to the vehicle itself for keyless entry into the vehicle.
Accordingly, in process 100, a location of an eye of a user is detected using an automated detection process and a position of a head of the user with respect to an object is determined based on the detected location of the eye. The determined position of the head enables use of the determined position of the head in various applications.
From operation 14, the process 200 moves to operation 16, where at least height and orientation information of the user is determined with respect to an object based on the detected location of the eye. For example, assuming a user is seated upright in a car seat, the position of the eye in a vertical dimension corresponds directly to the height of the user. However, when the user is near the object, the height calculated from the location of the eye(s) in a vertical dimension could be misleading. Thus, in an embodiment of the present invention, in a case where the user is too near to the object, an interocular distance between the eyes of the user, which corresponds to the distance to the user, can be correlated to a certain distance where a wider interocular distance generally corresponds to the user being close and a relatively narrow interocular distance indicates vice versa.
Further, when the user's head is rotated right or left, the interocular distance between the eyes may indicate a closer eye spacing with respect to the object. Accordingly, additional characterization may be implemented to determine head rotation, according to an embodiment of the present invention. For example, feature extraction of a nose of the user relative to the eyes can be used to distinguish between closer eye spacing due to head rotation and due to decreasing distance between the head and the object.
As an additional example, sensors may be provided to detect the location of the eyes of the user and the height and orientation information can be determined using a triangulation method in accordance with detection results of the sensors.
However, the present invention is not limited to any specific manner of determining height and orientation information of a user.
Further,
Referring to
As also shown in
Accordingly, the present invention provides an accurate method and apparatus for eye and position detection of a user.
Further, in various embodiments of the present invention, the position of the head of the user is determined in a three-dimensional space. For example, as shown in
While
The determination of a position of a user based on detected location(s) of an eye(s) of the user enables various applications of the position information of the user. For example, various mechanical devices, such as seats, mirrors and airbags can be adjusted in accordance with the determined position of the user.
Moreover, for example, pre-crash corrective actions can be automatically performed to implement the pre-crash corrective actions based on the determined position of a user. Such pre-crash corrective action could include, for example, activating a seat belt, performing appropriate braking action, performing appropriate speed control, performing appropriate vehicle stability control, etc. These are only intended as examples of pre-crash corrective action, and the present invention is not limited to these examples.
In addition, in various embodiments of the present invention, post-crash corrective actions can be automatically performed to implement the post-crash corrective actions based on the determined position of a user. Such post-crash corrective action could include, for example, automatically telephoning for assistance, automatically shutting off the engine, etc. These are only intended as examples of post-crash crash corrective action, and the present invention is not limited to these examples.
Referring to
From operation 26, process 400 of
From operation 20, the process 600 moves to operation 22, where messages are transmitted from a user in accordance with the detected blinking pattern. For example, eye blinking pattern of a disabled person is automatically detected and the detected eye blinking pattern is decoded into letters and/or words of the English alphabet to transmit messages from the disabled person using the eye blinking pattern. Further, a frequency of the eye blinking pattern is used for transmitting messages from the user, according to an aspect of the present invention.
Referring to
Therefore, the present invention also enables use of eye blinking pattern for communication purposes by detecting eye blinking pattern from multiple directions.
While various aspects of the present invention have been described using detection of eyes of a user, the present invention is not limited to detection of both eyes of the user. For example, an eye of a user can be detected and a position of a head of the user can be estimated using the detected eye of the user.
Although a few embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes may be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.
