Patent Application
20060092401
This invention relates generally to automotive optical sensing systems, and, more particularly, to a novel automotive optical sensing system that uses active illumination.
While the present invention is described, for illustrative purposes, as being applied to occupant classification and position detection in a vehicle, it will be understood that it may be employed in any automotive optical sensing system that benefits from active illumination and that must function in a wide variety of ambient lighting conditions. What is meant herein by “active illumination” is that the optical sensing system provides its own source of light. Examples of such automotive optical sensing systems include active night vision systems and lane tracking systems, for example.
Inflatable restraints, or air bags, are increasingly being used in vehicles to protect an occupant of a vehicle in the event of the vehicle being involved in a collision. Typically, a control system senses the occurrence of an accident and causes the air bag to deploy to protect the occupant from colliding with the windshield or dashboard, for example, of the vehicle. An optical, actively-illuminating, occupant proximity sensor system may comprise part of the control system. Such a system monitors the position of the occupant of the front-outboard passenger seat and automatically, i.e., without any action required of the driver or the passenger, determines whether to suppress deployment or allow deployment of the front air bag (and perhaps the side air bag also, if the vehicle is so equipped) applicable to this position. The determination of whether or not to suppress air bag deployment is made based on the occupant being in the “at-risk” zone or on classifying the occupant as being in a rear-facing infant seat (RFIS). Conversely, the system allows air bag deployment for a properly seated occupant, i.e., seated with hips back, rearward on the seat cushion and with back resting on the seat back, who is outside of the at-risk zone.
Upon determining the position/classification of the occupant of the front-outboard seat, the system sends a signal to the sensing and diagnostic module indicating whether to suppress or allow air bag deployment. The system also sends a signal to a display indicating whether the air bag is in a suppressed or an enabled state. If the air bag is in a suppressed state, the signal alerts the driver or passenger to take corrective action, if desirable.
There are many kinds of sensor systems under development. So far, there are mainly two kinds of occupant position sensors: infrared- and ultrasonic-based sensors. Compared to the ultrasonic sensor, the infrared sensor has the faster response time. An infrared occupant position sensor can classify and recognize an occupant in less than 10 milliseconds.
The typical infrared occupant position sensor has two main elements: an emitter and a receiver. The emitter is a light source and optical system that radiates infrared or near-infrared light and illuminates target objects. The receiver consists of an optical system and a detector. The optical system collects light scattered by the target objects and focuses the light signal on the detector.
It is desirable, for safety reasons as well as for aesthetics, for the light emitted and scattered in the optical system to not be visible to humans. In order to be substantially invisible, the light emitter needs to operate in a portion of the electromagnetic spectrum where the wavelength is greater than approximately 700 nm. The wavelength of the light also preferably needs to be less than approximately 1100 nm in order to be detected by image sensors having a reasonable cost, e.g., silicon-based image sensors. Thus, the useful range of wavelengths for actively-illuminated imager systems is approximately between 700 nm and 1100 nm.
With presently available technology, the most desirable wavelengths for operation of such systems are between 700 nm and 1100 nm. However, improvements in the performance of light emitting and light sensing technology at longer wavelengths will allow the viable extension of these concepts to longer wavelengths. Specifically, wavelengths up to 3000 nm will be shown to be useful in the present context, based upon the development of economically viable hardware.
Since an automotive optical system must operate outdoors, it is exposed to widely varying levels of ambient illumination. The performance of the optical sensor tends to be limited by these variations in the level of ambient light. From bright sunlight to twilight, the level of ambient lighting varies by a factor of 10,000, i.e., 80 dB. Besides sunlight, which is the dominant source of ambient light, other sources of ambient light including street lights, oncoming headlamps and tail lamps of other vehicles, and interior light sources within the vehicle itself may contribute to the light being sensed by a specific vehicle sensing system. All of these sources, referred to herein as ambient light sources, may adversely affect the ability of an actively (self) illuminated optical sensing system to accurately and reproducibly perform its functions.
In order to minimize the effects of interference from ambient light, known optical systems endeavour to maximize the number of photons from the emitter that are received by the detector. Further, in order to maximize the reception of photons, known optical systems operate at wavelengths at which atmospheric absorption is minimal. Thus, in keeping with the goal of maximum photon reception, known optical systems do not include any filtering of the light between the system's emitter and detector. However, with no filtering, the light from the emitter may often be overwhelmed by high levels of ambient light. Consequently, the optical system may not be able to discriminate between light emitted by the system and light originating from ambient sources. The consequences of this upon the performance of the overall sensing are serious. In known sensing systems, as ambient lighting conditions vary, information detected by the optical system may vary, even if there is no change in the actual physical state of the features of interest that are being sensed. Consequently, known optical sensing systems may respond in a non-optimum manner to variations in ambient lighting, reducing the utility of such systems.
What is needed in the art is an optical occupant proximity sensor system in which light from the emitter may be reliably sensed by the detector in a wide variety of ambient light conditions.
The present invention provides an embodiment of an occupant proximity sensor system including an optical system that limits the wavelength of the light from the emitter to a range of wavelengths that substantially matches a region of the electromagnetic spectrum in which extraneous light sources are weakest, i.e., a region of the electromagnetic spectrum that represents a minimum in the solar spectral irradiance at the surface of the earth. Alternatively, the emitter may emit its full bandwidth of light, while the detector limits the light that it receives from the emitter to the region of the electromagnetic spectrum in which extraneous light sources are weakest. In a further embodiment, the emitter is repetitively flashed on and off so that the baseline ambient light received by the detector may be measured and distinguished from the light that originates from the emitter.
The present invention further provides an embodiment of an occupant or driver eye gaze monitoring system that is substantially impervious to variations in ambient lighting.
The present invention considers three aspects of systems that photometrically sense information based on self-contained photon sources. Three specific concepts are embraced: (1) matching the wavelengths emitted by the photon sources to the wavelengths accepted for detection by the sensor element or array, (2) modulating the photon sources in some manner so as to allow for synchronous detection of photons from the scene being illuminated by the photons, and (3) taking advantage of characteristics of the medium involved to minimize the potential detection of photons originating from other uncontrolled photon sources. A specific implementation relates to the design of imaging systems for use in varying naturally illuminated conditions, wherein the selection of operational wavelengths is made such as to coincide with atmospheric absorption peaks (ranges of wavelengths in which ambient lighting is relatively low). Conventional practice, in contrast, suggests operating at wavelengths where the intensity of ambient lighting is as high as possible (i.e. atmospheric absorption is minimal).
Specific embodiments of the present invention include bandpass filters for limiting the sensitivity of the image sensor to a subset of the wavelength range of 700 nm to 100 nm. This subset of wavelength ranges may be chosen to match the wavelengths of specific image sources, or to compensate for (provide attenuation of) competing non-controllable light sources. For example, consider light emitting diodes; an 880 nm LED exhibits a full-width-half-maximum range in emitted wavelengths of approximately 50 nm. Thus, in an imager system that is being actively illuminated by such a light source, a bandpass filter would be applied to the image sensor or image sensor array having bandpass characteristics substantially equal to the characteristics of the light source, in this example, from approximately 850 nm through approximately 900 nm. By limiting the detector's response to this narrow range of wavelengths, we exclude a large proportion of the external light sources that may interfere with the sensor system and that may vary by a factor of 80 dB or more. Consequently, a majority of the light reaching the detector of the sensor/imager system will be light that originated in the active illumination portion of the system, and the total amount of light incident upon the image sensor will vary by substantially less than 80 dB, even though ambient light is varying by 80 dB. This assumes that the output of the active light source is of significant intensity in relation to the intensity of the external light sources in this wavelength range. Ideally, if the active illumination source dominates the external sources in this wavelength range, then there will be relatively little variation in light intensity at the detector, even in the presence of drastic variations in the intensity of the external sources.
According to one embodiment of the invention, an automotive occupant position restraint apparatus senses a position of an occupant of a motor vehicle. An optical energy source emits optical energy that is substantially limited to a first range of wavelengths that corresponds to an atmospheric absorption peak. An optical energy receiver receives the optical energy emitted by the optical energy source after the optical energy has been reflected within a passenger compartment of the motor vehicle. A range of wavelengths that the receiver is operable to receive is substantially limited to a second range of wavelengths that corresponds to the atmospheric absorption peak.
According to another embodiment of the present invention, an automotive optical sensing system includes an optical energy source emitting optical energy toward an object. An optical energy receiver receives the optical energy emitted by the optical energy source after the optical energy has been reflected by the object. A range of wavelengths that the receiver is operable to receive is substantially limited to a first range of wavelengths that corresponds to an atmospheric absorption peak.
According to yet another embodiment of the present invention, an automotive optical sensing system includes an optical energy source emitting optical energy toward an object. An optical energy receiver receives the optical energy emitted by the optical energy source after the optical energy has been reflected by the object. Two distinct ranges of wavelengths that the receiver is operable to receive include a first range of wavelengths that corresponds to an atmospheric absorption peak, and a second range of wavelengths that are close to the wavelengths of the first range of wavelengths, but are representative of the solar spectrum not in the presence of an absorption peak. The differential detection of these distinct sets of wavelengths providing additional ability to discriminate against the influence of ambient light.
An advantage of the present invention is that the interference of ambient light with the detection or sensing of light originating from the emitter is reduced.
Another advantage is that inexpensive silicon-based light detectors may be used.
Yet another advantage is that the baseline ambient light received by the detector may be measured and distinguished from the light that originates from the emitter.
It is understood that although the specific embodiments presented in this document refer to occupants of automobiles, identical systems may be of advantage in other vehicular systems, including trucks, busses and other forms of transportation in which it is desirable to identify the state of occupants.
Similarly, advantageous use may be made of these concepts for other optical emitter/detector systems that may operate in the presence of large variations in ambient illumination.
The present invention will now be described, by way of example, with reference to the accompanying drawings, in which:
Corresponding reference characters indicate corresponding parts throughout the several views. Although the drawings represent an embodiment of the present invention, the drawings are not necessarily to scale and certain features may be exaggerated in order to better illustrate and explain the present invention. The exemplifications set out herein illustrate an embodiment of the invention and such exemplifications are not to be construed as limiting the scope of the invention in any manner.
Referring now to the drawings, and particularly to
Mounted in a console 24 behind a rear view mirror 26 in vehicle 14 is an optical occupant proximity sensor system 28 (
Detector module 32 receives optical energy in the form of IR or NIR radiation or light after it has been scattered or otherwise reflected by objects in the front of the passenger compartment of the vehicle. Detector module 32 may also receive diffused light, i.e., light that has not been reflected by objects in the passenger compartment. Ideally, detector module would receive only light originating from emitter module 30. In practice, however, detector module 32 also receives light from extraneous sources such as the sun. After detector module 32 receives the light, the system may recognize occupant type, as well as occupant position and posture, with dynamic occupant position being determined using triangulation, as is well known in the art.
Modules 30 and 32 are electrically connected to control module 34 which classifies person 10 in passenger compartment 42 and determines if the person is out of the at-risk zone and is not seated in a rear facing infant seat. In this case, person 10 is safely seated and, accordingly, control system 34 will enable the air bag (not shown in
The novel emitter module 30 of the present invention will now be described in more detail. Emitter module 30 includes an optical energy source 82, which may be in the form of a light-emitting diode (LED) or laser source. LED 82 transmits light through a filtering element 84 (
In one embodiment, LED 82 emits near infrared light, and filtering element 84 is a bandpass filter. The band of wavelengths passed by a bandpass filter 84 may correspond to and/or overlap a band of wavelengths that is absorbed by water vapor in the earth's atmosphere more than are other nearby and comparable wavelengths. Since earth's atmosphere absorbs a high level of the sun's radiation in such a band of wavelengths, interference of sunlight with the light from LED 82 may be reduced. In contrast, since the light from LED 82 travels only a few meters in its path to detector module 32, a negligible amount of the light from LED 82 is absorbed by the atmosphere. Accordingly, emitter module 30 may include any type of light source that is suitably filtered to limit the emitted optical energy to a range of wavelengths that corresponds to and/or overlaps wavelengths associated with an atmospheric absorption peak and in which extraneous light sources are relatively weak.
The spectral irradiance of the sun at the surface of the earth is shown in
Although several absorption peaks are present in the solar irradiance spectrum, as shown in
Although two distinct absorption peaks lie in the preferred range of wavelengths as described in the preceding paragraph, it will be understood, as a result of the present invention, that a preferable choice of wavelengths will involve the absorption peak at approximately 930 nm, and will not involve the absorption peak at approximately 750 nm. One skilled in the art of optical sensing in the near infrared wavelength portion of the spectrum will understand that the sensitivity of silicon based detectors will generally decrease with increasing wavelength, as shown in
The absorption peaks at longer wavelengths such as approximately 1140 nm, 1400 nm, 1800 nm, and 2800 nm are also of interest in the context described in this invention. Although the infrared sources and detectors that operate at these wavelengths are more expensive than the emitters and silicon based detectors that operate at wavelengths below 1100 nm, there is less solar radiation in these regions, resulting in less interference from ambient lighting, even at lower levels of active system illumination. Consequently, it will be understood that this invention describes actively illuminated optical systems that operate at these wavelengths, just as described in the case of operation at 930 nm. However, it will be understood that given the present development of optical emitters and detectors for all of the wavelengths described in this document, the economical preference may lie with the wavelengths from 700 nm through 1100 nm.
It can be seen in
In order to set the operating point of optical system 28 to correspond with the atmospheric absorption peak at 930 nm, filtering element 84 may be in the form of a bandpass filter having a range or pass band approximately centered at 930 nm. For example, the pass band of bandpass filter 84 may be approximately between 890 nm and 980 nm, as indicated in
Other effects need to be considered as well when determining the optical properties of the detector filter means. This filter allows for the thermal drift of emissive sources such as light emitting diodes and laser diodes, among others. Such a drift tends to be towards increasing wavelengths as temperatures increase typically 0.2 nm/degree Kelvin of temperature rise of the diode substrate) and may total approximately 20 nm. By contrast, the interference filters that typically comprise bandpass filters have a strong performance variation with angle of incidence. Ideally, light will be incident upon such an interference filter at normal incidence (zero degrees), but in a real optical system, significantly larger angles of incidence may be involved. Reasonable numbers may be a decrease in the passband wavelengths of as much as 30 nm for light incident at 30 degrees with respect to the surface normal. A consequence of this is that it is necessary to allow for both of these influences (thermal drift to longer wavelengths, angle of incidence based shift to shorter wavelengths) in order to define the optimum band-pass characteristics for the detector filter described in this work. A reasonable choice of such an optimum filter would be the filter described above having a center wavelength of 960 nm and a bandpass of 75 nm. Although this is wider than the width of the atmosphere absorption peak, it represents a good choice in terms of the effects identified above.
Similarly, detector module 32 may be provided with an optical energy receiver or light detector in the form of a silicon-based light detector or imager array 86. Detector module 32 may also be provided with a filtering element 88 that limits the received optical energy to a range of wavelengths that corresponds to and/or overlaps wavelengths associated with an atmospheric absorption peak and in which extraneous light sources are relatively weak. Although the range of wavelengths passed by filtering element 88 may overlap the range of wavelengths emitted by emitter module 30, it is not necessary for the two ranges to be identical.
Filtering element 88 may be in the form of a bandpass filter having a range or pass band approximately centered at 930 nm. For example, the pass band of bandpass filter 88 may also be approximately between 890 nm and 980 nm and may also have a width of less than approximately 200 nm. However, it is to be understood that the pass band of either filter 84 or filter 88 may be significantly larger or smaller than the 890 nm to 980 nm pass band that is illustrated. For example, it is possible for the pass band of filter 84 and/or filter 88 to include all infrared wavelengths, as indicated by the shaded area that is labelled “infrared pass filter”. It is possible for the light detector 86 to be any type of light detector, such as a single point sensor, or an array of sensors, for example.
Either or both of filters 84, 88 may have a quite narrow pass band, with a width of 20-30 nm or less. With such a narrow pass band, the amount of light that reaches detector module 32 may become quite small. In order to compensate for this decreased amount of light, the detector module 32 may be provided with an increased level of sensitivity. For example, the detection integration time and/or the gain of imager array 86 may be increased to an appropriate level.
In an alternative embodiment, LED 82 is a gallium arsenide (GaAs) LED which has a peak emission intensity and efficiency at a wavelength of approximately 940 nm. A plot of the normalized intensity of a Fairchild GaAs LED as a function of wavelength is included in
In another alternative embodiment, although the emitter module includes a light source that emits light in a wide band of frequencies and wavelengths, the emitter module does not include any type of filtering element. In this embodiment, the filtering element in the detector module is relied upon to substantially limit the detected wavelengths to desired ranges or bands. More specifically, the filtering element in the detector module may be centered around the wavelength of 930 nm in order to match a relative minimum in the spectral irradiance at the surface of the earth.
In yet another alternative embodiment, the light emitter is modulated or switched on and off so as to allow for synchronous detection of reflected or scattered photons from the objects being illuminated. For example, active illumination may be provided during only alternate imager frames, thereby allowing the control module to perform differential sensing of information that results from the active illumination source. That is, the light source is only active during alternate image capture frames. The control module may switch the light source on and off, and analyze light signals received by the detector module while the light source is on. The analysis of these light signals may be performed dependent upon light signals received by the detector module while the optical energy source is off. For example, light signals received while the optical energy source is off represent extraneous light sources. The effects of extraneous light sources may be separated and removed from the light signals that are received while the light source is on. Any image content due to unattenuated (or attenuated) light sources that are not being switched on and off in this pattern may be (further) attenuated, thereby providing additional robustness in the determination of image content that is being illuminated by the active illumination light source. Thus, the influence of extraneous light sources is further suppressed.
Another embodiment relates to a vehicle sensing system that is designed to monitor the degree to which a vehicle occupant's eye or eyes are open. Such a sensing system has also been described as a driver gaze monitor system. As shown in
Emitter module 130 and detector module 132 are shown in
It is a goal of such sensing systems to provide consistent performance regardless of the time of day or night. Consequently, it is desirable to choose optical emitting means and optical detecting means that provide substantially consistent, reproducible detection performance even in the presence of very large changes in ambient lighting, such as the variations in sun light during a twenty-four hour period. In order to limit the susceptibility of the system to variations in ambient lighting, it is generally necessary to, in some manner, overpower the ambient light sources in a wavelength range of interest. It is to be observed from
A specific embodiment of such an optical system for monitoring the degree to which a vehicle occupant's eye or eyes are open may operate in a range of wavelengths including the wavelengths between 930 nm and 940 nm, as shown in
Further embodiments, such as have been suggested with regard to the occupant position sensing application described previously will be seen to apply with equal value to the driver gaze embodiment described in the immediately previous paragraph.
In yet another aspect of this embodiment, it is known that the light emitter may be modulated or switched on and off so as to allow for synchronous detection of reflected or scattered photons from the objects being illuminated. For example, active illumination may be provided during only alternate imager frames, thereby allowing the control module to perform differential sensing of information that results from the active illumination source. That is, the light source is only active during alternate image capture frames. The control module may switch the light source on and off, and analyze light signals received by the detector module while the light source is on. The analysis of these light signals may be performed dependent upon light signals received by the detector module while the optical energy source is off. For example, light signals received while the optical energy source is off represent extraneous light sources. The effects of extraneous light sources may be separated and removed from the light signals that are received while the light source is on. Any image content due to unattenuated (or attenuated) light sources that are not being switched on and off in this pattern may be (further) attenuated, thereby providing additional robustness in the determination of image content that is being illuminated by the active illumination light source. Thus, the influence of extraneous light sources is further suppressed.
It shall be apparent that alternative embodiments of this technique may be applied in other sensing systems. For example, the operation of sensing systems external to the shell of a vehicle may also benefit from these improvements. Such sensing systems that may advantageously benefit from operating at a near infrared wavelength range in which the effects of ambient illumination may be minimized or illuminated include, but are not limited to, lane tracking systems, collision avoidance systems, road obstacle detection systems, pre-crash sensing systems, blind spot detection systems, and backup warning systems. Many of these systems have been implemented using conventional illumination means, but their performance is not completely independent of variations in ambient illumination. Consequently, operating such systems according to the concepts described in the previous embodiments will result in improvements in system reliability and reproducibility of performance. In these cases, the effect of ambient light may be larger, due to the need to transmit and receive light across larger distances. However, these cases also involve the opportunity to use emitting light sources of higher intensity. Higher intensity light sources may comprise larger numbers of LEDs (for example 940 nm GaAs LEDs and/or others). Examples of such light sources may also include incandescent and halogen lamps that incorporate suitable filters to direct a limited spectral distribution of light (as of the wavelengths including the range of wavelengths from 930 nm through 940 nm) in specific directions, for example, in the pattern conventionally known as the “high beam” pattern of vehicle headlamps. Similar filtering may be applied to other broad band illumination sources. Narrow band illumination sources, including forms of lasers such as, but not limited to laser diodes, may also be advantageously applied in such applications.
It is to be understood that the present invention may be advantageously applied in other portions of the spectrum, such as at longer wavelengths in the infrared range, in order to reduce the effect of extraneous and uncontrollable illumination sources. In preferred embodiments, both the active illumination source and the detector are suitably constrained to limit the range of wavelengths that may be emitted or detected, and the operating wavelengths are chosen so as to reduce the influence of extraneous, uncontrollable illumination sources. For example, as can be seen in
The embodiments disclosed above are not intended to be exhaustive or to limit the invention to the precise forms disclosed in the detailed description. Rather, the embodiments have been chosen and described so that others skilled in the art may utilize their teachings.
Although described in the exemplary embodiments, it will be understood that various modifications may be made to the subject matter without departing from the intended and proper scope of the invention. That is, the optical sensing system of the present invention may be used in automotive applications other than an automotive supplemental restraint system, as described in the above embodiments. For example, the optical sensing system of the present invention may be used in active night vision systems, lane tracking systems, systems that monitor the direction of a driver's gaze, and/or systems that monitor the frequency of a driver's blinking in order to detect when the driver is falling asleep. Further, one of skill in the art can appreciate that the optical sensing system of the present invention may also be used in many non-automotive optical sensing applications.
