There are many applications in which it is useful to detect or image an object. Detecting an object determines the absence or presence of the object, while imaging an object results in a representation of the object. The object may be imaged or detected in daylight or in darkness, depending on the application.
Wavelength dependent imaging is one technique for imaging or detecting an object, and typically involves capturing one or more particular wavelengths that reflect off, or transmit through, an object. In some applications, solar or ambient illumination is used to detect or image an object, while in other applications additional illumination is used. Typical wavelength dependent imaging systems for detecting objects cannot obtain color images such as conventional photographs for viewing by a user while simultaneously obtaining machine vision-type images in the infrared for detecting objects, such as retroreflectors, using a single image sensor.
One aspect of the present invention provides an imaging system. The imaging system includes an image sensor, a first light source on-axis with the image sensor, and a controller. The image sensor is configured to generate an image of a field of view including a retroreflector. The first light source is configured to illuminate the retroreflector, and the controller is configured to output the image and location data for the retroreflector.
In the following Detailed Description, reference is made to the accompanying drawings, which form a part hereof, and in which is shown by way of illustration specific embodiments in which the invention may be practiced. In this regard, directional terminology, such as “top,” “bottom,” “front,” “back,” “leading,” “trailing,” etc., is used with reference to the orientation of the Figure(s) being described. Because components of embodiments of the present invention can be positioned in a number of different orientations, the directional terminology is used for purposes of illustration and is in no way limiting. It is to be understood that other embodiments may be utilized and structural or logical changes may be made without departing from the scope of the present invention. The following Detailed Description, therefore, is not to be taken in a limiting sense, and the scope of the present invention is defined by the appended claims.
There are a number of applications in which it is useful to determine whether a person's eyes are open or closed, or the location of a person's eyes. One such application is the detection of drowsiness in the operator of a motor vehicle. Another application includes locating the eyes to use as a reference point for facial recognition applications. Applications involving retroreflectors other than a human eye are also numerous, such as security applications and tracking applications.
Imaging system 100 is configured to generate RGB images of the field of view of detector 102 and locate or track retroreflectors within the field of view of detector 102. Imaging system 100 outputs RGB images and data relating to the location of any retroreflectors within the field of view. In one embodiment, imaging system 100 outputs either an RGB image or data relating to the location of a retroreflector based on an image generated by detector 102. In another embodiment, imaging system 100 simultaneously outputs both an RGB image and data relating to the location of the retroreflector based on a single image generated by detector 102.
Detector 102 comprises a color image sensor and lens system for focusing a field of view onto the image sensor to generate an image of the field of view. In one embodiment, the image sensor comprises a complimentary metal-oxide-semiconductor (CMOS) image sensor, a charge-coupled device (CCD) image sensor, or other suitable type of image sensor.
Light source 108 comprises one or more light sources for illuminating the field of view of detector 102. In one embodiment, light source 108 includes a single light source collocated with detector 102 to provide on-axis illumination. In another embodiment, light source 108 includes multiple light sources with at least one light source collocated with detector 102 to provide on-axis illumination and at least one light source not collocated with detector 102 to provide off-axis illumination. In one embodiment, light source 108 includes one or more light emitting diodes (LEDs), white light sources, vertical cavity surface-emitting lasers (VCSELs) with suitable diffusers as needed to widen the angle of illumination, or other suitable light sources.
Controller 106 controls the operation of and receives image data from detector 102 through communication link 104. Controller 106 also controls light source 108 through communication link 110 to turn light source 108 on or off. Controller 106 controls detector 102 to generate images of the field of view of detector 102. Controller 106 outputs RGB images generated by detector 102 on RGB image signal path 112 to a host device (not shown). Controller 106 also analyzes images generated by detector 102 to determine the location of retroreflectors within the images. Controller 106 outputs the location of the retroreflectors on retroreflector location signal path 114. In one embodiment, the retroreflector location data comprise Cartesian coordinates representing image sensor pixels.
First light source 108A illuminates retroreflector 120 with light as indicated generally at 122. Second light source 108B illuminates retroreflector 120 with light as indicated generally at 126. To detect retroreflector 120 according to one embodiment, two images of retroreflector 120 are generated simultaneously by detector 102 using a single image sensor. A first one of the images is generated using illumination from first light source 108A, which is located at a first angle 128 from axis 132 of detector 102. First light source 108A is close to or on axis 132 of detector 102 (on-axis). The second image is generated using illumination from second light source 108B, which is located at a second angle 130 from axis 132 of detector 102. In one embodiment, angle 130 is greater than angle 128. Second light source 108B is spaced apart from axis 132 of detector 102 (off-axis). Retroreflector 120 reflects light back to where it originated. Therefore, retroreflector 120 reflects most of incident light 122 received from first light source 108A back to first light source 108A, with some captured by detector 102, and retroreflector 120 reflects very little of incident light 126 received from second light source 108B back to detector 102, as indicated generally at 124.
The difference between the on-axis image generated by detector 102 with illumination from first light source 108A and the off-axis image generated by detector 102 with illumination from second light source 108B highlights or emphasizes retroreflector 120. The difference between the images highlights retroreflector 120 because the reflection from retroreflector 120 is detected only in the on-axis image. The diffuse reflections from other environmental features are largely cancelled out, leaving retroreflector 120 as the dominant feature in the differential image. In one embodiment, the differential image is used to detect and/or track the position or location of retroreflector 120.
Differential reflectivity off retroreflector 120 is dependent upon angle 128 between first light source 108A and axis 132 of detector 102, and angle 130 between second light source 108B and axis 132 of detector 102. In general, a smaller angle 128 will increase the retroreflector return. As used herein, “retroreflector return” refers to the intensity (brightness) that is reflected off retroreflector 120 and detected by detector 102. Accordingly, angle 128 is selected such that first light source 108A is on or close to axis 132 of detector 102. In one embodiment, angle 128 is within a range of approximately 0-2 degrees. In general, the size of angle 130 is chosen so that only low or no retroreflector return from second light source 108B is detected at detector 102. In one embodiment, angle 130 is within a range of approximately 3-15 degrees. In other embodiments, the size of angles 128 and 130 are different than the above-specified sizes. In one form of the invention, the size of angles 128 and 130 is determined based on the characteristics of a particular retroreflector 120.
In one embodiment, light sources 108A and 108B emit light that provides substantially equal image intensity (brightness) as sensed by detector 102. In one embodiment, light sources 108A and 108B emit light of different wavelengths. The selected wavelengths are within a range in which detector 102 responds. In one embodiment, light sources 108A and 108B are implemented as LEDs or multimode lasers having infrared or near infrared wavelengths. In another embodiment, light sources 108A and 108B are implemented as LEDs or white lights having visible wavelengths. Each light source 108A and 108B is implemented as one or multiple light sources.
In one form of the invention, a patterned filter layer is formed on image sensor 300 using three different types of filters according to the wavelengths being used by light sources 108A and 108B. In one embodiment, sensor 300 includes red light wavelength filters (R) 302, green light wavelength filters (G) 304A and 304B, and blue light wavelength filters (B) 306. The filters repeat in a two pixel by two pixel pattern over image sensor 300 to provide an RGGB image sensor 300. There are twice as many greens filters as red or blue filters in this embodiment because human perception of brightness depends most strongly on the green range. In this embodiment, image sensor 300 is a typical RGB image sensor for generating color images.
In this embodiment, image sensor 300 provides a first channel that is associated with the on-axis image and a second channel that is associated with the off-axis image. In other embodiments, the first channel is associated with the off-axis image and the second channel is associated with the on-axis image. In one embodiment, the patterned filter layer is deposited as a separate layer of sensor 300, such as, for example, on top of an underlying layer, using conventional deposition and photolithography processes while still in wafer form. In another embodiment, the patterned filter layer is created as a separate element between sensor 300 and incident light. In addition, the filter pattern can be configured in other suitable patterns. For example, the patterned filter layer can be formed into an interlaced striped or a non-symmetrical configuration (e.g., a three pixel by two pixel shape).
Various types of filter materials can be used in the patterned filter layer. In one embodiment, the filter materials include polymers doped with pigments or dyes. In other embodiments, the filter materials include interference filters, reflective filters, and absorbing filters made of semiconductors, other inorganic materials, or organic materials.
Referring back to
In this embodiment, image sensor 320 provides a first channel that is associated with the on-axis image and a second channel that is associated with the off-axis image. In other embodiments, the first channel is associated with the off-axis image and the second channel is associated with the on-axis image. In one embodiment, the patterned filter layer is deposited as a separate layer of image sensor 320, such as, for example, on top of an underlying layer, using conventional deposition and photolithography processes while still in wafer form. In another embodiment, the patterned filter layer is created as a separate element between sensor 320 and incident light. Various types of filter materials can be used in the patterned filter layer as previously described.
Referring back to
In this embodiment, image sensor 340 provides a first channel that is associated with the on-axis image and a second channel that is associated with the off-axis image. In other embodiments, the first channel is associated with the off-axis image and the second channel is associated with the on-axis image. In one embodiment, the patterned filter layer is deposited as a separate layer of sensor 340, such as, for example, on top of an underlying layer, using conventional deposition and photolithography processes while still in wafer form. In another embodiment, the patterned filter layer is created as a separate element between sensor 340 and incident light. Various types of filter materials can be used in the patterned filter layer as previously described.
Referring back to
In one embodiment, light source 108 includes a green light source for use with a detector 102 comprising an RGGB image sensor 300. A strong green retroreflector return through green light wavelength filters 304A and 304B provides the on-axis image, and the equivalent off-axis image is provided through red light wavelength filters 302 and blue light wavelength filters 306. Controller 106 (
In another embodiment, light source 108 includes an infrared light source for use with a detector 102 comprising an RGIB image sensor 340. A strong infrared retroreflector return through infrared light wavelength filters 342 provides the on-axis image, and the equivalent off-axis image is provided through red light wavelength filters 302, green light wavelength filters 304A, and blue light wavelength filters 306. Controller 106 (
In one embodiment, light source 108 provides a plurality of wavelengths of light including the wavelength of light passed by retroreflector filter 121 to retroreflector 120. In another embodiment, light source 108 provides only the wavelength of light passed by retroreflector filter 121 to retroreflector 120 to generate (together with ambient light) the retroreflected on-axis image, and ambient light is used for generating the image with bright retroreflectors at this wavelength. Ambient light alone is used for generating the image with dark retroreflectors as previously described with reference to
In one embodiment, light source 108 includes a white light source or red, green, and blue light sources for use with an RGGB image sensor 300 and a green light wavelength retroreflector filter 121. In another embodiment, light source 108 includes a green light source and ambient light is used in place of the white light source or red, green, and blue light sources for use with the RGGB image sensor 300 and the green light-transmitting retroreflector filter 121. A strong green retroreflector return through green light wavelength filters 304A and 304B provides the retroreflected on-axis image with strong retroreflection, while the other image is provided through red light wavelength filters 302 and blue light wavelength filters 306. Controller 106 generates the retroreflector 120 location data based on the difference between the image with bright retroreflectors and the image with dark retroreflectors. Controller 106 outputs retroreflector 120 location data and/or an RGB image, which is the combination of the images through the R, G and B filters. In this case, in one embodiment, the green channel strength is reduced during subsequent image processing so that the strong green illumination does not unbalance the colors. In other embodiments, a red or blue light source is used in place of the green light source, and retroreflector filter 121 is a red or blue light wavelength filter.
In one embodiment, light source 108 includes a white light source or red, green, and blue light sources and a first infrared light source and a second infrared light source for use with an RGI1/GBI2 image sensor 320 and a first infrared light wavelength retroreflector filter 121. In another embodiment, light source 108 includes a first infrared light source and a second infrared light source and ambient light is used in place of the white light source or red, green, and blue light sources for use with the RGI1/GBI2 image sensor 320 and the first infrared light wavelength-selecting retroreflector filter 121. A strong first infrared light return through first infrared light wavelength filters 322 provides an image with bright retroreflectors, while retroreflector filter 121 blocks the second infrared wavelength to yield an image with dark retroreflectors through second infrared light wavelength filters 324. Controller 106 (
In one embodiment, light source 108 includes an infrared light source and a white light source or red, green, and blue light sources for use with an RGIB image sensor 340 and an infrared light wavelength-selecting retroreflector filter 121. In another embodiment, light source 108 includes an infrared light source and ambient light is used in place of the white light source or red, green, and blue light sources for use with the RGIB image sensor 340 and the infrared light wavelength-selecting retroreflector filter 121. A strong infrared light return through infrared light wavelength-selecting filters 342 provides the image with bright retroreflectors, and the image with dark retroreflectors is provided through red light wavelength filters 302, green light wavelength filters 304A, and blue light wavelength filters 306. Controller 106 (
In other embodiments, other suitable light source 108 and retroreflector filter 121 combinations are used to provide on-axis-illuminated images with bright retroreflectors and similar images with dark retroreflectors for detecting and/or locating retroreflectors. In one embodiment, a gain factor is applied to light that is transmitted through regions of filter material. The gain factor is used to balance the scene signals in one or more images and maximize the feature signals in one or more images.
Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that a variety of alternate and/or equivalent implementations may be substituted for the specific embodiments shown and described without departing from the scope of the present invention. This application is intended to cover any adaptations or variations of the specific embodiments discussed herein. Therefore, it is intended that this invention be limited only by the claims and the equivalents thereof.