Approaches and apparatus for eye detection in a digital image

Information

  • Patent Grant
  • 8090157
  • Patent Number
    8,090,157
  • Date Filed
    Wednesday, February 7, 2007
    17 years ago
  • Date Issued
    Tuesday, January 3, 2012
    12 years ago
Abstract
A system for finding and providing images of eyes acceptable for review, recordation, analysis, segmentation, mapping, normalization, feature extraction, encoding, storage, enrollment, indexing, matching, and/or the like. The system may acquire images of the candidates run them through a contrast filter. The images may be ranked and a number of candidates may be extracted for a list from where a candidate may be selected. Metrics of the eyes may be measured and their profiles evaluated. Also, the spacing between a pair of eyes may be evaluated to confirm the pair's validity. Eye images that do not measure up to certain standards may be discarded and new ones may be selected.
Description
BACKGROUND

Related applications may include U.S. patent application Ser. No. 10/979,129, filed Nov. 3, 2004, which is a continuation-in-part of U.S. patent application Ser. No. 10/655,124, filed Sep. 5, 2003, which are hereby incorporated by reference, and U.S. patent application Ser. No. 11/382,373, filed May 9, 2006, which is hereby incorporated by reference.


U.S. patent application Ser. No. 11/275,703, filed Jan. 25, 2006, is hereby incorporated by reference.


U.S. Provisional Application No. 60/647,270, filed Jan. 26, 2005, is hereby incorporated by reference.


U.S. patent application Ser. No. 11/043,366, filed Jan. 26, 2005, is hereby incorporated by reference.


U.S. patent application Ser. No. 11/372,854, filed Mar. 10, 2006, is hereby incorporated by reference.


U.S. Provisional Application No. 60/778,770, filed Mar. 3, 2006, is hereby incorporated by reference.


SUMMARY

The invention is an approach and apparatus for localizing eyes of a human in a digital image to be processed for iris recognition.





BRIEF DESCRIPTION OF THE DRAWING


FIG. 1
a is a diagram of an overall illustrative structure of an eye finding system;



FIG. 1
b is a diagram with a group structure of the eye finding system;



FIG. 2
a is a diagram of an approach for determining a profile of an eye as provided by a measure of profile metrics;



FIG. 2
b is a diagram of a structure of a profiler;



FIGS. 3
a, 3b and 3c show an image of a selected eye, a pupil image 42 and a binary image 43 of a pupil, respectively;



FIG. 4 is a diagram of an overall iris recognition system;



FIG. 5 shows a diagram of a kernel having a box representing the diameter of a pupil and a box representing a pupil reflection;



FIG. 6 shows a box representing a region of interest having areas which are relatively light, dark and lighter than the relatively light area situated in the dark area representing a pupil model;



FIG. 7 is a histogram of the contrast or intensity values of areas of FIG. 6;



FIG. 8 is like FIG. 6 but does not have a lighter (modeling a typical reflection) than the relatively light area situated in the dark area representing a pupil;



FIG. 9 is a scale of marks representing pixels of a region of interest ranked according to lightness and darkness; and



FIGS. 10
a and 10b relate to eye finding using reflection and/or non-reflection measures.





DESCRIPTION

Eye detection may be the first step toward building a reliable automated iris recognition system in a natural context. Some iris recognition systems rely heavily on predetermined eye locations to properly zoom on the input eye prior to iris segmentation. In addition to biometrics, eye detection (also known as eye finding or eye localization) may support other new technology areas such as eye tracking and human computer interaction or driver drowsiness monitoring systems. Eye detection may serve social learning to identify eye directions like pointing gesture using eye directions.


The present approach and apparatus may be used for finding eyes within a digital image. A local contrast change profiling may be in eye finding. Instead of extracting multiple local features and search globally in the image as many COTS (commercial off-the-shelf) facial recognition packages are based on, the present approach may be based on a system engineering approach to construct the illumination scheme during eye image acquisition to shine the surface of the pupil surface and result into a high reflection point preferably within the pupil of the eye image or close to the pupil of the eye image. This specular reflection point may be used as a reference for an eye search in the digital image. Thus, during the image analysis, the search may be limited to a simplified localization scheme of the highest value pixels associated with these specular reflection pixels and analyze the very local features surrounding these hot spots to confirm an eye profile. To meet the requirements of a real-time system, the eye finding approach may be implemented as a cascade process that divides the local features of an eye into a primary feature of contrast profile associated with high pixel values, depict only potential eye pairs within a specified range, and then test the resulting valid pairs against a feature vector of two or more variables that includes a predefined regular shape fitting with multiple curve fitting measures.


The present approach may be for quickly and robustly localizing the eyes of a human eye in close-up or face images. The approach is based on sensing reflection points within the pupil region as a precursor to the analysis. The approach is formulated to work for cases where reflection is not present within the pupil. The technical approach locates eyes whether there is or there is no reflection. However, in case of the reflection, The detection may hence be simplified to search for these specific reflection points surrounded with dark contrast that represent the pupil. Then the region of interest centered at these potential locations may be processed to find an eye profile. Two valid eyes may be extracted that are within an expected range of eye positioning. The approach for finding eyes decomposes into the following steps. There may be a contrast filter to detect specular reflection pixels. There may be results prioritization to extract valid eye pair with maximum local contrast change. The eye pair may be defined as a valid pair if the two potential eyes are spaced within a predefined range. An adaptive threshold may be applied to detect a central blob. There may be curve fitting of the blob boundaries into a shape. Curve fitness and shape area coverage of the blob surface may be measured for validation. The approach described here may be part of a preprocessing technique used to locate the eyes of a human in a digital image to be processed for iris recognition.


Eye detection may be the first stage for any automated iris recognition analysis system and may be critical for consistent iris segmentation. Several eye detection algorithms may be developed as a basis for face detection. Eye finding approaches may be classified into several categories based upon knowledge based approaches, template matching, and eye socket corner detection. The present approach may address real-time operational requirements. One solution may be to cascade localized features of the eye to speed up the process.


Appearance based approaches using Eigenspace supervised classification technique that is based on learning from a set of training images may be used to capture the most representative variability of eye appearance. Template matching can be regarded as a brute force approach which may include constructing a kernel that is representative of a typical eye socket and convolve the image with the kernel template to identify the highest values of the convolution indicating a match of the eye the identified locations.


Knowledge based approaches may be based on specific rules that are captured by an expert that discriminate the eye local features from any other features. These sets of rules may then be tested against virtually all possible combination to identify the eye locations.


The present approach may provide for quickly and robustly localizing the eyes of a human eye in close-up or face images. The approach may be based on sensing reflection points within the pupil region as a precursor to the analysis. The approach may be also based on sensing the pupil profile in case of no reflection. If reflection is present, the detection may then be simplified to search for these specific reflection points surrounded with dark contrast that represent the pupil. The region of interest centered at these potential locations may then be processed to find an eye profile. Two valid pairs may be extracted that are within an expected range of eye positioning.


The present approach for finding eyes may decompose into the following. To start, the eye may be illuminated to generate a reflection reference point on the pupil surface. The captured wide-field image may be filtered using reflection detection contrast changes to find potential eye locations. For each potential eye location, the local contrast change between the central point and its surrounding pixels may be computed and results may be prioritized to extract valid eye pair with maximum local contrast change. The eye pair may be defined as a valid pair if the two potential eyes are spaced within a predefined range. For each eye of valid eye pair, an adaptive threshold may be executed on a cropped image of the central region of the potential eye to extract a blob of the pupil. Just a single blob may be depicted based on size, its distance to the central point of the cropped image, and how good it fits to a predefined fitting model (e.g., circular shape). With a predefined model shape, such as a circle or an ellipse, the blob edges may be fitted into the pupil fitting model. Curve fitness and model shape area coverage of the blob surface may be measured for validation.


A preprocessing technique may locate the eyes of a human in a digital image to be processed for iris recognition. An overall illustrative structure of an eye finding system 10 is shown in FIG. 1a. The system engineering for eye illumination is not necessarily shown in this system. A digital camera may be used for acquiring 9 images of candidates. The image may be put through a contrast filter 11 to detect relevant high contrast areas of the image. There may a prioritization (i.e., ranking) of the significant spots in the image in block 12. Of these, N candidates may be extracted in block 13. A candidate may have a coordinate c1 (x, y). An output of block 13 may go to block 14 where a new first eye may be selected from the candidate list. From block 14, the eye image candidate may go to a block 15 for a measurement of profile metrics. A profile of the eye may go to a diamond 17 where a determination of the validity of the profile is made. If the profile is not valid, then that first eye may be deleted from the list at block 17. Then at a diamond 18, a count is checked to note whether it is greater than zero. If not, the then this approach is stopped at place 19. If so, then a new first eye may be selected at block 14 from the list from block 13. The profile metrics of this new first eye may be measured at block 15 and passed on to diamond 16 to determine the validity of the profile. If the profile is valid, then the selected first eye may go to place 20, and a second eye is selected at block 21 from the list of candidates from block 13 having a coordinate c2 (x, y). The spacing of the first and second eyes may be determined at block 22 as D(c1 (x, y), c2 (x, y)). The spacing may be checked to see whether it is within an appropriate range at a diamond 23. If not, then the second eye may be deleted from the list at block 24. If so, then metrics of the profile of the second eye may be measured at block 25. The profile metric may be forwarded to a diamond 26 for a determination of the validity of the profile. If the profile is not valid, then the second eye may be deleted from the list at block 24 and at diamond 27, a question of whether the count is greater than zero. If so, then another second eye may be selected from the list at block 21, and the approach via the blocks 21, 23 and 25, and diamonds 23, 26 and 27 may be repeated. If not, then the approach for the second eye may end at place 19. If the profile is valid at diamond 26, then the selected second eye may go to place 20.


A higher level approach to system 10 in FIG. 1a may include an output of the contrast filtering 11 going a select candidate block 111. An output from block 111 may go to a validate profile block 112. Outputs from block 112 may go to a select candidate block 114 and a result block 20, or eliminate candidate block 113. An output of block 113 may go to the select candidate block 111 and/or to the stop place 19. An output from block 114 may go to a validate pair block 115. Block 115 may provide an output to a validate profile block 116. Outputs from block 116 may go to an eliminate candidate 117 and/or to the result block 20. Outputs of block 117 may go the select candidate 114 and the stop place 19. The processing in system 10 may be digital, although it may be analog, or it may be partially digital and analog.



FIG. 1
b is a diagram with a group structure of the eye finding system 10. The corresponding components (according to reference numbers) of FIG. 1a may have additional description. The candidates noted herein may refer to various images of eyes. A camera 9 may be connected to the contrast filter 11. An output of the filter 11 may go to a ranking mechanism 12, which in turn is connected to the candidates extractor 13. The output of extractor 13 may go to a candidate determiner 14 for selecting a new first candidate. Mechanism 12, extractor 13 and determiner 14 constitute a candidate selector 111.


An output of determiner 14 may go to a metric profiler 15 which in turn has an output connected to a profile evaluator 16. Profiler 15 and evaluator 16 may constitute profile validator 112. Outputs of evaluator 16 may go to candidate determiner 21, resulter 20 and candidate remover 17. Remover may have an output that goes to a counter 18. Candidate remover 17 and counter 18 may constitute a candidate eliminator 113. If counter 18 has a count of greater than zero, an output may go to the candidate determiner 14 for selection of a new candidate. If the output is not greater than zero, then an output may go to the stopper 19.


A candidate determiner 21 for selecting a 2nd candidate may have an output to a space measurer 22. The candidate Space measurer 22 may have an output to the range indicator 23 which may indicate whether the two candidates are at an appropriate distance from each other for validation. Measure 22 and indicator 23 may constitute a pair validator 115. Candidate determiner 21 and previously noted ranking mechanism 12 and candidates extractor 13 may constitute a candidate selector 114. If the pair of candidates is valid then an output from validator 115 may go to a profiler 25, or if the pair is not valid then an output from validator 115 may go to a candidate remover 24. An output of profiler 25 may go to a profile evaluator 26 which may determine whether the profile of the second candidate is valid or not. If valid, then an output of evaluator 26 may provide second candidate information to the resulter 20. If invalid, then an output of evaluator 26 may provide a signal to the candidate remover 24. Profiler 25 and profiler evaluator 26 may constitute a profile validator 116. An output of candidate remover may go to a counter 27. If the counter 27 indicates a value greater than zero then an output may go to the candidate determiner 21 for selecting a second candidate. If the counter 27 indicates a value not greater than zero, then an output may go to a stopper 19. The candidate remover 24 and counter 27 may constitute a candidate eliminator 117.



FIG. 2
a shows the approach for determining a profile of an eye as provided by a measure profile metrics or eye profiling block 15, 25. An image 41 of a selected eye (FIG. 3a) may go to an extract pupil region block 31. The block dimension is determined based on the maximum expected value of the pupil diameter. A maximum pupil input 44 may be provided to block 31. An output from block may be a pupil image 42 (FIG. 3b) which goes to an adaptive thresholding block 32. A percent input 45 may be provided to block 32. The pixel distribution to compute the intensity histogram may be provided to block 32. An output of block 32 may be a binary image 43 (FIG. 3c) of the pupil which effectively covers a region of interest. The output of block 32 may go to a find contours block 33. The found contours of image 43 may go to a select n (two or more) most centralized contours block 34. The selected most centralized contours may go to a curve fitting block 35 to curve fit the boundary of the pupil blob to a circle, ellipse or the like. The circle may be adequate for virtually all cases. The output of the curve fitting block may go to a diamond 36 to indicate the level of curve fitness and its' adequacy. The approach is to loop through the n depicted contours to pick the contour that fits the most or best to the model based on the perimeter and coverage fitting. An output 46 from diamond 36 may provide pupil information such as the curve fitting, whether the item is an eye, based upon the fitness measures, the percent of pixels within the curve that fit well the model, the radius and center of the pupil model, and the proportion of the blob that is contained within the pupil model.



FIG. 2
b is a structural version of FIG. 2a. A pupil region extractor 31 of profiler 15, 25 may be connected to an output of the candidate selector 111 or 114 of FIG. 1b. An image 41 and a maximum pupil signal 44 may be input to extractor 31. An output of the extractor 31 may be connected to an adaptive thresholder 32. A percent input 45 may be provided to the thresholder 32. The output 43 (e.g., binary image) may go to a contours finder 33. An input to a most centralized contour picker 34 may be from contours finder 33. An output of the picker 34 may go to a curve fitter 35. An input to the selector of the best curve to fit the model diamond 36 may be from the curve fitter 35. An output 46 may provide pupil information 46 to a profile evaluator 16 or 26.


For the thresholding of block 32, the threshold may be adaptively set based upon the histogram distribution of the intensities of the pixel within the region of interest. A minimum threshold is based upon the coverage of the object of interest (pupil) in pixels with respect to the size of the ROI image (i.e., region of interest). The percentage of the blob size with respect to the ROI is assumed to be at least the ratio of the minimum expected size of a pupil blob (i.e., pupil surface) with respect to the ROI surface (chosen to be the same size of the maximum expected pupil diameter). Hence, the percentage ratio, λ, may be computed with the following equation.










λ
min

=




E


[

S
p

]



S
ROI





π






R
m
2



4






R
M
2




=

.7854



(


R
m


R
M


)

2







(
1
)








Where Rm and RM represent the minimum and maximum possible values of expected radius of the pupil, Sp is the minimum surface of the pupil, SROI is a surface that is a region of interest, and E[ ] is an expected value operator.


Fitness metrics may be used within the eye profiling procedure. At least two metrics can be detected to measure how good the estimated regular shape fits the detected curve at the boundary of the pupil blob. The first curve fitting metric may incorporate the following formula.







η
1

=


1
N





Blob




u


(






F


(

x
,
y

)


-

f


(

x
,
y

)





F


(

x
,
y

)


-


F
c



(

x
,
y

)






-
ɛ

)





x




y








In the above equation, the curve f(x, y) represents the boundary of the blob, F(x, y) is the border curve of estimated fitting shape, and Fc (x, y) is the moment center of the model shape. N in the above equation represents the length of the curve f(x, y) the operator u( ) is the step function and ε<<1 is a tolerance factor.


Another consideration may be given to measuring the proportion of the blob within the estimated model curve. A fitting metrics may be basically the ratio of the estimated shape surface coverage or intersection of the surface of the model and the blob over the blob surface.








η
2

=


Surface






(

blob


F


(

x
,
y

)



)



S
blob



,





where Sblob is the surface of the blob.


A rectilinear image rotation angle may be noted. An iris image capture system that captures both eyes simultaneously may provide a way to measure a head tilt angle. By detecting pupil regions of both eyes during an eye finding procedure, one may calculate the angle of the line passing through both pupil center masses and the horizontal axis of the camera. The eye finder system 10 may then extract both eye images at the estimated orientation axis of the eyes. A misalignment in line detection may be further addressed using the nature of the matching approach which accounts for any non-significant eye orientation. The advantage of the present preprocessing approach is that one may reduce the amount of shifting of bits during the matching process to a few bits thus yielding to faster time response of the system. If rotation correction is not performed, the matching uncertainty may be set to maximum and thus the barcode bit shifting is set to its maximum. On the other hand, if such correction is performed, the matching process may be limited to just a few bits shifted to account for any misalignments of eyes with images in the database.



FIG. 2
b is a diagram of a structure of a profiler.


The overall eye detection system is shown in FIG. 4. It shows a camera 61 that may provide an image with a face in it to the eye finder 10 as noted herein. The eyefinder 10, 62 may provide an image of one or two eyes that go to the iris segmentation block 63. A polar segmentation (POSE) system in block 63 may be used to perform the segmentation. POSE may be based on the assumption that image (e.g., 320×240 pixels) has a visible pupil where iris can be partially visible. There may be pupil segmentation at the inner border between the iris and pupil and segmentation at the outer border between the iris and the sclera and iris and eyelids. An output having a segmented image may go to a block 64 for mapping/normalization and feature extraction. An output from block 64 may go to an encoding block 65 which may provide an output, such as a barcode of the images to block put in terms of ones and zeros. The coding of the images may provide a basis for storage in block 66 of the eye information which may be used for enrolling, indexing, matching, and so on, at block 67, of the eye information, such as that of the iris and pupil, related to the eye.



FIG. 5 shows a diagram of a kernel 70 of a candidate which may be one of several candidates. Box 71 may be selected to fit within a circular shape that would represent the minimum possible diameter of the pupil. Box 72 may be selected to fit within a circular shape that might represent the maximum size of the reflection. The actual circular shapes in FIG. 5 may be used instead of the boxes 70 and 71; however, the circular shape requires much computation and the square shape or box may be regarded as being an adequate approximation. This mechanism may be used to locate pupil location candidates.


A blob suspected of being a pupil may be profiled with a fitness curve on its outer portion. If the curve fits a predefined model like a circle, then one may give it a score of a certain percent of fitness. A second part of the fitness check is to determine what percentage of the pixels of the pupil is contained within the model curve. If the fitness percentages are significant enough to a predefined level, then one might assume that the object scrutinized is a pupil. If so, then the object is checked relative to a range of distance between two eyes.


A threshold level, λ, may be adaptive based on contrast, illumination, and other information. The threshold may be determined with the equation noted herein for λmin. FIG. 6 shows a box 73 which may be a region of interest. An area 74 may be of a first color which is relatively light. An area 75 may be of a second color that is dark. An area 76 may be of a third color that is lighter than the first color. A histogram may be taken of the contents of box or region 73. The histogram may look like the graph of FIG. 7. The ordinate axis represents the number of pixels having a contrast or intensity (i.e., lightness/darkness) value of the values represented on the abscissa axis, which range from 0 to 255, i.e., from dark to light, respectively. The result is two peaks 78 and 79 with a middle point 77 which may be associated with the λmin. The plot 81 appears normal. Other plots having one peak, a flat peak or peaks, peaks having a large separation, or other appearance that appear abnormal relative to plot generally indicate an unacceptable situation. One may note that the present approach utilizes adaptive thresholding which has a threshold that is not fixed or arbitrary. The depicted threshold is limited with the minimum value of that defined by equation (1).


There may be a situation where there is no reflection to be found on a pupil. FIG. 8 is like FIG. 6 which has an area 76 of reflection on pupil 75 which FIG. 8 does not have. However, an area 86 of reflection may be assumed for the pupil 85 in FIG. 8. The pixels of a region of interest or kernel 87 may be ranked according to lightness and darkness as represented by marks on a scale 95 of diagram 90 as shown in FIG. 9. An arrow 96 represents a direction of increasingly lighter pixels. An arrow 97 represents a direction of increasingly darker pixels. For illustrative purposes, each mark may represent a pixel; although each mark could represent any number of pixels or a fraction of a pixel or pixels. The kernel 87 size may be N pixels. N pixels may be represented by a group 93 of marks on a scale 95. The reflection 86 may be represented by “Nrfc”. “Nrfc” may refer to the reflection 86 pixels. The “Nrfc” pixels may be represented by a group 91 of marks on scale 95. “N-Nrfc” may represent the dark area 85. The “N-Nrfc” pixels may be represented by a group 92 of marks on scale 95.


In cases where we have reflections on the pupil, the measure may be defined as the argument of the maximum difference between the reflection pixel measure (local maxima) within the reflection spot and the average mean of the dark pixels that represent the pupil profile. Hence,








C
pupil



(

x
,
y

)


=

arg







max

(

x
,
y

)




(


v
max

-

μ
o


)







The vector {right arrow over (v)}(n) is the kernel elements sorted in a descending order based on the intensity values as shown in FIG. 9. An average value of intensity may be calculated for each group of pixels.


For the “Nrfc” group 91, one may have the local maxima of the reflection spot vmax estimated as the average mean of only the first K elements of the reflection pixels. K may be selected to be such as K<<Nrfc. For the “N-Nrfc” group 92, one may have “μo”,







μ
o

=


1

N
krn








N
rfc

<
n
<

N
krn






v
->



(
n
)









FIGS. 10
a and 10b relate to eye finding using reflection and/or non-reflection measures relative to eyes 101 and 104, respectively. For a situation of no actual reflection on the pupil, then there may be a representative value of the dark pixels in the bottom scale that maximize the argument 1/μo. This may be true for either condition whether there is reflection or no reflection. Hence, the formulas may be combined into one to work for both situations as indicated by the following equation,








C
pupil



(

x
,
y

)


=

arg







max

(

x
,
y

)





(


(


ϑ
max

-

μ
o


)



μ
o



(

x
,
y

)



)

.







In the present specification, some of the matter may be of a hypothetical or prophetic nature although stated in another manner or tense.


Although the invention has been described with respect to at least one illustrative example, many variations and modifications will become apparent to those skilled in the art upon reading the present specification. It is therefore the intention that the appended claims be interpreted as broadly as possible in view of the prior art to include all such variations and modifications.

Claims
  • 1. A computer implemented eye detection system for detecting eyes in a digital image, the system comprising: a digital image capture device; anda processor, the processor including: a filter;a first eye candidate selector connected to the filter;a first profile validator connected to the eye candidate selector, the first profile validator including measurements of eye candidate pupil contours;a first eye candidate eliminator connected to the first profile validator and the first eye candidate selector;a second eye candidate selector connected to the first profile validator;a pair validator connected to the second eye candidate selector, the pair validator including a space measurer that determines if the first and second eye candidates are at an appropriate distance from each other;a second profile validator connected to the pair validator; anda second eye candidate eliminator connected to the second profile validator and the second eye candidate selector;wherein the first profile validator comprising a first profiler connected to the first eye candidate selector, and a first profile evaluator connected to the first profiler, the first eye candidate eliminator and the second eye candidate selector;wherein the second profile validator comprises a second profiler connected to the pair validator, and a second profile evaluator connected to the second profiler and the second eye candidate eliminator;wherein the first profiler comprises: a pupil region extractor connected to the first eye candidate selector;an adaptive thresholder connected to the pupil region extractor;a contours finder connected to the adaptive thresholder;a contour picker connected to the contours finder;a curve fitter connected to the contour picker; anda curve selector connected to the curve fitter and the first profile evaluator; andwherein the second profiler comprises: a pupil region extractor connected to the pair validator;an adaptive thresholder connected to the pupil region extractor;a contours finder connected to the adaptive thresholder;a contour picker connected to the contours finder;a curve fitter connected to the contour picker; anda curve selector connected to the curve fitter and the second profile evaluator.
  • 2. The system of claim 1, further comprising: a resulter connected to the first and second profile validators; anda stopper connected to the first and second eye candidate eliminators.
  • 3. The system of claim 1, wherein: the first eye candidate selector comprises: a ranking mechanism connected to the filter;an eye candidates extractor connected to the ranking mechanism; anda first eye candidate determiner connected to the eye candidate extractor, a first profiler and the first eye candidate eliminator; andthe second eye candidate selector comprises: the ranking mechanism connected to the filter;the eye candidates extractor connected to the ranking mechanism; anda second eye candidate determiner connected to the first profile evaluator, the pair validator and the second eye candidate eliminator.
  • 4. A computer implemented method for finding an eye in a digital image, comprising: providing a digital image capture device and a processor;obtaining a digital image containing eye candidates from the digital image capture device; andprocessing the digital image to find an eye with the processor, the processing including: contrast filtering the eye candidates;selecting a first eye from the eye candidates;validating a profile of the first eye;selecting a second eye from the eye candidates;validating an amount of space between the first and second eyes;validating a profile of the second eye, wherein validating the profile of the first and second eyes includes measuring pupil contours;eliminating the first eye if the first eye has an invalid profile;eliminating the second eye if the amount of space between the first and second eyes is invalid; andeliminating the second eye if the amount of space between the first and second eyes is valid and the second eye has an invalid profile.
  • 5. A computer implemented eye detection system for detecting eyes having pupils in a digital image, the system comprising: a digital image capture device; anda processor, the processor including: a filter;a first eye candidate selector connected to the filter;a first profile validator connected to the eye candidate selector, the profile validator including measurements of pupil contours;a first eye candidate eliminator connected to the first profile validator and the first eye candidate selector;a second eye candidate selector connected to the first profile validator;a pair validator connected to the second eye candidate selector, the pair validator including a space measurer that determines if the first and second eye candidates are at an appropriate distance from each other;a second profile validator connected to the pair validator; anda second eye candidate eliminator connected to the second profile validator and the second eye candidate selector;wherein the first profile validator comprises: a first profiler connected to the first eye candidate selector; anda first profile evaluator connected to the first profiler, the first eye candidate eliminator and the second eye candidate selector; andwherein the second profile validator comprises: a second profiler connected to the pair validator; anda second profile evaluator connected to the second profiler and the second eye candidate eliminator;wherein the first profiler comprises: a pupil region extractor connected to the first eye candidate selector;an adaptive thresholder connected to the pupil region extractor;a contours finder connected to the adaptive thresholder;a contour picker connected to the contours finder;a curve fitter connected to the contour picker; anda curve selector connected to the curve fitter and the first profile evaluator; andwherein the second profiler comprises: a pupil region extractor connected to the pair validator;an adaptive thresholder connected to the pupil region extractor;a contours finder connected to the adaptive thresholder;a contour picker connected to the contours finder;a curve fitter connected to the contour picker; anda curve selector connected to the curve fitter and the second profile evaluator.
  • 6. An eye finder system for finding eyes in a digital image comprising: a camera;a filter connected to the camera;an eye candidate lister connected to the filter;an eye selector connected to the eye candidate lister;an eye profile evaluator connected to the eye selector; andan eye eliminator connected to the eye profile evaluator;wherein the eye eliminator comprises: a deleter, connected to the eye profile evaluator, for removing the eye from a list provided by the eye candidate lister if the eye profile evaluator indicates a profile of the eye to be invalid; anda counter connected to the deleter and to the eye selector;if the counter has a count greater than zero, then the eye selector may select a new eye from the list provided by the eye candidate lister; orif the counter has a count not greater than zero, then the system may stop.
  • 7. The system of claim 1, wherein an adaptive threshold is computed adaptively based upon a pupil size and a contrast distribution of an image region having a pupil and its surrounding area.
  • 8. The system of claim 1, wherein the pair validator comprises: a range indicator connected to the space measurer and the second profile validator.
  • 9. The system of claim 1, further comprising: a resulter connected to the first and second profile validators; andwherein the resulter is for receiving a valid pair of eye candidates.
  • 10. The system of claim 1, wherein: the filter is a contrast filter.
  • 11. The system of claim 10, wherein the contrast filter is based upon a pupil dark profile with respect to its surroundings.
  • 12. The system of claim 10, wherein the contrast filter is based upon a light reflection profile with respect to its surroundings.
  • 13. A computer implemented method for finding an eye in a digital image, comprising: providing a digital image capture device and a processor;obtaining a digital image containing eye candidates from the digital image capture device; andprocessing the digital image to find an eye with the processor, the processing including: contrast filtering the eye candidates;selecting a first eye from the eye candidates;validating a profile of the first eye;selecting a second eye from the eye candidates;validating an amount of space between the first and second eyes;validating a profile of the second eye, wherein validating the profile of the first and second eyes includes measuring pupil contours;eliminating the first eye if the first eye has an invalid profile;eliminating the second eye if the amount of space between the first and second eyes is invalid; andeliminating the second eye if the amount of space between the first and second eyes is valid and the second eye has an invalid profile.
  • 14. The method of claim 13, wherein: the selecting a first eye comprises: prioritizing the eye candidates according to contrast into a first list of eye candidates;extracting a number of eye candidates from the first list of eye candidates into a second list of eye candidates; andselecting a first eye from the second list of eye candidates; andthe selecting a second eye comprises selecting a second eye from the second list of eye candidates.
  • 15. The method of claim 14, wherein: the validating a profile of first eye comprises: measuring profile metrics of the first eye; anddetermining whether the profile metrics of the first eye are valid; andthe validating a profile of second eye comprises: measuring profile metrics of the second eye; anddetermining whether the profile metrics of the second eye are valid.
  • 16. The method of claim 15, wherein the validating an amount of space between the first and second eyes comprises: measuring a spacing between the first and second eyes; anddetermining whether the spacing is within a valid range.
  • 17. An eye finder system for finding eyes in a digital image comprising: a camera;a filter connected to the camera;an eye candidate lister connected to the filter;an eye selector connected to the eye candidate lister;an eye profile evaluator connected to the eye selector, the eye profile evaluator measuring pupil image and pupil contours, and including a pair validator, the pair validator including a space measurer that determines if the first and second eye candidates are at an appropriate distance from each other; andan eye eliminator connected to the eye profile evaluator, the eye eliminator comprising: a deleter, connected to the eye profile evaluator, for removing the eye from a list provided by the eye candidate lister if the eye profile evaluator indicates a profile of the eye to be invalid; anda counter connected to the deleter and to the eye selector;wherein if the counter has a count greater than zero, then the eye selector may select a new eye from the list provided by the eye candidate lister; orif the counter has a count not greater than zero, then the system may stop.
  • 18. The system of claim 17, further comprising: a second eye selector connected to the eye profile evaluator;an eye pair evaluator connected to the second eye selector; anda second profile evaluator connected to the eye pair evaluator.
  • 19. The system of claim 18, wherein if the profile evaluator and the second profile evaluator indicate that a first eye and a second eye have appropriate metrics, and the eye pair evaluator determines that the first eye and second eye are appropriately spaced from each other, then the first and second eyes or portions of them are acceptable for review, recordation, analysis, segmentation, mapping, normalization, feature extraction, encoding, storage, enrollment, indexing, matching, and/or the like.
  • 20. The system of claim 18, wherein if just one eye is indicated by one or more profile evaluators to have appropriate metrics, then the eye or a portion of it is acceptable for review, recordation, analysis, segmentation, mapping, normalization, feature extraction, encoding, storage, enrollment, indexing, matching, and/or the like.
  • 21. The system of claim 17, wherein the eye profile evaluator comprises: a pupil region extractor connected to the eye selector;an adaptive thresholder connected to the pupil region extractor;a contours finder connected to the adaptive thresholder;a contour picker connected to the contours finder;a curve fitter connected to the contour picker; anda best curve selector connected to the curve fitter.
Parent Case Info

This application is a continuation-in-part of U.S. patent application Ser. No. 11/275,703, filed Jan. 25, 2006, which claims the benefit of U.S. Provisional Application No. 60/647,270, filed Jan. 26, 2005. This application is a continuation-in-part of U.S. patent application Ser. No. 11/043,366, filed Jan. 26, 2005. This application is a continuation-in-part of U.S. patent application Ser. No. 11/372,854, filed Mar. 10, 2006; This application claims the benefit of U.S. Provisional Application No. 60/778,770, filed Mar. 3, 2006.

Government Interests

The government may have rights in the present invention.

US Referenced Citations (396)
Number Name Date Kind
4641349 Flom et al. Feb 1987 A
4836670 Hutchinson Jun 1989 A
5231674 Cleveland et al. Jul 1993 A
5291560 Daugman Mar 1994 A
5293427 Ueno et al. Mar 1994 A
5359382 Uenaka Oct 1994 A
5404013 Tajima Apr 1995 A
5551027 Choy et al. Aug 1996 A
5572596 Wildes et al. Nov 1996 A
5608472 Szirth et al. Mar 1997 A
5664239 Nakata Sep 1997 A
5717512 Chmielewski, Jr. et al. Feb 1998 A
5751836 Wildes et al. May 1998 A
5859686 Aboutalib et al. Jan 1999 A
5860032 Iwane Jan 1999 A
5896174 Nakata Apr 1999 A
5901238 Matsushita May 1999 A
5909269 Isogai et al. Jun 1999 A
5953440 Zhang et al. Sep 1999 A
5956122 Doster Sep 1999 A
5978494 Zhang Nov 1999 A
6005704 Chmielewski, Jr. et al. Dec 1999 A
6007202 Apple et al. Dec 1999 A
6012376 Hanke et al. Jan 2000 A
6021210 Camus et al. Feb 2000 A
6028949 McKendall Feb 2000 A
6055322 Salganicoff et al. Apr 2000 A
6064752 Rozmus et al. May 2000 A
6069967 Rozmus et al. May 2000 A
6081607 Mori et al. Jun 2000 A
6088470 Camus et al. Jul 2000 A
6091899 Konishi et al. Jul 2000 A
6101477 Hohle et al. Aug 2000 A
6104431 Inoue et al. Aug 2000 A
6108636 Yap et al. Aug 2000 A
6119096 Mann et al. Sep 2000 A
6120461 Smyth Sep 2000 A
6134339 Luo Oct 2000 A
6144754 Okano et al. Nov 2000 A
6246751 Bergl et al. Jun 2001 B1
6247813 Kim et al. Jun 2001 B1
6252977 Salganicoff et al. Jun 2001 B1
6282475 Washington Aug 2001 B1
6285505 Melville et al. Sep 2001 B1
6285780 Yamakita et al. Sep 2001 B1
6289113 McHugh et al. Sep 2001 B1
6299306 Braithwaite et al. Oct 2001 B1
6308015 Matsumoto Oct 2001 B1
6309069 Seal et al. Oct 2001 B1
6320610 Van Sant et al. Nov 2001 B1
6320612 Young Nov 2001 B1
6320973 Suzaki et al. Nov 2001 B2
6323761 Son Nov 2001 B1
6325765 Hay et al. Dec 2001 B1
6330674 Angelo et al. Dec 2001 B1
6332193 Glass et al. Dec 2001 B1
6344683 Kim Feb 2002 B1
6370260 Pavlidis et al. Apr 2002 B1
6377699 Musgrave et al. Apr 2002 B1
6393136 Amir et al. May 2002 B1
6400835 Lemelson et al. Jun 2002 B1
6424727 Musgrave et al. Jul 2002 B1
6424845 Emmoft et al. Jul 2002 B1
6433818 Steinberg et al. Aug 2002 B1
6438752 McClard Aug 2002 B1
6441482 Foster Aug 2002 B1
6446045 Stone et al. Sep 2002 B1
6483930 Musgrave et al. Nov 2002 B1
6484936 Nicoll et al. Nov 2002 B1
6490443 Freeny, Jr. Dec 2002 B1
6493363 Roger et al. Dec 2002 B1
6493669 Curry et al. Dec 2002 B1
6494363 Roger et al. Dec 2002 B1
6503163 Van Sant et al. Jan 2003 B1
6505193 Musgrave et al. Jan 2003 B1
6506078 Mori et al. Jan 2003 B1
6508397 Do Jan 2003 B1
6516078 Yang et al. Feb 2003 B1
6516087 Camus Feb 2003 B1
6516416 Gregg et al. Feb 2003 B2
6522772 Morrison et al. Feb 2003 B1
6523165 Liu et al. Feb 2003 B2
6526160 Ito Feb 2003 B1
6532298 Cambier et al. Mar 2003 B1
6540392 Braithwaite Apr 2003 B1
6542624 Oda Apr 2003 B1
6546121 Oda Apr 2003 B1
6553494 Glass Apr 2003 B1
6580356 Alt et al. Jun 2003 B1
6591001 Oda et al. Jul 2003 B1
6591064 Higashiyama et al. Jul 2003 B2
6594377 Kim et al. Jul 2003 B1
6594399 Camus et al. Jul 2003 B1
6598971 Cleveland Jul 2003 B2
6600878 Pregara Jul 2003 B2
6614919 Suzaki et al. Sep 2003 B1
6652099 Chae et al. Nov 2003 B2
6674367 Sweatte Jan 2004 B2
6690997 Rivalto Feb 2004 B2
6708176 Strunk et al. Mar 2004 B2
6711562 Ross et al. Mar 2004 B1
6714665 Hanna et al. Mar 2004 B1
6718049 Pavlidis et al. Apr 2004 B2
6718665 Hess et al. Apr 2004 B2
6732278 Baird, III et al. May 2004 B2
6734783 Anbai May 2004 B1
6745520 Puskaric et al. Jun 2004 B2
6750435 Ford Jun 2004 B2
6751733 Nakamura et al. Jun 2004 B1
6753919 Daugman Jun 2004 B1
6754640 Bozeman Jun 2004 B2
6760467 Min et al. Jul 2004 B1
6765470 Shinzaki Jul 2004 B2
6766041 Golden et al. Jul 2004 B2
6775774 Harper Aug 2004 B1
6785406 Kamada Aug 2004 B1
6793134 Clark Sep 2004 B2
6819219 Bolle et al. Nov 2004 B1
6829370 Pavlidis et al. Dec 2004 B1
6832044 Doi et al. Dec 2004 B2
6836554 Bolle et al. Dec 2004 B1
6837436 Swartz et al. Jan 2005 B2
6845479 Park Jan 2005 B2
6853444 Haddad Feb 2005 B2
6867683 Calvesio et al. Mar 2005 B2
6873960 Wood et al. Mar 2005 B1
6896187 Stockhammer May 2005 B2
6905411 Nguyen et al. Jun 2005 B2
6920237 Chen et al. Jul 2005 B2
6930707 Bates et al. Aug 2005 B2
6934849 Kramer et al. Aug 2005 B2
6950139 Fujinawa Sep 2005 B2
6954738 Wang et al. Oct 2005 B2
6957341 Rice et al. Oct 2005 B2
6972797 Izumi Dec 2005 B2
6992562 Fuks et al. Jan 2006 B2
7053948 Konishi May 2006 B2
7071971 Elberbaum Jul 2006 B2
7084904 Liu et al. Aug 2006 B2
7136581 Fujii Nov 2006 B2
7183895 Bazakos et al. Feb 2007 B2
7184577 Chen et al. Feb 2007 B2
7197173 Jones et al. Mar 2007 B2
7204425 Mosher, Jr. et al. Apr 2007 B2
7277891 Howard et al. Oct 2007 B2
7298873 Miller, Jr. et al. Nov 2007 B2
7315233 Yuhara Jan 2008 B2
7362210 Bazakos et al. Apr 2008 B2
7362370 Sakamoto et al. Apr 2008 B2
7362884 Willis et al. Apr 2008 B2
7365771 Kahn et al. Apr 2008 B2
7406184 Wolff et al. Jul 2008 B2
7414648 Imada Aug 2008 B2
7417682 Kuwakino et al. Aug 2008 B2
7418115 Northcott et al. Aug 2008 B2
7421097 Hamza et al. Sep 2008 B2
7443441 Hiraoka Oct 2008 B2
7460693 Loy et al. Dec 2008 B2
7471451 Dent et al. Dec 2008 B2
7486806 Azuma et al. Feb 2009 B2
7518651 Butterworth Apr 2009 B2
7537568 Moehring May 2009 B2
7538326 Johnson et al. May 2009 B2
7542945 Thompson et al. Jun 2009 B2
7580620 Raskar et al. Aug 2009 B2
7593550 Hamza Sep 2009 B2
7639846 Yoda Dec 2009 B2
7722461 Gatto et al. May 2010 B2
7751598 Matey et al. Jul 2010 B2
7756301 Hamza Jul 2010 B2
7756407 Raskar Jul 2010 B2
7761453 Hamza Jul 2010 B2
7777802 Shinohara et al. Aug 2010 B2
7804982 Howard et al. Sep 2010 B2
20010026632 Tamai Oct 2001 A1
20010027116 Baird Oct 2001 A1
20010047479 Bromba et al. Nov 2001 A1
20010051924 Uberti Dec 2001 A1
20010054154 Tam Dec 2001 A1
20020010857 Karthik Jan 2002 A1
20020033896 Hatano Mar 2002 A1
20020039433 Shin Apr 2002 A1
20020040434 Elliston et al. Apr 2002 A1
20020062280 Zachariassen et al. May 2002 A1
20020077841 Thompson Jun 2002 A1
20020089157 Breed et al. Jul 2002 A1
20020106113 Park Aug 2002 A1
20020112177 Voltmer et al. Aug 2002 A1
20020114495 Chen et al. Aug 2002 A1
20020130961 Lee et al. Sep 2002 A1
20020131622 Lee et al. Sep 2002 A1
20020139842 Swaine Oct 2002 A1
20020140715 Smet Oct 2002 A1
20020142844 Kerr Oct 2002 A1
20020144128 Rahman et al. Oct 2002 A1
20020150281 Cho Oct 2002 A1
20020154794 Cho Oct 2002 A1
20020158750 Almalik Oct 2002 A1
20020164054 McCartney et al. Nov 2002 A1
20020175182 Matthews Nov 2002 A1
20020186131 Fettis Dec 2002 A1
20020191075 Doi et al. Dec 2002 A1
20020191076 Wada et al. Dec 2002 A1
20020194128 Maritzen et al. Dec 2002 A1
20020194131 Dick Dec 2002 A1
20020198731 Barnes et al. Dec 2002 A1
20030002714 Wakiyama Jan 2003 A1
20030012413 Kusakari et al. Jan 2003 A1
20030014372 Wheeler et al. Jan 2003 A1
20030020828 Ooi et al. Jan 2003 A1
20030038173 Blackson et al. Feb 2003 A1
20030046228 Berney Mar 2003 A1
20030053663 Chen et al. Mar 2003 A1
20030055689 Block et al. Mar 2003 A1
20030055787 Fujii Mar 2003 A1
20030058492 Wakiyama Mar 2003 A1
20030061172 Robinson Mar 2003 A1
20030061233 Manasse et al. Mar 2003 A1
20030065626 Allen Apr 2003 A1
20030071743 Seah et al. Apr 2003 A1
20030072475 Tamori Apr 2003 A1
20030073499 Reece Apr 2003 A1
20030074317 Hofi Apr 2003 A1
20030074326 Byers Apr 2003 A1
20030076161 Tisse Apr 2003 A1
20030076300 Lauper et al. Apr 2003 A1
20030076984 Tisse et al. Apr 2003 A1
20030080194 O'Hara et al. May 2003 A1
20030091215 Lauper et al. May 2003 A1
20030092489 Veradej May 2003 A1
20030095689 Volkommer et al. May 2003 A1
20030098776 Friedli May 2003 A1
20030099379 Monk et al. May 2003 A1
20030099381 Ohba May 2003 A1
20030103652 Lee et al. Jun 2003 A1
20030107097 McArthur et al. Jun 2003 A1
20030107645 Yoon Jun 2003 A1
20030108224 Ike Jun 2003 A1
20030108225 Li Jun 2003 A1
20030115148 Takhar Jun 2003 A1
20030115459 Monk Jun 2003 A1
20030116630 Carey et al. Jun 2003 A1
20030118212 Min et al. Jun 2003 A1
20030118217 Kondo et al. Jun 2003 A1
20030123711 Kim et al. Jul 2003 A1
20030125054 Garcia Jul 2003 A1
20030125057 Pesola Jul 2003 A1
20030126560 Kurapati et al. Jul 2003 A1
20030131245 Linderman Jul 2003 A1
20030131265 Bhakta Jul 2003 A1
20030133597 Moore et al. Jul 2003 A1
20030140235 Immega et al. Jul 2003 A1
20030140928 Bui et al. Jul 2003 A1
20030141411 Pandya et al. Jul 2003 A1
20030149881 Patel et al. Aug 2003 A1
20030152251 Ike Aug 2003 A1
20030152252 Kondo et al. Aug 2003 A1
20030156741 Lee et al. Aug 2003 A1
20030158762 Wu Aug 2003 A1
20030158821 Maia Aug 2003 A1
20030159051 Hollnagel Aug 2003 A1
20030163739 Armington et al. Aug 2003 A1
20030169334 Braithwaite et al. Sep 2003 A1
20030169901 Pavlidis et al. Sep 2003 A1
20030169907 Edwards et al. Sep 2003 A1
20030173408 Mosher, Jr. et al. Sep 2003 A1
20030174049 Beigel et al. Sep 2003 A1
20030177051 Driscoll et al. Sep 2003 A1
20030182151 Taslitz Sep 2003 A1
20030182182 Kocher Sep 2003 A1
20030189480 Hamid Oct 2003 A1
20030189481 Hamid Oct 2003 A1
20030191949 Odagawa Oct 2003 A1
20030194112 Lee Oct 2003 A1
20030195935 Leeper Oct 2003 A1
20030198368 Kee Oct 2003 A1
20030200180 Phelan, III et al. Oct 2003 A1
20030210139 Brooks et al. Nov 2003 A1
20030210802 Schuessier Nov 2003 A1
20030218719 Abourizk et al. Nov 2003 A1
20030225711 Paping Dec 2003 A1
20030228898 Rowe Dec 2003 A1
20030233556 Angelo et al. Dec 2003 A1
20030235326 Morikawa et al. Dec 2003 A1
20030235411 Morikawa et al. Dec 2003 A1
20030236120 Reece et al. Dec 2003 A1
20040001614 Russon et al. Jan 2004 A1
20040002894 Kocher Jan 2004 A1
20040005078 Tillotson Jan 2004 A1
20040006553 de Vries et al. Jan 2004 A1
20040010462 Moon et al. Jan 2004 A1
20040012760 Mihashi et al. Jan 2004 A1
20040019570 Bolle et al. Jan 2004 A1
20040023664 Mirouze et al. Feb 2004 A1
20040023709 Beaulieu et al. Feb 2004 A1
20040025030 Corbett-Clark et al. Feb 2004 A1
20040025031 Ooi et al. Feb 2004 A1
20040025053 Hayward Feb 2004 A1
20040029564 Hodge Feb 2004 A1
20040030930 Nomura Feb 2004 A1
20040035123 Kim et al. Feb 2004 A1
20040037450 Bradski Feb 2004 A1
20040037452 Shin Feb 2004 A1
20040039914 Barr et al. Feb 2004 A1
20040042641 Jakubowski Mar 2004 A1
20040044627 Russell et al. Mar 2004 A1
20040046640 Jourdain et al. Mar 2004 A1
20040049687 Orsini et al. Mar 2004 A1
20040050924 Mletzko et al. Mar 2004 A1
20040050930 Rowe Mar 2004 A1
20040052405 Walfridsson Mar 2004 A1
20040052418 DeLean Mar 2004 A1
20040059590 Mercredi et al. Mar 2004 A1
20040059953 Purnell Mar 2004 A1
20040104266 Bolle et al. Jun 2004 A1
20040117636 Cheng Jun 2004 A1
20040133804 Smith et al. Jul 2004 A1
20040146187 Jeng Jul 2004 A1
20040148526 Sands et al. Jul 2004 A1
20040160518 Park Aug 2004 A1
20040162870 Matsuzaki et al. Aug 2004 A1
20040162984 Freeman et al. Aug 2004 A1
20040169817 Grotehusmann et al. Sep 2004 A1
20040172541 Ando et al. Sep 2004 A1
20040174070 Voda et al. Sep 2004 A1
20040190759 Caldwell Sep 2004 A1
20040193893 Braithwaite et al. Sep 2004 A1
20040219902 Lee et al. Nov 2004 A1
20040233038 Beenau et al. Nov 2004 A1
20040240711 Hamza et al. Dec 2004 A1
20040252866 Tisse et al. Dec 2004 A1
20040255168 Murashita et al. Dec 2004 A1
20050008200 Azuma et al. Jan 2005 A1
20050008201 Lee et al. Jan 2005 A1
20050012817 Hampapur et al. Jan 2005 A1
20050029353 Isemura et al. Feb 2005 A1
20050041867 Loy et al. Feb 2005 A1
20050052566 Kato Mar 2005 A1
20050055582 Bazakos et al. Mar 2005 A1
20050063567 Saitoh et al. Mar 2005 A1
20050084137 Kim et al. Apr 2005 A1
20050084179 Hanna et al. Apr 2005 A1
20050099288 Spitz et al. May 2005 A1
20050102502 Sagen May 2005 A1
20050110610 Bazakos et al. May 2005 A1
20050125258 Yellin et al. Jun 2005 A1
20050127161 Smith et al. Jun 2005 A1
20050129286 Hekimian Jun 2005 A1
20050134796 Zelvin et al. Jun 2005 A1
20050138385 Friedli et al. Jun 2005 A1
20050138387 Lam et al. Jun 2005 A1
20050146640 Shibata Jul 2005 A1
20050151620 Neumann Jul 2005 A1
20050152583 Kondo et al. Jul 2005 A1
20050193212 Yuhara Sep 2005 A1
20050199708 Friedman Sep 2005 A1
20050206501 Farhat Sep 2005 A1
20050206502 Bernitz Sep 2005 A1
20050207614 Schonberg et al. Sep 2005 A1
20050210267 Sugano et al. Sep 2005 A1
20050210270 Rohatgi et al. Sep 2005 A1
20050210271 Chou et al. Sep 2005 A1
20050238214 Matsuda et al. Oct 2005 A1
20050240778 Saito Oct 2005 A1
20050248725 Ikoma et al. Nov 2005 A1
20050249385 Kondo et al. Nov 2005 A1
20050255840 Markham Nov 2005 A1
20060093190 Cheng et al. May 2006 A1
20060147094 Yoo Jul 2006 A1
20060165266 Hamza Jul 2006 A1
20060274919 LoIacono et al. Dec 2006 A1
20070036397 Hamza Feb 2007 A1
20070140531 Hamza Jun 2007 A1
20070160266 Jones et al. Jul 2007 A1
20070206840 Jacobson Sep 2007 A1
20070211924 Hamza Sep 2007 A1
20070274570 Hamza Nov 2007 A1
20070274571 Hamza Nov 2007 A1
20070286590 Terashima Dec 2007 A1
20080005578 Shafir Jan 2008 A1
20080075334 Determan et al. Mar 2008 A1
20080075441 Jelinek et al. Mar 2008 A1
20080104415 Palti-Wasserman et al. May 2008 A1
20080148030 Goffin Jun 2008 A1
20080211347 Wright et al. Sep 2008 A1
20080252412 Larsson et al. Oct 2008 A1
20080267456 Anderson Oct 2008 A1
20090046899 Northcott et al. Feb 2009 A1
20090092283 Whillock et al. Apr 2009 A1
20090316993 Brasnett et al. Dec 2009 A1
20100002913 Hamza Jan 2010 A1
20100033677 Jelinek Feb 2010 A1
20100034529 Jelinek Feb 2010 A1
20100142765 Hamza Jun 2010 A1
20100182440 McCloskey Jul 2010 A1
20100239119 Bazakos et al. Sep 2010 A1
Foreign Referenced Citations (188)
Number Date Country
0484076 May 1992 EP
0593386 Apr 1994 EP
0878780 Nov 1998 EP
0899680 Mar 1999 EP
0910986 Apr 1999 EP
0962894 Dec 1999 EP
1018297 Jul 2000 EP
1024463 Aug 2000 EP
1028398 Aug 2000 EP
1041506 Oct 2000 EP
1041523 Oct 2000 EP
1126403 Aug 2001 EP
1139270 Oct 2001 EP
1237117 Sep 2002 EP
1477925 Nov 2004 EP
1635307 Mar 2006 EP
2369205 May 2002 GB
2371396 Jul 2002 GB
2375913 Nov 2002 GB
2402840 Dec 2004 GB
2411980 Sep 2005 GB
9161135 Jun 1997 JP
9198545 Jul 1997 JP
9201348 Aug 1997 JP
9147233 Sep 1997 JP
9234264 Sep 1997 JP
9305765 Nov 1997 JP
9319927 Dec 1997 JP
10021392 Jan 1998 JP
10040386 Feb 1998 JP
10049728 Feb 1998 JP
10137219 May 1998 JP
10137221 May 1998 JP
10137222 May 1998 JP
10137223 May 1998 JP
10248827 Sep 1998 JP
10269183 Oct 1998 JP
11047117 Feb 1999 JP
11089820 Apr 1999 JP
11200684 Jul 1999 JP
11203478 Jul 1999 JP
11213047 Aug 1999 JP
11339037 Dec 1999 JP
2000005149 Jan 2000 JP
2000005150 Jan 2000 JP
2000011163 Jan 2000 JP
2000023946 Jan 2000 JP
2000083930 Mar 2000 JP
2000102510 Apr 2000 JP
2000102524 Apr 2000 JP
2000105830 Apr 2000 JP
2000107156 Apr 2000 JP
2000139878 May 2000 JP
2000155863 Jun 2000 JP
2000182050 Jun 2000 JP
2000185031 Jul 2000 JP
2000194972 Jul 2000 JP
2000237167 Sep 2000 JP
2000242788 Sep 2000 JP
2000259817 Sep 2000 JP
2000356059 Dec 2000 JP
2000357232 Dec 2000 JP
2001005948 Jan 2001 JP
2001067399 Mar 2001 JP
2001101429 Apr 2001 JP
2001167275 Jun 2001 JP
2001222661 Aug 2001 JP
2001292981 Oct 2001 JP
2001297177 Oct 2001 JP
2001358987 Dec 2001 JP
2002119477 Apr 2002 JP
2002133415 May 2002 JP
2002153444 May 2002 JP
2002153445 May 2002 JP
2002260071 Sep 2002 JP
2002271689 Sep 2002 JP
2002286650 Oct 2002 JP
2002312772 Oct 2002 JP
2002329204 Nov 2002 JP
2003006628 Jan 2003 JP
2003036434 Feb 2003 JP
2003108720 Apr 2003 JP
2003108983 Apr 2003 JP
2003132355 May 2003 JP
2003150942 May 2003 JP
2003153880 May 2003 JP
2003242125 Aug 2003 JP
2003271565 Sep 2003 JP
2003271940 Sep 2003 JP
2003308522 Oct 2003 JP
2003308523 Oct 2003 JP
2003317102 Nov 2003 JP
2003331265 Nov 2003 JP
2004005167 Jan 2004 JP
2004021406 Jan 2004 JP
2004030334 Jan 2004 JP
2004038305 Feb 2004 JP
2004094575 Mar 2004 JP
2004152046 May 2004 JP
2004163356 Jun 2004 JP
2004164483 Jun 2004 JP
2004171350 Jun 2004 JP
2004171602 Jun 2004 JP
2004206444 Jul 2004 JP
2004220376 Aug 2004 JP
2004261515 Sep 2004 JP
2004280221 Oct 2004 JP
2004280547 Oct 2004 JP
2004287621 Oct 2004 JP
2004315127 Nov 2004 JP
2004318248 Nov 2004 JP
2005004524 Jan 2005 JP
2005011207 Jan 2005 JP
2005025577 Jan 2005 JP
2005038257 Feb 2005 JP
2005062990 Mar 2005 JP
2005115961 Apr 2005 JP
2005148883 Jun 2005 JP
2005242677 Sep 2005 JP
WO 9717674 May 1997 WO
WO 9721188 Jun 1997 WO
WO 9802083 Jan 1998 WO
WO 9808439 Mar 1998 WO
WO 9932317 Jul 1999 WO
WO 9952422 Oct 1999 WO
WO 9965175 Dec 1999 WO
WO 0028484 May 2000 WO
WO 0029986 May 2000 WO
WO 0031677 Jun 2000 WO
WO 0036605 Jun 2000 WO
WO 0062239 Oct 2000 WO
WO 0101329 Jan 2001 WO
WO 0103100 Jan 2001 WO
WO 0128476 Apr 2001 WO
WO 0135348 May 2001 WO
WO 0135349 May 2001 WO
WO 0140982 Jun 2001 WO
WO 0163994 Aug 2001 WO
WO 0169490 Sep 2001 WO
WO 0186599 Nov 2001 WO
WO 0201451 Jan 2002 WO
WO 0219030 Mar 2002 WO
WO 0235452 May 2002 WO
WO 0235480 May 2002 WO
WO 02091735 Nov 2002 WO
WO 02095657 Nov 2002 WO
03003910 Jan 2003 WO
WO 03002387 Jan 2003 WO
WO 03054777 Jul 2003 WO
WO 03077077 Sep 2003 WO
WO 2004029863 Apr 2004 WO
WO 2004042646 May 2004 WO
WO 2004055737 Jul 2004 WO
WO 2004089214 Oct 2004 WO
WO 2004097743 Nov 2004 WO
WO 2005008567 Jan 2005 WO
WO 2005013181 Feb 2005 WO
WO 2005024698 Mar 2005 WO
WO 2005024708 Mar 2005 WO
WO 2005024709 Mar 2005 WO
WO 2005029388 Mar 2005 WO
WO 2005062235 Jul 2005 WO
WO 2005069252 Jul 2005 WO
WO 2005093510 Oct 2005 WO
WO 2005093681 Oct 2005 WO
WO 2005096962 Oct 2005 WO
WO 2005098531 Oct 2005 WO
WO 2005104704 Nov 2005 WO
WO 2005109344 Nov 2005 WO
WO 2006012645 Feb 2006 WO
WO 2006023046 Mar 2006 WO
WO 2006051462 May 2006 WO
WO 2006063076 Jun 2006 WO
WO 2006081209 Aug 2006 WO
WO 2006081505 Aug 2006 WO
WO 2007101269 Sep 2007 WO
WO 2007101275 Sep 2007 WO
WO 2007101276 Sep 2007 WO
WO 2007103698 Sep 2007 WO
WO 2007103701 Sep 2007 WO
WO 2007103833 Sep 2007 WO
WO 2007103834 Sep 2007 WO
WO 2008016724 Feb 2008 WO
WO 2008019168 Feb 2008 WO
WO 2008019169 Feb 2008 WO
WO 2008021584 Feb 2008 WO
WO 2008031089 Mar 2008 WO
WO 2008040026 Apr 2008 WO
Related Publications (1)
Number Date Country
20070189582 A1 Aug 2007 US
Provisional Applications (2)
Number Date Country
60778770 Mar 2006 US
60647270 Jan 2005 US
Continuation in Parts (4)
Number Date Country
Parent 11275703 Jan 2006 US
Child 11672108 US
Parent 11672108 US
Child 11672108 US
Parent 11043366 Jan 2005 US
Child 11672108 US
Parent 11372854 Mar 2006 US
Child 11043366 US