The present invention generally relates to photography and, more particularly, to digital cameras that are equipped to determine whether the eyes of a photographed person are open or closed.
Photographs of persons with their eyes closed are not desirable. However, amateur photographers often take quick snapshots of groups of humans (for example, family photographs). Since it is natural for humans to blink their eyes, one or more persons in a photograph may have their eyes closed. Therefore, it is desirable to provide an improved digital camera that alerts the camera operator when one of the persons in a photograph have their eyes closed or performs some other corrective action that enables the photographer to take better pictures. Conventional camera are not equipped to provides such features. It is further envisioned that this camera may be used for personal identification applications based on iris recognition technology.
In accordance with the present invention, an improved digital camera is provided that detects pictures of human subjects whose eyes are not closed. The improved digital camera generally includes an eye detection activation element that is operable to initiate an eye detection mode; an image sensor that captures image data for a target subject; an image processor that converts the image data into a digital image form; and an eye detection component that evaluates the digital image form for the target subject and determines whether at least one eye of the target subject is closed.
Further areas of applicability of the present invention will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description and specific examples, while indicating the preferred embodiment of the invention, are intended for purposes of illustration only and are not intended to limit the scope of the invention.
The present invention will become more fully understood from the detailed description and the accompanying drawings, wherein:
Operation of the digital camera system 10 is further described in relation to
In accordance with the present invention, the camera operator may activate an eye detection feature prior to taking the photograph as shown at step 32. For instance, when the camera operator ascertains that the target subject has eyes, the camera operator activates the eye detection feature using the eye detection activation element. Conversely, if the target subject does not include eyes, the camera operator may deactivate the eye detection feature using the eye detection activation element. It is envisioned that the eye detection activation element may take a variety of forms, including (but not limited to) push buttons, radial knobs, a touch screen display, or a combination thereof. Alternatively, digital camera system may be configured so that the eye detection feature is performed each time a photograph is taken.
In an eye detection mode, the eye detection component 20 is operable to evaluate the digital image form at step 36 to determine whether the eyes of any faces in the target subject are open or closed. The eye detection component 20 may employ one or more suitable algorithms to evaluate the digital image form. An exemplary eye detection algorithm is further described in an article by Saad A. Sirohey and Azriel Rosenfeld, entitled Eye Detection In A Face Image Using Linear and Non-Linear Filters, 36 P
When the eyes of the target subject are deemed open, the eye detection component 20 moves the digital image form from the transient memory space to a permanent memory location as shown at step 40. In an alternative embodiment, the digital camera system may be employed in personal identification applications. To facilitate the identification process, the camera is preferably operable to perform iris recognition processing on the digital image form only when the eyes of the target subject are deemed open. Iris recognition may be implemented using one of a various well known algorithms.
On the other hand, when at least one eye of the target subject is deemed closed, the eye detection component 20 is further operable to perform at least one additional camera function based on this observation. In one embodiment, the eye detection component 20 cooperatively operates with the user interface to provide an alert to the camera operator as shown at step 42. It is readily understood that the alert may be audible, visual, or some other suitable form.
In addition to merely being alerted, the camera operator may be given the option to perform one or more corrective actions in relation to the recently taken photographs. For example, the camera operator may view the recently taken photograph using a display provided by the camera as shown at step 35. When viewing the photographs, the camera operator may determine if the photograph meets their needs, or if another photograph should be taken of the target subject. Furthermore, the camera operator may be given the option of deleting the corresponding digital image from the transient memory space (or to designate the digital image form for subsequent deletion). It is understood that the digital camera may be equipped with a view screen or other display device (e.g., an LCD display) for viewing the most recently taken photograph(s).
In one exemplary embodiment, the camera may be configured to support a single picture review mode and/or a multiple picture review mode. In a single picture review mode, the camera operator is prompted to individually review each photograph immediately after it has been taken. The camera operator is then given the option to retain or discard the photograph as shown at step 44. If the camera operator designates the photograph as being acceptable, then the corresponding digital image form is moved from its transient memory space to a permanent data store as shown at 40. Conversely, if the camera operator designates the photograph as being unacceptable, the corresponding digital image form is deleted from the transient memory space as shown at 46. In either case, it is understood that the camera provides suitable user input elements to ascertain the operators selection.
In a multiple picture review mode, the camera operator initially defers reviewing recently taken photographs. Thus, the camera operator may shoot a sequence of photographs. At the time the photographs are taken, the camera operator is provided an alert for each photograph that is deemed to have one or more closed eyes therein. At some later time, the camera operator is given the option to take some corrective action with respect to each of the deficient photographs. For instance, each recently captured photograph is individually displayed to the camera operator. Alternatively, only the defective photographs may be displayed to the camera operator. In either case, the camera operator is then given the option to retain or discard the displayed photograph. To facilitate review, the photographs may be sorted sequentially in time or, alternatively (at operators discretion), may be sorted according to the degree that the eyes are deemed to be closed. Once an option has been selected, the next photograph is displayed to the camera operator. In this way, the camera operator may select and save the preferred photographs from a series of photographs that may have been taken over a short period of time.
Alternatively, the camera may perform some corrective action without intervention by the camera operator. For instance, the camera may disregard photographs where the subject's eyes are deemed closed without prompting the operator. To decreasing processing time, each image in a sequence of images may not be evaluated by the eye detection component. If the subject is blinking, then a few milliseconds are generally needed to the subject's eyes to fully open. Therefore, when blinking is detected in an image, the camera may simple ignore the next few images in the sequence of images (e.g., 2-3 images depending on the speed of the camera). In other words, the camera does not apply the eye detection algorithm to this image data, thereby reducing processing time.
In another exemplary embodiment, the camera eliminates the subject's closed eyes from the photograph by digitally enhancing the captured image data as shown in
During the same sequence of captured images or at some subsequent time, the stored image data may be used to corrective deficient images as further described below. When the eyes of the target are deemed closed, the camera may apply a substitution process to digitally replace the closed eye image data with open eye image data taken from previously stored image data for the subject. To do so, the applicable image data is retrieved at step 62 from the database. Known blending techniques are then used at step 64 to incorporate portions of the image data having eyes of the subject into the most recently captured image data of the subject. In this way, the camera may enhance the captured image data.
In another aspect of the present invention, the digital camera system 10 may be further equipped with face detection component 22. The face detection component cooperatively operates with the eye detection component to evaluate the digital image form as shown in
Each partitioned data segment then serves as an input to the eye detection component. For each partitioned data segment, eye detection processing is performed at step 78 as described above. When one eye in the face contained in the partitioned data segment is deemed closed, the camera proceeds with an additional camera function at step 82. When the eyes in the face contained in the partitioned data segment are deemed open, then the next partitioned data segment is retrieved for processing by the eye detection component as shown at step 76. Once all of the partitioned data segments have been analyzed (and it is determined that the eyes in each of the faces contained in the target subject are open), the eye detection component 20 will move the digital image form to a permanent memory location as shown at step 86.
The description of the invention is merely exemplary in nature and, thus, variations that do not depart from the gist of the invention are intended to be within the scope of the invention. Such variations are not to be regarded as a departure from the spirit and scope of the invention.