Patent Application
20120057039
1. Field
The invention relates generally to the field of photography and, more particularly, to an automated process for capturing an image that includes a particular person, such as the photographer.
2. Background
It is a common desire of photographers to be included in scenes that they photograph. A frequently used solution is to enlist the aid of another individual to take the photograph. This, of course, is not always a feasible or convenient solution. Most cameras are equipped with self-timers that delay the shutter release for some period of time so that the photographer can position the camera on a stable surface and then move into the camera's field of view. Additionally, some cameras are equipped with a remote control so that the photographer can trigger the shutter release after taking a desired position in front of the camera. Another common solution that is frequently used with small handheld cameras is for the photographer to hold the camera in an outstretched arm with the camera lens pointed in the direction of the photographer. This approach is often unsuccessful since the camera's viewfinder is typically not visible from the front of the camera and numerous attempts may be required before an image is captured without the photographer's face being cropped out of the scene.
An embodiment of the present invention provides a camera with an automated image capture mode for insuring that a particular subject appears in the captured image. An image of the particular subject, which may be the photographer, is initially captured. Subsequently, another image capture is automatically triggered when the same person is detected within the camera's field of view.
In another embodiment of the invention, a motion sensor within the camera is employed to begin a search for the person whose image was previously stored, when the camera is subjected to a sudden movement.
The embodiments of the invention are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings in which like references indicate similar elements. It should be noted that references to “an” or “one” embodiment of the invention in this disclosure are not necessarily to the same embodiment, and they mean at least one.
Throughout the description, for the purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced without some of these specific details. In other instances, well-known structures and devices are shown in block diagram form to avoid obscuring the underlying principles of embodiments of the present invention.
A combined circuit schematic and block diagram of some relevant constituent electronic components of a portable camera device 10 (also applicable to camera device 20) is presented in
A camera controller 18 (e.g., implemented as a combination of programmed data processing components and hardwired logic circuitry) manages the process of capturing images. It does so by generating various commands that are signaled to the components of the camera device 10, in response to a user of the device 10 actuating a shutter release button 19, to take an exposure. The button 19 may be a physical button on an outside surface of the housing of the device 10, or it may be a virtual button displayed on a touch screen of the device 10. The camera may also include a digital viewfinder function that is also conducted by the controller 18. The digital viewfinder function lets the user see a lower resolution version of the just-captured image on the display screen 12. The digital viewfinder function may also produce live video of the scene using a selected camera of the device 10; the user can thus preview the scene before actuating the shutter release button to take a picture.
In the case of a camera device 10 having both a front facing camera and a rear facing camera, the controller 18 may be further coupled to a second, different image sensor array via separate reset and readout signal lines. The second array would also be coupled to share the image storage 17 and display screen 12. A different imaging lens would also be provided for gathering light onto the second sensor array, where the latter combination could be aimed at a substantially opposite direction as the array 15 and imaging lens 16.
A process for implementing an embodiment of the present invention is shown in
Once an acceptable image of the subject's face has been captured, the automated picture taking process continues with monitoring image data from a camera, which may be a rear-facing camera, to determine if the face in the previously captured image appears in the camera's field of view (operation 36). For instance, a rapid sequence of images captured by the camera while the camera is aimed at a desired scene, are analyzed to detect when the subject has moved into the camera's field of view. This may be accomplished using suitable facial recognition software. Facial recognition algorithms with accuracy sufficient for the purposes of this invention are widely available. For example, facial recognition is included in the iPhoto™ application by Apple Inc., which uses facial detection to identify faces of people in photographs and facial recognition to match faces that look like the same person. When the subject has been recognized in the camera's field of view, a countdown may be initiated at 38 to give the photographer (camera user) and/or the subject additional time to compose the scene before a picture of the scene is taken. This may be implemented using a variable timer that may be set in advance by the photographer. The photographer and/or the subject may be alerted to the countdown with a sequence of beeps or light flashes. At the end of the countdown, the image is captured at 40, i.e. the picture is taken.
An image capture process in accordance with another embodiment of the invention is shown in
In accordance with another embodiment of the invention, a portable electronic device has a camera function implemented by a camera controller, to conduct a process for taking a picture of a scene. Once that process is initiated (e.g., through user input via a menu selection), the controller causes the capture of an image of a subject and then analyzes the image to detect facial features of the subject therein. Next, after the presence of facial features has been confirmed, the controller causes the capture of several images of the scene (e.g., a continuous shooting sequence), and then analyzes these scene images using facial recognition to detect the subject's face therein. In particular, the previously captured image of the subject may be analyzed to extract certain facial features from it, and these are then compared using a facial recognition algorithm to facial features extracted from the scene images, until a match is detected. This means that the subject has now moved into the scene. In response, the controller causes a subsequent capture of an image of the scene, which is then stored in the device as the desired picture.
The camera function may use a single image sensor array to which the controller is coupled, to cause the capture of all of the images during the picture taking process. Alternatively, multiple sensor arrays may be used, e.g. a first image sensor array to to capture the subject's image, and a second, different image sensor array to capture the scene images and the subsequent scene image. In one embodiment, the first and second sensor arrays are aimed in substantially opposite directions.
The camera controller may be programmed to signal an alert to a user of the device in response to having detected facial features of the subject in the subject's image. This alert may be text on a display screen of the device, indicating to the user that the subject should now move into the scene for the picture taking process to continue.
The camera controller may also be programmed to begin a countdown in response to having detected the subject's face. Upon finishing the countdown, it signals the subsequent capture of an image of the scene, where at this point the scene has most likely been finalized and the subject is likely present in the scene.
In one embodiment, the controller can conduct the entire picture taking process without any user input, once the process has been initiated. In other embodiments, the controller can prompt the user before taking action. For instance, the user may be prompted to accept the detected facial features of the subject, before the controller signals the capture of the sequence of scene images. In yet another embodiment, once the controller signals that the scene images are to be capture, it automatically begins to analyze these scene images as they are being produced (without user input being required) and then automatically begins the countdown once the subject's face has been detected.
In yet another embodiment, the device has a motion sensor to which the controller is coupled to sense movement of the device. The controller is to analyze the scene images using facial recognition, to detect the subject's face therein, upon sensing movement of the device (e.g., the device being flipped or rotated.)
It will be apparent from this description that aspects of the present invention may be embodied, at least in part, in software, hardware, firmware, or in combination thereof. That is, the techniques may be carried out in a computer system or other data processing system in response to its processor, such as a microprocessor, executing sequences of instructions contained in a memory, such as ROM, volatile RAM, non-volatile memory, cache, or a remote storage device. In various embodiments, hardwired circuitry may be used in combination with software instructions to implement the present invention. Thus, the techniques are not limited to any specific combination of hardware circuitry and software or to any particular source for the instructions executed by the data processing system. In addition, throughout this description, various functions and operations are described as being performed by or caused by software code to simplify description. However, those skilled in the art will recognize that what is meant by such expressions is that the functions result from execution of code by a processor, such as a microprocessor.
A machine readable medium can be used to store software and data which when executed by the data processing system causes the system to perform various methods of the present invention. This executable software and data may be stored in various places including for example ROM, volatile RAM, non-volatile memory, and/or cache. Portions of this software and/or data may be stored in any one of these storage devices.
The invention also relates to apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored or transmitted in a machine-readable medium. A machine readable medium includes any mechanism that provides (i.e., stores and/or transmits) information in a form accessible by a machine (e.g., a computer, network device, personal digital assistant, manufacturing tool, any device with a set of one or more processors, etc.). For example, a machine readable medium includes recordable/non-recordable media such as, but not limited to, a machine-readable storage medium (e.g., any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, flash memory, magnetic or optical cards, or any type of media suitable for storing electronic instructions), or a machine-readable transmission medium such as, but not limited to, any type of electrical, optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.).
Additionally, it will be understood that the various embodiments described herein may be implemented with a variety of data processing systems. For example, such data processing systems may be a cellular telephone or a personal digital assistant (PDA) or an entertainment system or a media player (e.g., an iPod) or a consumer electronic device, etc., each of which can be used to implement one or more of the embodiments of the invention.
Throughout the foregoing specification, references to “one embodiment,” “an embodiment,” “an example embodiment,” etc., indicate that the embodiment described may include a particular feature, structure, or characteristic, but every embodiment may not necessarily include the particular feature, structure, or characteristic. Moreover, such phrases are not necessarily referring to the same embodiment. When a particular feature, structure, or characteristic is described in connection with an embodiment, it is submitted that it is within the knowledge of one skilled in the art to bring about such a feature, structure, or characteristic in connection with other embodiments whether or not explicitly described. Various changes may be made in the structure and embodiments shown herein without departing from the principles of the invention. Further, features of the embodiments shown in various figures may be employed in combination with embodiments shown in other figures.
In the description as set forth above and claims, the terms “coupled” and “connected,” along with their derivatives, may be used. It should be understood that these terms are not intended to be synonymous with each other. Rather, in particular embodiments, “connected” is used to indicate that two or more elements are in direct physical or electrical contact with each other. “Coupled” may mean that two or more elements are in direct physical or electrical contact. However, “coupled” may also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other.
Some portions of the detailed description as set forth above are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of operations leading to a desired result. The operations are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.
It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the discussion as set forth above, it is appreciated that throughout the description, discussions utilizing terms such as “processing” or “computing” or “calculating” or “determining” or “displaying” or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system's registers and memories into other data similarly represented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.
The invention also relates to apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored or transmitted in a machine-readable medium, such as, but is not limited to, a machine-readable storage medium (e.g., any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, flash memory, magnetic or optical cards, or any type of media suitable for storing electronic instructions), or a machine-readable transmission medium such as, but not limited to, any type of electrical, optical, acoustical or other form of propagated signals (e.g., carrier waves, infrared signals, digital signals, etc.).
The algorithms and displays presented herein are not inherently related to any particular computer system or other apparatus. Various general purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatuses to perform the method operations. The structure for a variety of these systems appears from the description above. In addition, the invention is not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the invention as described herein.
Embodiments of the invention may include various operations as set forth above or fewer operations or more operations or operations in an order that is different from the order described herein. The operations may be embodied in machine-executable instructions that cause a general-purpose or special-purpose processor to perform certain operations. Alternatively, these operations may be performed by specific hardware components that contain hardwired logic for performing the operations, or by any combination of programmed computer components and custom hardware components.
Throughout the foregoing description, for the purposes of explanation, numerous specific details were set forth in order to provide a thorough understanding of the invention. It will be apparent, however, to one skilled in the art that the invention may be practiced without some of these specific details. Accordingly, the scope and spirit of the invention should be judged in terms of the claims that follow as well as the legal equivalents thereof.
