Patent Application
20150373260
The present disclosure relates to an image pickup device, a method of controlling an image pickup device, and a computer program.
There has been widely used a digital camera that performs digital coding on an image captured by an image sensor including a solid-state image pickup element such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), instead of a silver halide camera for taking a photograph with the use of a film or a photographic plate. According to a digital camera, it is possible to store an image that has been subjected to digital coding in a memory, thereby performing image processing and image management with the use of a computer, and further there is no problem in terms of the life of a film.
By using a digital camera that electronically captures a photograph, a user who is not accustomed to operation can easily take a clear photograph. However, it is difficult for a beginner to instantly learn a skill regarding at which timing a shutter release button should be pushed to take a photograph having a good composition.
In view of this, there is devised a technology for allowing a beginner who is not accustomed to handling of a digital camera to easily take a photograph having a good composition, and literatures disclosing such a technology are published.
For example, Patent Literature 1 discloses a technology in which a digital camera detects a composition to automatically execute image pickup processing and, when a composition specified by a user is detected, the detection is notified to the user. Patent Literature 2 discloses a technology in which an image inputted from an optical finder is displayed on a screen of a digital camera in order to prevent a failure of photographing caused by a time lag after a shutter release button is pushed but before operation for exposing light to an image pickup element is started.
Patent Literature 1: JP 2011-139498A
Patent Literature 2: JP 2012-99984A
However, in a case where a composition is detected to execute automatic image pickup, in a usual digital camera, a time lag from the detection of the composition to start of image pickup operation exists, and therefore a composition to be captured and an image actually captured are different. In particular, this phenomenon is remarkable in a case where an object moving at a high speed is captured. Even in a case where the technology disclosed in Patent Literature 2 is used, a timing of operation of a shutter release button ultimately depends on a user, and therefore the time lag from the detection of the composition to the start of the image pickup operation is not solved.
In view of this, the present disclosure provides an image pickup device, a method of controlling the image pickup device, and a computer program, each of which is novel and improved and is capable of executing detection of a composition and automatic image pickup processing in consideration of a time lag from the detection of the composition to start of image pickup operation.
According to the present disclosure, there is provided an image pickup device including: a composition detection unit configured to calculate a time at which an object that is specified by a user and is included in a captured image achieves a composition specified by the user; a time calculation unit configured to calculate a time after start instruction of image pickup operation is issued but before an image is captured; and an image pickup control unit configured to start image pickup processing of the image in response to the start instruction of the image pickup operation. The composition detection unit issues the start instruction of the image pickup operation to the image pickup control unit the time calculated by the time calculation unit before the time at which the composition specified by the user is achieved.
According to the present disclosure, there is provided a method of controlling an image pickup device, including: calculating a time at which an object that is specified by a user and is included in a captured image achieves a composition specified by the user; calculating a time after start instruction of image pickup operation is issued but before an image is captured; and starting image pickup processing of the image in response to the start instruction of the image pickup operation. The start instruction of the image pickup operation is issued the calculated time before the time at which the composition specified by the user is achieved.
According to the present disclosure, there is provided a computer program that causes a computer to: calculate a time at which an object that is specified by a user and is included in a captured image achieves a composition specified by the user; calculate a time after start instruction of image pickup operation is issued but before an image is captured; and start image pickup processing of the image in response to the start instruction of the image pickup operation. The start instruction of the image pickup operation is issued the calculated time before the time at which the composition specified by the user is achieved.
As described above, according to the present disclosure, it is possible to provide an image pickup device, a method of controlling the image pickup device, and a computer program, each of which is novel and improved and is capable of executing detection of a composition and automatic image pickup processing in consideration of a time lag from the detection of the composition to start of image pickup operation.
Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the appended drawings. Note that, in this specification and the drawings, elements that have substantially the same function and structure are denoted with the same reference signs, and repeated explanation is omitted.
Note that description will be provided in the following order.
<1. One embodiment of present disclosure>
[External appearance example of image pickup device]
[Functional configuration example of image pickup device]
[Operation example of image pickup device]
<2. Conclusion>
With reference to the drawings, there will be described an external appearance example of an image pickup device according to an embodiment of the present disclosure as an example of an information processing device of the present disclosure.
As illustrated in
The display unit 120 displays an image captured by the image pickup device 100 or displays various kinds of setting screens of the image pickup device 100. As described below, a touch panel is provided in the display unit 120, and a user who uses the image pickup device 100 can operate the image pickup device 100 by touching the touch panel provided in the display unit 120 with an operation member such as a finger.
The operation unit 130 causes a user to operate the image pickup device 100 and includes a button, a switch, and the like for operating the image pickup device 100.
It is needless to say that the external appearance of the image pickup device 100 is not limited to the above example. It is also needless to say that the button and the switch constituting the operation unit 130 are not limited to those illustrated in
The image pickup device 100 according to the embodiment of the present disclosure automatically starts image pickup operation when an object specified by a user reaches a position specified by the user. At the time of the automatic image pickup operation, the image pickup device 100 according to the embodiment of the present disclosure considers a time lag from detection of a composition to start of the image pickup operation. By considering the time lag from the detection of the composition to the start of the image pickup operation, the image pickup device 100 according to the embodiment of the present disclosure can achieve image pickup in the composition intended by the user.
It is needless to say that the external appearance of the image pickup device 100 according to the embodiment of the present disclosure illustrated in
In the above description, with reference to
As illustrated in
The image pickup unit 102 includes, for example, an imager including a solid-state image pickup element such as a lens, a charge coupled device (CCD), or a complementary metal oxide semiconductor (CMOS), a timing generator that controls, for example, an exposure timing to an image sensor, a sample and hold circuit, and an interface unit that provides original data of an image obtained by exposure to the imager to the following circuit. Note that, in this embodiment, there has been described a configuration in which the image pickup unit 102 is provided in the image pickup device 100. However, the present disclosure is not limited to the above example, and the lens may be detachable from the image pickup device 100.
The control unit 110 controls operation of the image pickup device 100. The control unit 110 may control the operation of the image pickup device 100 by, for example, reading computer programs recorded in the flash memory 140 and sequentially executing the computer programs. A specific configuration example of the control unit 110 will be described below.
As described above, the display unit 120 displays an image captured by the image pickup device 100 with the use of the image pickup unit 102 and displays various kinds of setting screens of the image pickup device 100. As illustrated in
As described above, the operation unit 130 causes a user to operate the image pickup device 100 and includes the button, the switch, and the like for operating the image pickup device 100. The control unit 110 executes various kinds of processing in response to an operation state of the operation unit 130. Examples of the various kinds of processing executed by the control unit 110 in response to the operation state of the operation unit 130 include processing of turning on/off the power source of the image pickup device 100, processing of changing magnification at the time of image pickup, processing of changing other image pickup conditions, and processing of capturing a still image or a moving image.
The flash memory 140 is a nonvolatile memory in which various kinds of computer programs needed to perform the processing of the control unit 110 and various kinds of data are stored. The RAM 150 is a working memory used at the time of the processing of the control unit 110.
Note that the control unit 110, the display unit 120, the operation unit 130, the flash memory 140, and the RAM 150 are connected to one another via a bus 160 and can communicate with one another.
In the above description, with reference to
As illustrated in
The composition detection unit 111 detects whether or not an image captured by the image pickup unit 102 has a composition intended by a user. In this embodiment, the composition detection unit 111 detects whether or not the image captured by the image pickup unit 102 has the composition intended by the user by detecting whether or not a feature point of an object specified by the user is positioned in a location specified by the user on the display unit 120. In a case where the feature point of the object specified by the user exits in the image captured by the image pickup unit 102, the composition detection unit 111 calculates a time it takes for the feature point to reach the location specified by the user on the display unit 120.
In a case where the composition detection unit 111 detects that the feature point of the object specified by the user exists in the location specified by the user on the display unit 120, the composition detection unit 111 issues a trigger for automatically starting the image pickup operation to the image pickup control unit 113 (described below).
However, after the composition detection unit 111 issues the trigger to the image pickup control unit 113 but before the image pickup control unit 113 starts image pickup processing, there exists a time lag caused by a hardware factor and a software factor of the image pickup device 100 and an environment at the time of image pickup. Therefore, in a case where the composition detection unit 111 issues the trigger to the image pickup control unit 113 when the feature point of the object specified by the user exists in the location specified by the user on the display unit 120, it is not possible to obtain an image having the composition intended by the user if the object moves.
In view of this, in this embodiment, the time lag calculation unit 112 (described below) calculates a time after the composition detection unit 111 issues the trigger to the image pickup control unit 113 but before the image pickup control unit 113 starts the image pickup processing. Then, the composition detection unit 111 issues the trigger for automatically starting the image pickup operation to the image pickup control unit 113 in consideration of the time calculated by the time lag calculation unit 112.
As described above, the image pickup device 100 according to this embodiment can obtain an image having the composition intended by the user by considering the time after the composition detection unit 111 issues the trigger to the image pickup control unit 113 but before the image pickup control unit 113 starts the image pickup processing.
As described above, the time lag calculation unit 112 calculates the time (time lag) after the composition detection unit 111 issues the trigger to the image pickup control unit 113 but before the image pickup control unit 113 starts the image pickup processing. As described above, the time lag is caused by the hardware factor and the software factor of the image pickup device 100 and the environment at the time of the image pickup.
Examples of the factors causing the time lag include inherent processing speed of the image pickup device 100, driving speed of a shutter curtain and an aperture of the image pickup device 100, inherent driving speed of a shutter curtain and an aperture of the lens, processing speed that is dynamically changed depending on a control state of the image pickup device 100, and processing speed that is dynamically changed depending on a control state of the lens. The time lag calculation unit 112 calculates the time (time lag) after the composition detection unit 111 issues the trigger to the image pickup control unit 113 but before the image pickup control unit 113 starts the image pickup processing, the time being caused by the above factors causing the time lag, and notifies the calculated time to the composition detection unit 111. The composition detection unit 111 issues the trigger to the image pickup control unit 113 in consideration of the time calculated by the time lag calculation unit 112.
The image pickup control unit 113 executes the image pickup processing in response to push of the shutter button 132 by a user or reception of the trigger for starting the image pickup operation from the composition detection unit 111. In response to the push of the shutter button 132 by the user or the reception of the trigger for starting the image pickup operation from the composition detection unit 111, the image pickup control unit 113 instructs the image pickup unit 102 to acquire a captured image. The captured image acquired in response to the instruction of the image pickup control unit 113 is stored in the flash memory 140 or is displayed in the display unit 120.
The composition specification unit 114 causes a user to specify an arbitrary composition. Information on the composition that the composition specification unit 114 causes the user to specify is stored in the flash memory 140. In a case were the composition specification unit 114 causes the user to specify the composition, the composition specification unit 114 causes the user to specify a target object (or a feature point of the object) and a position where the user wants the object to appear in an image.
In the above description, with reference to
First, the image pickup device 100 causes a user to specify an arbitrary composition (Step S101). This specification of the composition in Step S101 is executed by, for example, the composition specification unit 114. The image pickup device 100 causes the user to specify the arbitrary composition in Step S101 and then stores the specified composition (Step S102). Step S102 is executed by, for example, storing, in the flash memory 140, information on the composition that the composition specification unit 114 has caused the user to specify.
For example, the composition specification unit 114 causes a user to specify a feature point of a target object (Step S111).
As a matter of course, the feature points of the object to be specified by the user are not limited to the above examples. For example, the image pickup device 100 may prepare examples of feature points of objects in advance and cause a user to select one of the examples or may cause a user to specify a feature point of an arbitrary substance.
Because the touch panel 122 is provided in the image pickup device 100 in this embodiment, as illustrated in
The feature point of the target object is specified by the user, and then, for example, the composition specification unit 114 causes the user to specify a composition by causing the user to specify a position of the specified feature point (Step S112).
Because the touch panel 122 is provided in the image pickup device 100 in this embodiment, as illustrated in
The arbitrary composition is specified by the user as described above, and then, for example, the composition specification unit 114 stores the specified composition in the flash memory 140 (Step S102).
Referring back to
The feature point detection processing is executed in Step S103, and then the image pickup device 100 determines whether or not the feature point specified by the user is included in the image captured in the image pickup unit 102 (Step S104). This determination processing in Step S104 is executed by, for example, the composition detection unit 111.
As a result of the determination in Step S104, in a case where the feature point specified by the user is not included in the image captured in the image pickup unit 102, processing returns to Step S103, and the image pickup device 100 executes the feature point detection processing again. Meanwhile, as a result of the determination in Step S104, in a case where the feature point specified by the user is included in the image captured in the image pickup unit 102, the image pickup device 100 calculates a moving direction and a moving speed of the detected feature point (Step S105). Processing for calculating the moving direction and the moving speed of the detected feature point in Step S105 is executed by, for example, the composition detection unit 111.
The moving direction and the moving speed of the detected feature point are calculated in Step S105, and then, in a case where the detected feature point straightly moves, the image pickup device 100 calculates a time at which the detected feature point reaches the position specified by the user (Step S106). This calculation processing of the time in Step S106 is executed by, for example, the composition detection unit 111.
The time at which the feature point reaches the position specified by the user is calculated in Step S106, and then the image pickup device 100 calculates a time lag after a trigger for starting image pickup is issued but before image pickup processing is started, thereby determining a release correction time (Step S107). The release correction time is a time corresponding to a difference between the time at which the feature point reaches the position specified by the user and a time at which the trigger for starting the image pickup is actually issued. This calculation processing of the time lag in Step S107 is executed by, for example, the time lag calculation unit 112.
As described above, examples of the factors causing the time lag include the inherent processing speed of the image pickup device 100, the driving speed of the shutter curtain and the aperture of the image pickup device 100, the inherent driving speed of the shutter curtain and the aperture of the lens, the processing speed that is dynamically changed depending on the control state of the image pickup device 100, and the processing speed that is dynamically changed depending on the control state of the lens. The time lag from a state of the image pickup device 100 or the lens is influenced by a processing property of the image pickup device 100.
The time lag calculation unit 112 calculates, in Step S107, the time after the composition detection unit 111 issues the trigger to the image pickup control unit 113 but before the image pickup control unit 113 starts the image pickup processing, the time being caused by the above factors, and notifies the calculated time to the composition detection unit 111.
Note that the time lag calculation unit 112 may obtain the time lag by calculation in a case where the feature point of the object is detected, but, in a case where the processing property of the image pickup device 100 is not high, the time lag calculation unit 112 may refer to a fixed value held in advance on the basis of a setting state of the image pickup device 100.
The release correction time is determined in Step S107, and then the image pickup device 100 determines whether or not a timing reaches the release timing on the basis of the time calculated in Step S106 and the release correction time determined in Step S107 (Step S108). This determination on whether or not a timing reaches the release timing in Step S108 is executed by, for example, the composition detection unit 111.
For example, in a case where the time T1 at which the feature point reaches the position specified by the user is calculated in Step S106 and the time lag (release correction time) after the trigger for starting the image pickup is issued but before the image pickup operation is started is calculated to be 20 milliseconds in Step S107, the image pickup device 100 issues the trigger for starting the image pickup 20 milliseconds before the time T1 calculated in Step S106, and therefore it is possible to capture an image in which the feature point exists in the position specified by the user.
As a result of the determination in Step S108, in a case where it is determined that a timing does not reach the release timing, the image pickup device 100 executes the feature point detection processing in Step S103 again. Meanwhile, as a result of the determination in Step S108, in a case where it is determined that a timing reaches the release timing, the image pickup device 100 performs release by issuing the trigger for starting the image pickup (Step S109). The trigger for starting the image pickup is issued to the image pickup control unit 113 by the composition detection unit 111.
As described above, the image pickup device 100 according to the embodiment of the present disclosure executes the automatic image pickup processing in consideration of the time lag after the trigger for starting the image pickup is issued but before the image pickup operation is started. As described above, by executing the automatic image pickup processing in consideration of the time lag, the image pickup device 100 according to the embodiment of the present disclosure can capture the image having the composition specified by the user as the user intended.
Note that the image pickup device 100 may display the image captured by the release in display unit 120 after the release is executed in Step S109, and, in a case where the image is displayed, the image pickup device 100 may overlap, on the image, the release correction time and a position of the object obtained when the release timing has not been corrected. In the above example, the image pickup device 100 may overlap the release correction time expressed as “20 msec” or the like on the image. The image pickup device 100 may express the position of the object obtained when the release timing has not been corrected by shifting the object through image processing, or may express the position with the use of an arbitrary mark or the like instead of the object.
As described above, because the release correction time and the position of the object obtained when the release timing has not been corrected are presented by overlapping the release correction time and the position on the image, improvement in a user's skill of using the image pickup device 100 can be expected in terms of the release timing.
In the above description, in a case where the object straightly moves in the image, the image pickup device 100 calculates the time T1 at which the feature point reaches the position specified by the user and the time lag after the trigger for starting the image pickup is issued but before the image pickup operation is started. However, the object to be captured by the user with the use of the image pickup device 100 does not necessarily move straightly in the image and may curvedly move in the image. In a case where the object to be captured by the user with the use of the image pickup device 100 curvedly moves, it is considered that, even in a case where the time lag is calculated as described above and the image pickup device 100 determines that a timing reaches the release timing, the object does not actually exist in the position specified by the user.
Therefore, in a case where the image pickup device 100 causes the user to specify the composition, a wider range may be specified instead of specification of one position, i.e., so-called specification of a pinpoint position on the screen. The image pickup device 100 specify, in advance, not only the pinpoint position but also a range in which the object is assumed to straightly move, and, when it is detected that the object enters the range, as described above, the image pickup device 100 may calculate the time T1 at which the feature point reaches the position specified by the user and the time lag after the trigger for starting the image pickup is issued but before the image pickup operation is started.
The image pickup device 100 may automatically determine whether to specify the pinpoint position or specify the wider range on the basis of motion of the feature point. For example, in a case where the feature point straightly moves within a range of a certain time, the image pickup device 100 may execute the automatic image pickup processing while a pinpoint position is specified.
In the above description, the image pickup device 100 executes the automatic image pickup processing of the still image in consideration of the time lag after the trigger for starting the image pickup is issued but before the image pickup operation is started. However, the present disclosure is not limited to the above example.
For example, the image pickup device 100 may start image pickup operation of a moving image when an object reaches the vicinity of a location specified by a user and terminate the image pickup operation of the moving image when the object departs from the vicinity of the location specified by the user. The image pickup device 100 may extract a frame in which the object exists in the location specified by the user from the moving image captured as described above and record the frame as a still image.
As described above, according to the embodiment of the present disclosure, it is possible to provide the image pickup device 100 that executes automatic image pickup processing of a still image in consideration of a time lag after a trigger for starting image pickup is issued but before image pickup operation is started. By executing the automatic image pickup processing of the still image in consideration of the time lag, even in a case where an object moves at a high speed, the image pickup device 100 can capture a still image having a composition intended by a user.
Steps in processes executed by devices in this specification are not necessarily executed chronologically in the order described in a sequence chart or a flow chart. For example, Steps in processes executed by devices may be executed in a different order from the order described in a flow chart or may be executed in parallel.
Further, a computer program can be created which causes hardware such as a CPU, ROM, or RAM, incorporated in each of the devices, to function in a manner similar to that of structures in the above-described devices. Furthermore, it is possible to provide a recording medium having the computer program recorded thereon. Moreover, by configuring respective functional blocks shown in a functional block diagram as hardware, the hardware can achieve a series of processes.
In the embodiment of the present disclosure described above, the description has been provided by exemplifying a digital still camera. However, a device to which the present disclosure is applied is not limited to the above example. It is needless to say that, for example, the technology in the present disclosure can be similarly applied to a mobile phone, a game console, a personal computer, a tablet terminal, and other information processing devices, in each of which a camera is mounted.
The preferred embodiments of the present disclosure have been described above with reference to the accompanying drawings, whilst the present disclosure is not limited to the above examples, of course. A person skilled in the art may find various alterations and modifications within the scope of the appended claims, and it should be understood that they will naturally come under the technical scope of the present disclosure.
Additionally, the present technology may also be configured as below.
(1)
An image pickup device including:
a composition detection unit configured to calculate a time at which an object that is specified by a user and is included in a captured image achieves a composition specified by the user;
a time calculation unit configured to calculate a time after start instruction of image pickup operation is issued but before an image is captured; and
an image pickup control unit configured to start image pickup processing of the image in response to the start instruction of the image pickup operation,
wherein the composition detection unit issues the start instruction of the image pickup operation to the image pickup control unit the time calculated by the time calculation unit before the time at which the composition specified by the user is achieved.
(2)
The image pickup device according to (1),
wherein the time calculation unit calculates the time in consideration of a processing time that dynamically changes in accordance with a control state of the image pickup device.
(3)
The image pickup device according to (1) or (2),
wherein the image pickup control unit overlaps information on the time calculated by the time calculation unit on the image obtained by the image pickup operation executed based on the instruction from the composition detection unit.
(4)
The image pickup device according to any one of (1) to (3),
wherein the image pickup control unit overlaps information on a position of the object obtained when the image pickup operation is executed at the time calculated by the composition detection unit on the image obtained by the image pickup operation executed based on the instruction from the composition detection unit.
(5)
The image pickup device according to (4),
wherein the image pickup control unit extracts the object and uses the object as the information on the position of the object.
(6)
The image pickup device according to any one of (1) to (5),
wherein the composition detection unit issues the start instruction of the image pickup operation to the image pickup control unit as long as the object is included in the captured image.
(7)
The image pickup device according to any one of (1) to (6),
wherein the composition detection unit changes, in accordance with a specification method of the composition by the user, a timing to start calculation of the time at which the composition specified by the user is achieved.
(8)
The image pickup device according to (7),
wherein the specification method of the composition by the user is changed depending on whether the object straightly moves or curvedly moves.
(9)
A method of controlling an image pickup device, including:
a composition detection step of calculating a time at which an object that is specified by a user and is included in a captured image achieves a composition specified by the user;
a time calculation step of calculating a time after start instruction of image pickup operation is issued but before an image is captured; and
an image pickup control step of starting image pickup processing of the image in response to the start instruction of the image pickup operation,
wherein in the composition detection step, the start instruction of the image pickup operation is issued in the image pickup control step the time calculated in the time calculation step before the time at which the composition specified by the user is achieved.
(10)
A computer program that causes a computer to perform:
a composition detection step of calculating a time at which an object that is specified by a user and is included in a captured image achieves a composition specified by the user;
a time calculation step of calculating a time after start instruction of image pickup operation is issued but before an image is captured; and
an image pickup control step of starting image pickup processing of the image in response to the start instruction of the image pickup operation,
wherein in the composition detection step, the start instruction of the image pickup operation is issued in the image pickup control step the time calculated in the time calculation step before the time at which the composition specified by the user is achieved.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2013-022247 | Feb 2013 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2014/051219 | 1/22/2014 | WO | 00 |
