Patent Grant
10712636
Field of the Invention
The present invention relates to a polarization acquisition apparatus used to acquire polarization information from captured images.
Description of the Related Art
Rotating a polarizing filter provided on an image capturing optical path enables controlling an amount of light reflected by an object. For example, when image capturing of an underwater object is performed, removing a reflected light from a water surface by the polarizing filter enables providing an image in which the object is clearly image-captured. Moreover, when image capturing of trees is performed, reducing a reflected light from leaves enables providing a colorful image.
Japanese Patent Laid-Open No. 2011-191469 discloses an intermediate lens barrel attached between a camera and an interchangeable lens and provided with a polarizing filter and a mechanism rotating the filter. Japanese Patent Laid-Open No. 2009-232347 discloses a camera that acquires multiple images while rotating a polarizing filter or inserting and removing the filter into and from an optical path to change a degree of a polarizing filter effect.
On the other hand, Japanese Patent Laid-Open No. 11-041514 discloses a camera capable of removing, by using polarization information acquired from a captured image, a specular reflection component from the captured image. Furthermore, Japanese Patent Laid-Open No. 2012-080065 discloses an image sensor provided with a wire-grid polarizer to make polarized lights whose polarization directions are mutually different enter multiple pixels, thereby being capable of extracting polarization information including mutually different items.
However, the polarizing filters disclosed in Japanese Patent Laid-Open Nos. 2011-191469 and 2009-232347 can only adjust their polarizing filter effects on one captured image uniformly, that is, cannot provide mutually different polarizing filter effects on mutually different areas of the captured image. In addition, image capturing using these polarizing filters makes it impossible to adjust their polarizing filter effects on the captured image after the image capturing.
Moreover, the cameras disclosed in Japanese Patent Laid-Open Nos. 11-041514 and 2012-080065 each need to be a camera dedicated for acquiring the polarization information and to use illumination light dedicated therefore, which restricts objects to be image-captured and image capturing conditions.
The present invention provides a polarization acquisition apparatus capable of making it possible to acquire polarization information of captured images after image capturing, without restricting objects to be image-captured and image capturing conditions, and provides an image capturing apparatus and system using the polarization acquisition apparatus, thereby being capable of producing another image than the captured image.
The present invention provides as an aspect thereof a polarization acquisition apparatus is configured to allow an image capturing system including an interchangeable lens and an image capturing apparatus to perform image capturing using polarized light. The polarization acquisition apparatus includes a variable polarization axis element in which a direction of its polarization transmission axis is variable, a driver configured to operate the variable polarization axis element so as to change the direction of the polarization transmission axis, a first mount to which the image capturing apparatus is detachably attachable, and a second mount to which the interchangeable lens is detachably attachable. The driver is configured to operate the variable polarization axis element so as to change the direction of the polarization transmission axis to at least three directions when the image capturing is performed multiple times.
The present invention provides as another aspect thereof an image capturing apparatus used with the above polarization acquisition apparatus and an image capturing system including the image capturing apparatus and the above polarization acquisition apparatus. The image capturing apparatus is configured to perform the image capturing multiple times while changing the direction of the polarization transmission axis to the at least three directions to produce multiple captured images.
Further features and aspects of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
It is desirable that the polarization acquisition apparatus 10 be provided with a mount to which at least one of the camera body 30 and the interchangeable lens 20 is detachably attachable. Providing such a mount enables a user to attach the polarization acquisition apparatus 10 to various types of camera bodies 30 and interchangeable lenses 20 as needed. That is, providing the mount enables acquiring polarization information from all captured images acquirable by image capturing using the various types of camera bodies 30 and interchangeable lenses 20, with no restriction on objects to be image-captured and image capturing conditions. Moreover, using the acquired polarization information enables producing an image having a different texture from those of the captured images.
The polarization acquisition apparatus 10 is constituted by at least one variable polarization axis element, an element driver configured to change a state of the variable polarization axis element and an element controller configured to control the driver. The element driver and the element controller constitute a driver configured to operate the variable polarization axis element.
The variable polarization axis element has a polarization transmission axis (hereinafter simply referred to as “a transmission axis). The variable polarization axis element transmits a polarized light having a polarization direction (that is, a polarization axis) parallel to a direction of the transmission axis and inhibits transmission of polarized lights having other polarization directions.
The element driver changes the state of the variable polarization axis element to change the direction of the transmission axis (that is, of the polarization axis of the transmitted polarized light) about an optical axis of the interchangeable lens 20, in other words, in a plane orthogonal to the optical axis; the plane is orthogonal to a paper of
Although the variable polarization axis element may have any configuration, most simply, it may have a configuration using a polarizing plate that transmits a linearly polarized light whose polarization direction is parallel to a direction of a transmission axis of the polarizing plate. When such a polarizing plate is used, the driver rotates the polarizing plate about the optical axis of the interchangeable lens 20 to change the direction of the transmission axis of the polarizing plate.
A variable polarization axis element 11 may be used that has a configuration illustrated in
The variable polarization axis element 11 illustrated in
With these relations, a phase difference Δλ (nm) provided by the variable phase plate 2 and a direction θ of the transmission axis of the variable polarization axis element satisfy following relation:
θ=90°−Δ×180°
where Δ=δ/λ; δ represents a phase difference (nm), and λ represents a wavelength (nm).
For example, when the phase differences are 0λ, ¼λ, ½λ and ¾λ, the directions θ of the transmission axis are 90°, 45°, 0° and −45°, respectively. Thus, changing the phase difference provided by the variable phase plate 2, that is, changing the state of the variable polarization axis element enables changing the direction of the transmission axis of the variable polarization axis element.
Next, description will be made of a relation between the phase difference and the direction of the transmission axis, through an example in which the phase difference provided by the variable phase plate 2 is ¼λ (corresponding to when the direction of the transmission axis is 45°).
In
As illustrated in
Similarly, in the case of
These show that, when the phase difference provided by the variable phase plate 2 is ¼λ, the entering light is transmitted through the variable polarization axis element with an intensity equivalent to that when the entering light is transmitted through the polarizing plate 3 whose transmission axis has a direction of 45°. In other words, when the phase difference provided by the variable phase plate 2 is ¼ λ, the direction of the transmission axis of the variable polarization axis element can be regarded as a 45° direction.
Changing the phase difference provided by the variable phase plate 2 to 0, ½ λ and ¾λ changes the direction of the transmission axis of the variable polarization axis element to 90°, 0° and −45° (135°). In other words, the phase difference Δλ (nm) provided by the variable phase plate 2 and the direction θ (°) of the transmission axis of the variable polarization axis element satisfy the above-described relation of:
θ=90°−Δ×180°.
When the variable phase plate 2 is used in the variable polarization axis element, the element driver has, in order to change the direction of the transmission axis of the variable polarization axis element, a function of changing a state of an optical element included in the variable polarization axis element. For example, when a liquid crystal element is used as the variable phase plate 2, the element driver controls orientation directions of liquid crystal molecules by applying a voltage to change the phase difference, thereby enabling changing the direction of the transmission axis.
Any configuration may be employed to control the element driver. For example, the element driver may be controlled in response to a control signal from the camera body 30. It is necessary for this configuration to provide, to the polarization acquisition apparatus 10, a communicator configured to receive the control signal from the camera body 30 and an element controller configured to control the element driver in response to the control signal. The camera body 30 and the interchangeable lens 20 normally communicate control signals for controlling the interchangeable lens 20, information on a state of the interchangeable lens 20 and others with each other through electronic contacts provided in their mounts. Thus, it is only necessary to provide an electronic contact as the communicator to the polarization acquisition apparatus 10 so as to enable communication with the camera body 30 and the interchangeable lens 20. Alternatively, instead of providing the electronic contact, a sender/receiver for wireless LAN such as Wi-Fi™ may be provided as the communicator to each of the polarization acquisition apparatus 10 and the camera body 30 in order to enable communication therebetween.
As the control signal from the camera body 30, for example, a release signal as an image capturing start signal output in response to a user's operation (release operation) of an image capturing instruction button may be used. The camera body 30 may perform, in response to each reception of the release signal, image capturing and change the direction of the transmission axis of the variable polarization axis element or may perform, in response to reception of one release signal, image capturing multiple times while sequentially changing the direction of the transmission axis of the variable polarization axis element to predetermined multiple directions.
After each image capturing, it is desirable to send information on the direction of the transmission axis (that is, of the polarization axis of the transmitted polarized light) from the polarization acquisition apparatus 10 to the camera body 30 to cause the camera body 30 to store an acquired captured image and the information on the direction of the transmission axis in association with each other. Alternatively, the camera body 30 may store together multiple captured images acquired by performing image capturing multiple times while changing the direction of the transmission axis in association with the directions of the transmission axis.
The image capturing system of this embodiment performs image capturing at least three times with mutually different directions of the transmission axis and acquires polarization information from at least three acquired captured images (image data).
Next, description will be made of a principle of acquisition of the polarization information in this embodiment, through an example in which a reflected light from an object is expressed as illustrated in
An ellipse in
Moreover, arrows represent amplitudes in a long axis direction and a short axis direction. A square of the amplitude represented by the arrow in each of the long and short axis directions provides a light intensity of the polarized light (hereinafter also referred to as “polarization intensity”). The square of the amplitude in the long axis direction corresponds to “a” (maximum polarization intensity) in
This example shows a state where a polarized light that vibrates in a direction corresponding to α=30° has the maximum polarization intensity. The light intensity I(θ) satisfies following expression (1) for any θ.
I(θ)=(a−b)cos2(θ−α)+b (1)
This means that measuring the light intensities (that is, acquiring captured images) with at least three mutually different directions of the transmission axis of the variable polarization axis element enables calculating α, a and b.
For example,
On the other hand, as long as satisfying this condition, there is no restriction on the direction θ with which the image data is acquired. Therefore, it is not necessary to adjust the direction of the transmission axis (polarization direction) while confirming an image before image capturing, and thus image capturing for acquiring at least three image data can be automatically performed under a predetermined condition.
Although a typical polarized light includes information on its light intensity and amplitude, this embodiment acquires information on the direction α and the polarization intensities a and b as the polarization information. Although the direction α and the polarization intensities a and b in the polarization information may be separately stored, it is desirable that the direction α and the polarization intensities a and b be stored in association with the image data having been used to calculate these values.
As described above, using the polarization acquisition apparatus 10 of this embodiment enables acquiring the polarization information from the captured images after image capturing, with no restriction on the objects to be image-captured and the image capturing conditions.
In addition, using the acquired polarization information α, a and b enables producing an image (hereinafter referred to as “a texture changed image”) having a different texture from those of the captured images. The texture herein corresponds to a reflected light from an object to be image-captured. Description will be made of a relation between the polarization information and the reflected light from in producing the texture changed image.
A reflected light from an object (substance) is divided into a specular reflection component directly reflected on a surface of the object and a diffuse reflection component diffusely reflected inside the object or on the surface thereof. The specular reflection component reflects at the surface of the object under Fresnel reflection conditions. In Fresnel reflection, except under some conditions, an S-polarized light has a stronger intensity than that of a P-polarized light. Therefore, in the specular reflection component, a polarization intensity changes depending on directions, that is, has a directional dependency. On the other hand, the diffuse reflection component including lights reflected by the object in various directions has no directional dependency.
Thus, this embodiment considers, in the acquired polarization information, a component “a-b” that changes depending on the direction θ as the specular reflection component and considers a component “b” that does not change depending on the direction θ as the diffuse reflection component. This embodiment calculates the specular reflection component and the diffuse reflection component depending on a purpose to produce the texture changed image. This makes it possible to produce an image including a different reflected light (that is, a different texture) from those of the captured images, without using a dedicated illumination light.
Processes to calculate the polarization information α, a and b from the captured images and to produce the texture changed image by using the polarization information may be performed by an image processor provided in the camera body 30 or by an image processing apparatus, such as a personal computer, provided separately from the camera body 30.
Next, description will be made of a specific embodiment according to Embodiment 1, as a second embodiment (Embodiment 2).
The interchangeable lens 20 and the camera body 30 are provided with mounts 42 and 43 to make a detachable attachment of the interchangeable lens 20 to the camera body 30 possible. The interchangeable lens includes an image capturing lens 21 as an image capturing optical system, a lens driver 22 and a lens controller 23. The image capturing lens 21 images light (reflected light) from an object to form an object image. The lens driver 22 is configured to perform a focusing drive, a stop drive and others of the image capturing lens 21. The lens controller 23 is configured to control operations of the lens driver 22.
On the other hand, the camera body 30 includes an image sensor 35, a camera controller 31 and an information recorder 32. The image sensor 35 is a photoelectric conversion element, such as a CCD sensor or a CMOS sensor, configured to perform image capturing (photoelectric conversion) of the object image formed by the image capturing lens 21. The camera controller is configured to cause the image sensor 35 to perform the image capturing and configured to produce a captured image from an output signal from the image sensor 35. The information recorder 32 is configured to record the captured image (image data) produced by the camera controller 31.
The lens controller 23 and the camera controller 31 communicate lens information and control signals with each other through communication contacts 52 and 53 provided in the mounts 42 and 43.
The polarization acquisition apparatus 10 includes the variable polarization axis element 11 described in Embodiment 1, an element driver 12 and an element controller 13. The element driver 12 and the element controller 13 constitute a driver. The element driver 12 is configured to operate the variable polarization axis element 11 so as to change the direction of the transmission axis of the variable polarization axis element 11. The element controller 13 is configured to control operations of the element driver 12, that is, of the variable polarization axis element 11.
The element controller 13 communicates various information with the lens and camera controllers 23 and 31 through communication contacts 51a and 51b provided to the mounts 41a and 41b and the communication contacts 52 and 53 of the interchangeable lens 20 and the camera body 30. The element controller further relays the communication between the lens and camera controllers 23 and 31. As described above, the camera system can perform image capturing not using the polarization acquisition apparatus 10 as illustrated in
The variable polarization axis element 11 includes, as illustrated in
The element driver 12 is configured to control a voltage applied to the variable phase plate 2 to operate the variable phase plate 2 so as to change orientation directions of liquid crystal molecules, thereby enabling selectively changing the phase difference provided by the variable phase plate 2 to 0λ, ¼λ and 2/λ. Changing the phase difference enables changing the direction of the transmission axis of the variable polarization axis element 11 to a 90° direction (first direction), a 45° direction (second direction) and a 0° direction (third direction).
The camera body 30 in this embodiment has, as image capturing modes, a first image capturing mode and a second image capturing mode. The first image capturing mode is a mode in which image capturing is performed once in response to one release operation, and the second image capturing mode is a mode in which image capturing is performed multiple times (three times in this embodiment) in response to one release operation.
Description will be made of operations of the polarization acquisition apparatus 10 in the first and second image capturing modes. Description will be herein made of only operations of the polarization acquisition apparatus 10 necessary for acquiring the polarization information, that is, description will not be made of operations not relating to the acquisition of the polarization information, such as operations for relaying the communication between the interchangeable lens 20 and the camera body 30 and relaying control of the interchangeable lens 20 by the camera body 30.
First, description will be made of the operations in the second image capturing mode. In the second image capturing mode, the camera body 30 (camera controller 31) acquires, in response to one release operation, three captured images necessary for acquiring the polarization information. The element controller 13 that has received a release signal generated by the release operation from the camera controller 31 sets the direction of the transmission axis of the variable polarization axis element 11 to the first direction as an initial direction. The initial direction is desirable to be a direction in which energy necessary for driving the variable polarization axis element 11 is minimum. Thus, in this embodiment, the element controller 13 sets, as the initial direction, the 90° direction in which the phase difference is 0λ.
After completion of the setting of the initial direction, the element controller 13 sends, to the camera controller 31, information on the initial direction of the transmission axis and indicating the completion of the setting thereof. The camera controller 31 that has received this information performs first image capturing.
The release signal is generated in response to a full-press operation of the image capturing instruction button that is a two-step operable button. In response to a half-press operation of the image capturing instruction button, the element controller 13 sets the direction of the transmission axis to the initial direction and sends the information on the initial direction and indicating the completion of the setting thereof to the camera controller 31. The camera controller 31 holds this state for a predetermined period of time without starting the first image capturing.
In response to the full-press operation of the image capturing instruction button performed within the predetermined period of time after the setting of the initial direction in response to the half-press operation, the camera controller 31 starts the first image capturing. If the full-press operation of the image capturing instruction button is not performed within the predetermined period of time, the element controller 13 restarts its operation from the setting of the direction of the transmission axis to the initial direction.
This restart of the operation enables acquiring the polarization information even when energization of the polarization acquisition apparatus 10 is stopped because the full-press operation of the image capturing instruction button is not performed within the predetermined period of time.
After completion of the first image capturing, the camera controller 31 sends information indicating the completion of the first image capturing to the element controller 13. The element controller 13 that has received the information indicating the completion of the first image capturing sets the direction of the transmission axis to the second direction and sends information on the second direction and indicating completion of the setting thereof to the camera controller 31.
The camera controller 31 that has received the above information performs second image capturing and sends, after completion of the second image capturing, information indicating the completion thereof to the element controller 13. The element controller 13 that has received the information indicating the completion of the second image capturing sets the direction of the transmission axis to the third direction and sends information on the third direction and indicating completion of the setting thereof to the camera controller 31. The camera controller 31 that has received this information performs third image capturing.
Then, the camera controller 31 stores, to the information recorder 32, three captured images acquired by performing the image capturing three times and the information on the first to third directions received from the element controller 13 in association with one another.
Next, description will be made of the operations in the first image capturing mode. In the first image capturing mode, the element and camera controllers 13 and 31 divisionally perform the operations performed in the second image capturing mode.
The polarization acquisition apparatus 10 holds, in its internal memory (not illustrated), information on the direction of the transmission axis of the variable polarization axis element 11. In response to a user's selection of the first image capturing mode in the camera body 30, the camera controller 31 sends information indicating that the first image capturing mode is set to the element controller 13.
The element controller 13 that has received this information stores information on the initial direction (first direction) to the internal memory, sets the direction of the transmission axis of the variable polarization axis element 11 to the initial direction stored in the internal memory and holds this state for a predetermined period of time.
In response to the full-press operation of the image capturing instruction button performed within the predetermined period of time, the camera controller 31 performs first image capturing. After completion of the first image capturing, the camera controller 31 stores a captured image and the information on the initial direction stored in the internal memory of the element controller 13 and received therefrom in association with each other. In addition, the camera controller 31 sends information indicating the completion of the first image capturing to the element controller 13. The element controller 13 that has received this information changes the direction of the transmission axis to the second direction before next image capturing, holds this state for the predetermined period of time and stores information on the second direction to the internal memory.
In response to the full-press operation of the image capturing instruction button performed within the predetermined period of time, the camera controller 31 performs second image capturing. After completion of the second image capturing, the camera controller 31 stores a captured image and the information on the second direction stored in the internal memory of the element controller 13 and received therefrom in association with each other. Furthermore, the camera controller 31 sends information indicating the completion of the second image capturing to the element controller 13. The element controller 13 that has received this information changes the direction of the transmission axis to the third direction before next image capturing, holds this state for the predetermined period of time and stores information on the third direction to the internal memory.
Then, in response to the full-press operation of the image capturing instruction button performed within the predetermined period of time, the camera controller 31 performs third image capturing. After completion of the third image capturing, the camera controller 31 stores a captured image and the information on the third direction stored in the internal memory of the element controller 13 and received therefrom in association with each other. Thereafter, the element controller 13 changes the direction of the transmission axis to the initial direction.
If the full-press operation of the image capturing instruction button is not performed within the predetermined period of time, the camera controller stops energization of the polarization acquisition apparatus 10. When the full-press operation of the image capturing instruction button is performed after the predetermined period of time in the first image capturing mode, the element controller 13 stores the initial direction and sets the direction of the transmission axis to the initial direction, as in a case where the first image capturing mode is newly selected. Thereby, if the full-press operation of the image capturing instruction button is not performed within the predetermined period of time, next image capturing is performed from a state where the direction of the transmission axis is set to the initial direction.
As described above, in each of the first and second image capturing modes, it is necessary for acquiring the polarization information to perform image capturing three times so as to acquire three captured images. The acquired three captured images can be used not only to acquire the polarization information, but also as images for recording.
Ix=x·(a−b)+b.
As understood from comparison of the captured images illustrated in
Although each of
Moreover, although this embodiment uses the VA liquid crystal element as the variable phase plate 2, different liquid crystal elements may be used depending on characteristics required for the variable phase plate 2. For example, a TN liquid crystal element or an OCB liquid crystal element may be used.
Furthermore, although this embodiment described the case of changing the direction of the transmission axis of the variable polarization axis element 11 in acquiring the captured images to the three directions whose angles are 90°, 45° and 0°, captured images may be acquired with other directions or four or more directions of the transmission axis.
Each of the above-described embodiments enables realizing a polarization acquisition apparatus capable of acquiring polarization information from captured images after image capturing, with no restrictions on object to be image-captured and image capturing conditions. Using the polarization information acquired by this polarization acquisition apparatus makes it possible to produce an image different from the captured images.
Embodiment(s) of the present invention can also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be referred to more fully as a ‘non-transitory computer-readable storage medium’) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the above-described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s). The computer may comprise one or more processors (e.g., central processing unit (CPU), micro processing unit (MPU)) and may include a network of separate computers or separate processors to read out and execute the computer executable instructions. The computer executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read only memory (ROM), a storage of distributed computing systems, an optical disk (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™), a flash memory device, a memory card, and the like.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2015-124769, filed on Jun. 22, 2015, which is hereby incorporated by reference herein in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2015-124769 | Jun 2015 | JP | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 8004675 | Lefaudeux | Aug 2011 | B2 |
| 8259067 | Butterworth | Sep 2012 | B2 |
| 8508646 | Katerberg | Aug 2013 | B2 |
| 8913113 | Kanamori | Dec 2014 | B2 |
| 20120319222 | Ozawa et al. | Dec 2012 | A1 |
| 20140362200 | Kanamori | Dec 2014 | A1 |
| 20170075050 | Yamagata | Mar 2017 | A1 |
| Number | Date | Country |
|---|---|---|
| H06160940 | Jun 1994 | JP |
| 07244237 | Sep 1995 | JP |
| 11041514 | Feb 1999 | JP |
| 2003167267 | Jun 2003 | JP |
| 2009232347 | Oct 2009 | JP |
| 2010121935 | Jun 2010 | JP |
| 2010124011 | Jun 2010 | JP |
| 2011029903 | Feb 2011 | JP |
| 2011040839 | Feb 2011 | JP |
| 2011-191469 | Sep 2011 | JP |
| 2011191469 | Sep 2011 | JP |
| 2012080065 | Apr 2012 | JP |
| 2012513607 | Jun 2012 | JP |
| 2014042126 | Mar 2014 | JP |
| 2014182328 | Sep 2014 | JP |
| 2014236243 | Dec 2014 | JP |
| 2015035735 | Feb 2015 | JP |
| 2016520859 | Jul 2016 | JP |
| 2009147814 | Dec 2009 | WO |
| 2010074719 | Jul 2010 | WO |
| 2014165698 | Oct 2014 | WO |
| Entry |
|---|
| Office Action issued in Japanese Appln. No. 2015-124769 dated Feb. 5, 2019. English Translation provided. |
| Office Action issued in Japanese Application No. 2015-124769 dated Jun. 25, 2019. English translation provided. |
| Office Action issued in Japanese Appln No. 2015-124769 dated Oct. 29, 2019. English translation provided. |
| Number | Date | Country | |
|---|---|---|---|
| 20160370688 A1 | Dec 2016 | US |
