Patent Application
20200280676
The present invention relates to a processing apparatus, an imaging apparatus, and an imaging system, each of which obtains polarization information using a plurality of images each of which is obtained in a different polarization state of light from an object.
One conventionally known method emphasizes or reduces a predetermined feature in an image of an object by obtaining a polarization state of light from the object. For example, a polarization filter (polarization plate) attached to a front surface of a lens of an imaging apparatus and used to capture an image of an object can emphasize a texture of the object in the image, such as a color and a contrast, and emphasize or reduce reflected light.
U.S. Patent Application Publication No. (“US”) 2009/0079982 and Japanese Patent Application Laid-Open No. (“JP”) 2016-145924 disclose a method for obtaining polarization information of an object by imaging the object a plurality of times while changing a phase difference by applying voltage to a liquid crystal variable phase plate (or retardation plate).
However, the method disclosed in US 2009/0079982 and JP 2016-145924 require the plurality of images of the object in order to obtain polarization information. Thus, an image data amount increases and processing speed decreases.
The present invention provides a processing apparatus, an imaging apparatus, and an imaging system, each of which can reduce a data amount of a plurality of images obtained in different polarization state of light from an object.
The present invention provides a processing apparatus configured to obtain polarization information using a plurality of images each of which is obtained in a different polarization state of light from an object, including a processor configured to select one reference image from the plurality of images and to generate difference data relating to a difference between the reference image and an image other than the reference image, and a recorder configured to record the reference image and the difference data.
An imaging apparatus and an imaging system as one aspect of the present invention including the above processing apparatus also constitute another aspect of the present invention.
Further features of the present invention will become apparent from the following description of exemplary embodiments with reference to the attached drawings.
Referring now to the accompanying drawings, a detailed description will be given in detail of embodiments according to the present invention. Corresponding elements in respective figures will be designated by the same reference numerals, and a duplicate description thereof will be omitted.
The imaging system 100 may include a memory 50 and an image display unit 70. In this case, the memory 50 temporarily records the image data that has been read out of the image sensor 20. The image processing unit 60 processes the image data and then the image recording unit 40 records or the display unit 70 displays the image data.
The imaging system 100 may include an image editing unit (image processor) 80. The image editing unit 80 edits an image based on the image data recorded on the image recording unit 40 or the image data temporarily recorded on the memory 50. The image editing unit 80 can generate a polarized image.
A description will now be given of an acquisition of polarization information, an image recording method, and a reference image selecting method in the imaging system 100.
The imaging system 100 obtains polarization information by capturing the light from the object while changing the polarization state. This embodiment uses the polarization unit 10 for obtaining the polarization information. The polarization unit 10 is disposed on an optical path of the imaging system 100 (between the object and the imaging apparatus 20) and has a function of controlling the polarization state of transmitting light. When the controller 90 controls a transmission polarization state of the polarization unit 10 during imaging, a plurality of images (captured images) can be obtained in a different polarization state, and the image processing unit 60 can obtain the polarization information from the plurality of images.
The number of images to be obtained is two or more, or may be three or more. Obtaining two images provides information on a luminance difference and a luminance ratio of two polarization states. Obtaining three or more images provides further information such as a degree of linear polarization and a polarization angle.
The image processing unit 60 selects one reference image (single reference image) from the obtained plurality of images. A method for selecting the reference image is not particularly limited. For example, a reference polarization state may be set to a state in which linearly polarized light vibrating in a vertical direction relative to the ground is transmitted, and an image captured in or closest to the reference polarization state may be selected as a reference image. Since an image captured in the reference polarization state can always provide a P-polarized light component relative to the ground, an image with a low luminance value, in which reflected light from the ground or a plane parallel to the ground is suppressed, could be set to the reference image. A first captured image may be selected as the reference image. An image having the lowest luminance may be selected. An image captured in a polarization state that is set when an imaging condition (such as a focus position, a shutter speed, an F-number (aperture value), and ISO) is determined may be selected.
The image processing unit 60 generates difference data between an image other than the reference image and the reference image. The image recording unit 40 records the reference image data and the difference data. These data may be recorded on the cloud. Recording of the difference data would save a recording capacity. When the polarization information is to be obtained, an object image is continuously captured while the polarization state changes and therefore a composition variation in the object is small. Thus, recording of difference data saves a large recorded capacity.
The color and the brightness of the object change due to a change in the polarization state, and the change in the image due to the polarization state can be obtained from the difference data. A proper setting of the polarization state during imaging makes it possible to obtain desired polarization information as difference data.
The reference image data and the difference data may be recorded on one file generated in the image recording unit 40. Thereby, the reference image data and the difference data would be recorded in association with each other. Furthermore, a small number of files can make a desired file to be easily found even when a lot of objects are imaged.
When the reference image data and the difference data are recorded on one file, a thumbnail image may be generated based on the reference image for further improvement in a visibility of the file. Thereby, a user can easily find the desired file without opening the file.
Referring now to
In the step S101, the controller rotates the polarization filter of the polarization accessory 11, and make the single-lens camera 21 capture a plurality of images in transmission axis angles different from each other. A memory in the single-lens camera 21 records a plurality of captured image.
This embodiment sequentially sets the angle of the transmission axis of the polarization filter to 0°, 45°, 90°, and 135°, and obtains four images. In this embodiment, the transmission axis angle is defined as an angle θ that is formed between the transmission axis and the x axis, as illustrated in
In the step S102, the controller makes the image processing unit select one reference image from four images recorded on the memory in the step S101. In this embodiment, the image processing unit selects the first captured image (an image obtained in a transmission axis at 0°) as the reference image.
In the step S103, the controller makes the image processing unit generate difference data between an image other than the reference image and the reference image.
In the step S104, the controller records the reference image data and three pieces of difference data on one container file in the image recording unit.
In the step S105, the controller makes the image processing unit generate a thumbnail image of the container file using the reference image, and records the thumbnail image in the same file as the reference image data and the difference data.
The steps S103 and S104 are completely separated in
A format of the data recording is not particularly limited, and the data may be recorded, for example, according to a recording format that is set in advance in the single-lens camera 21. If the user does not specify the data format in advance, recording may be performed according to the initial setting recorded on the single-lens camera 21.
This embodiment does not record image data other than the reference image. That is, the image data other than the reference image is deleted. However, when the setting where all the images can be recorded is turned on, all the images may be recorded.
This embodiment sets the transmission axis angle to 0°, 45°, 90°, and 135° for obtaining the images, but the present invention is not limited to this embodiment and can arbitrarily set it. Various changes can be made according to the required polarization information.
The polarization accessory 15 is a polarization unit using a liquid crystal variable phase plate, and can control the transmission axis angle through the voltage driving of the liquid crystal variable phase plate.
The polarization accessory 15 may include a quarter waveplate, two liquid crystal variable phase plates, and a polarization plate. The liquid crystal variable phase plate rotates liquid crystal molecules in an element plane by applying voltage, and controls the phase difference amount given to the light that has passed through elements disposed in front of the liquid crystal variable phase plate.
The polarization accessory 15 may be composed of a ¼λ plate, one liquid crystal variable phase plate, and a polarization plate. The liquid crystal variable phase plate rotates liquid crystal molecules perpendicularly to an element plane by applying a voltage and controls an in-plane phase difference amount.
When the above phase difference amount and the in-plane phase difference amount are collectively defined as an effective phase difference amount, the polarization accessory 15 can control the transmission axis angle if the effective phase difference amount is made variable according to the voltage driving.
The imaging method and the image recording method follow the flowchart in
A description will now be given of a coloring factor of an image and the effective phase difference amount. When an image is obtained with the polarization accessory 15 using the liquid crystal variable phase plate, coloring of the image that is visually unrecognized by a human eye may occur. This coloring occurs due to the transmission axis angle of the polarization accessory 15 using the liquid crystal variable phase plate determined according to the effective phase difference amount, and due to the presence of a wavelength dispersion in the effective phase difference amount. That is, since the transmission axis angle differs depending on the wavelength, incident light even having the same polarization state may or may not transmit depending on the wavelength, causing the coloring. Since the coloring becomes more conspicuous as the effective phase difference becomes larger, selecting an image captured with a sufficiently small effective phase difference as the reference image is equivalent to selecting an image without coloring.
The sufficiently small effective phase difference amount in acquiring the reference image may be less than 0.1λ, or may be 0.05λ where λ is a center wavelength.
During imaging, the sufficiently small effective phase difference amount set to the first imaging enables the first image to be selected as the reference image.
When an image captured in the reference polarization state is selected as the reference image, the polarization accessory 15 may be disposed so that the direction of the polarization plate is 90°, and the image may be captured with the sufficiently small effective phase difference amount.
The imaging system according to this embodiment provides the camera with the image processing unit and the image recording unit, but the present invention is not limited to this embodiment. A processing apparatus other than the camera may have the image processing unit and the image recording unit. The processing apparatus is, for example, a portable information terminal such as a PC, a smartphone, a tablet terminal, and a notebook PC. The processing apparatus may include an apparatus having an image processing unit (for example, a PC or a smartphone) and an apparatus having an image recording unit (for example, a server).
Although having the same configuration as that of the imaging system according to the first embodiment, an imaging system according to this embodiment performs processing different from that of the imaging system according to the first embodiment. This embodiment will discuss differences from the first embodiment.
Referring now to
In the step S201, the controller rotates a polarization filter of a polarization accessory 11 to make a single-lens camera 21 capture a plurality of images at different transmission axis angles. A memory in the single-lens camera 21 records the plurality of captured images. This embodiment sequentially sets the transmission axis angles of the polarization filter to 0°, 45°, and 90°, and obtains three images.
In the step S202, the controller makes an image processing unit select one reference image from the three images recorded on the memory. In this embodiment, the image processing unit selects an image obtained at a transmission axis angle of 90° as a reference image. The image obtained at the transmission axis angle of 90° corresponds to an image captured in the reference polarization state.
In the step S203, the controller first makes the image processing unit obtain polarization information. Next, the controller makes the image processing unit generate data based on the polarization information.
In this embodiment, the image processing unit calculates a maximum light intensity Imax and a minimum light intensity Imin for each pixel as polarization information. The maximum light intensity Imax and the minimum light intensity Imin are the maximum value and the minimum value, respectively, of light intensity I(θ) relative to the transmission axis angle expressed by the following expression (1).
I(θ)=(Imax−Imin)cos2(θ−α)+Imin (1)
The image processing unit uses three or more images captured at different transmission axis angles and applies a luminance value at the transmission axis angle θ of each pixel to the expression (1) so as to calculate the maximum light intensity Imax and the minimum light intensity Imin.
In this embodiment, the image processing unit generates, as data based on the polarization information, difference data between the maximum light intensity Imax and the minimum light intensity Imin, and difference data between the reference image and Imin.
In the step S204, the controller records the reference image data and the data based on the polarization information on one container file in an image recording unit.
In the step S205, the controller makes the image processing unit generate a thumbnail image of the container file using the reference image, and records the thumbnail image in the same file as the reference image data and the data based on the polarization information.
This embodiment records the polarization information obtained from the plurality of images as difference data, and thereby saves a higher recording capacity than that when recording the plurality of images as they are.
Even if any problem occurs in obtaining polarization information, recording one image (or a single image) as the reference image makes it possible to correct errors in the polarization information based on the reference image, or to use the reference image instead of the polarization information.
In this embodiment, the number of images to be obtained is three, and the polarization information includes the maximum light intensity Imax and the minimum light intensity Imin, but the present invention is not limited to this embodiment. Four or more images may be obtained and the other polarization information (for example, a polarization angle α, a degree of linear polarization, a Stokes parameter, etc.) may be obtained.
If four or more images are obtained, or if the obtained polarization information volume is small, the recording method according to this embodiment would make a large capacity reduction effect.
The smartphone 200 includes an image display unit 93 illustrated in
Referring now to
In the step S301, the controller obtains three images at different transmission axis angles where each image is captured through each lens via each polarization accessory. A memory in the smartphone 200 records these three captured images.
In this embodiment, the polarization accessories include polarization filters having transmission axis angles of 0°, 45°, and 90°, respectively, and provide three images in transmission axis of 0°, 45°, and 90°.
In the step S302, the controller makes the image processing unit select one reference image (single reference image) from the three images recorded on the memory. In this embodiment, the image processing unit selects an image having the lowest luminance as the reference image. Since the image having the low luminance has a small information volume, selecting the image having the low luminance as the reference image would save a recording capacity.
In the step S303, the controller makes the image processing unit obtain polarization information. Thereafter, the controller makes the image processing unit generate data based on the polarization information. In this embodiment, the image processing unit calculates, as the polarization information, a polarization angle α in addition to the maximum light intensity Imax and the minimum light intensity Imin described in the second embodiment. In this embodiment, the image processing unit generates, as data based on the polarization information, difference data between the maximum light intensity Imax and the minimum light intensity Imin and difference data between the reference image and the minimum light intensity Imin.
In the step S304, the controller records the reference image data, the data based on the polarization information, and the polarization angle α on a single container file in the image recording unit.
In the step S305, the controller makes the image processing unit generate a thumbnail image of the container file using the reference image, and records the thumbnail image in the same file as the reference image data, the data based on the polarization information, and the polarization angle α.
Referring now to
In the step S401, the controller displays, on the image display unit 93, a polarized image generated using the polarization information calculated in the step S303 in
In the step S402, the controller makes the image editing unit generate a polarized image using the reference image and the polarization information based on an instruction from the user. The method of generating the polarized image is not particularly limited. For example, a polarized image having a texture different from that of the captured image can be generated using the maximum light intensity Imax and the minimum light intensity Imin. Where the intensity I of each pixel is expressed by a linear sum I=k1−Imin+k2(Imax−Imin), the appearance of the image can be changed by adjusting the coefficients k1 and k2. As the coefficient k2 decreases, an effect of reducing the specular reflection can be obtained. As the coefficient k2 increases, an effect of emphasizing the relief pattern or the texture of the surface can be obtained. As the user changes the coefficient, the image generated using the coefficient is displayed on the image display unit 93 and the user can create a polarized image having a desired texture while viewing the generated image.
In the step S403, the controller records the polarized image generated in the step S403 on the same file as the reference image data, the data based on the polarization information, and the polarization angle α.
In the step S404, the controller generates a thumbnail of the container file in the image processing unit using the polarized image recorded in the step S403, and overwrites the thumbnail image generated in the step S305.
In this embodiment, the smartphone 200 has an image editing unit, but the present invention is not limited to this embodiment. An apparatus different from the smartphone 200 may include an image editing unit.
The imaging system 100D temporarily records data of a plurality of images on a camera or the cloud. At this time, these data may be recorded on a single file.
After obtaining the plurality of images, the PC 300 executes the processing in the steps S302 to S305 in
Each obtained data may be recorded on each camera or the image recording unit in the PC 300, or may be recorded on the cloud.
In the imaging system 100D, a PC is taken as an example of a processing apparatus, but the present invention is not limited to this embodiment. For example, a portable information terminal such as a smartphone, a tablet terminal, and a notebook PC may be used as the processing apparatus.
The above embodiments can provide a processing apparatus, an imaging apparatus, and an imaging system, each of which can save the data amount of a plurality of images obtained in different polarization states of light from an object.
While the present invention has been described with reference to exemplary embodiments, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. The scope of the following claims is to be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.
This application claims the benefit of Japanese Patent Application No. 2019-036133, filed on Feb. 28, 2019 which is hereby incorporated by reference herein in its entirety.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2019-036133 | Feb 2019 | JP | national |
