Patent Application
20250193511
This application claims priority of Japanese Patent Application No. 2021-90956 filed on May 31, 2021, which is incorporated by reference in this application.
The present invention relates to an imaging device and an image processing device.
Recently, imaging devices and control methods for imaging devices have been known in which, when continuously capturing still images while moving the front curtain and rear curtain, the blackout period of a live view image can be shortened.
A disclosure of the imaging device comprises: an imaging element configured to output a sequence of first frames at a first interval through first image capture, and output a second frame through an instruction for second image capture; a thinning unit configured to perform thinning of the sequence of first frames and outputs the thinned first frames at a second interval; and a control unit configured to control an output timing of the second frame on the basis of the first interval or the second interval and an instruction timing for the second image capture.
Another disclosure of the imaging device comprises: an imaging element configured to output a sequence of first frames at a first interval through first image capture, and output a second frame through an instruction for second image capture; a generation unit configured to generate a third frame by thinning the second frame; and a data processing unit configured to control an output timing of the third frame generated by the generation unit at a timing in which the first frames are not outputted at the first interval due to the instruction for the second image capture.
A disclosure of the image processing device is configured to be able to connect to an imaging element configured to output a sequence of first frames at a first interval through first image capture, and output a second frame through an instruction for second image capture, the image processing device comprises: a thinning unit configured to perform thinning of the sequence of first frames and outputs the thinned first frames at a second interval; and a control unit configured to control an output timing of the second frame on the basis of the first interval or the second interval and an instruction timing for the second image capture.
Another disclosure of the image processing device is configured to be able to connect to an imaging element configured to output a sequence of first frames at a first interval through first image capture, and output a second frame through an instruction for second image capture, the image processing device comprises: a generation unit configured to generate a third frame by thinning the second frame; and a data processing unit configured to control an output timing of the third frame generated by the generation unit at a timing in which the first frames are not outputted at the first interval due to the instruction for the second image capture.
In the embodiments below, an example will be described in which a blackout-free live view display with a natural appearance is realized using an imaging element that can perform so-called dual pipeline processing. A blackout is the momentary appearance of a black image over the entire screen of a display unit (rear monitor and electronic viewfinder) of an imaging device while capturing a still image.
The read circuit 112 reads the LV data or still image data from the pixel region 111. The LV data is image data for repeatedly displaying an LV image in the display unit 103. The still image data is image data captured as a result of a trigger (hereinafter, the imaging trigger) such as the depressing of a shutter release button or the elapsing of a time setting in a timer.
The read circuit 112 thins out prescribed pixel signals from the pixel group of the pixel region 111 to read the LV data when the imaging trigger has not been inputted, and reads the pixel signals from the pixel group of the pixel region 111 without thinning out if the imaging trigger has been inputted. The still image data may alternatively be thinned by the imaging element 101 but in such a case, the number of thinned pixels would be set to be less than for the LV data.
In Embodiment 1, an imaging state where LV data is repeatedly outputted without the input of the imaging trigger is referred to as the LV imaging state, and an imaging state where the imaging trigger has been inputted is referred to as the still image capture state.
The LV data is outputted from the read circuit 112 to an LV data processing unit 121 of the image processing unit 102, and the still image data is outputted to a still image data processing unit 122 of the image processing unit 102.
The image processing unit 102 has the LV data processing unit 121 and the still image data processing unit 122. The image processing unit 102 may specifically be realized by an integrated circuit such as a field-programmable gate array (FPGA), for example, or may be realized by a processor executing image processing programs stored in a memory.
The LV data processing unit 121 performs image processing on a sequence of LV data repeatedly outputted from the imaging element 101 and outputs the same to the display unit 103. Specifically, the LV data processing unit 121 executes generation of evaluation values for 3A (automatic exposure control (AE), auto white balance (AWB), and autofocus (AF)), developing, distortion correction, image resizing, and thinning, and adjusts the LV data so as to be displayable in the display unit 103, for example.
The still image data processing unit 122 performs image processing on the still data outputted from the imaging element 101 and stores the same in the recording medium 104. Specifically, the still data processing unit 122 executes generation of evaluation values for 3A, developing, distortion correction, and image resizing, and adjusts the still image data so as to be recordable in the recording medium 104.
Thus, the imaging device 100 executes dual pipeline processing in which the imaging element 101 outputs the LV data and the still image data in parallel and the image processing unit 102 performs image processing in parallel on the LV data and the still image data. This dual pipeline processing resolves the issue of blackouts in the LV data.
The display unit 103 displays the repeatedly outputted LV data. The display unit 103 specifically is a monitor provided on the rear surface of the imaging device 100 and/or an electronic viewfinder that can be viewed by the photographer's eye being directly against the viewfinder, for example.
The recording medium 104 is a memory that records the still image data. The recording medium 104 may be fixed to the imaging device 100 or be detachable therefrom. The still image data recorded in the recording medium 104 may be displayable in the display unit 103.
The imaging trigger output control unit 105 controls the timing at which the imaging trigger is outputted to the read circuit 112 on the basis of frame rate (FR) information, and outputs the imaging trigger to the read circuit 112. The frame rate information is the frame rate during exposure of the imaging element 101 and/or the frame rate during thinning. The imaging trigger output control unit 105 generates a timing signal 200 to be described later, and counts an ascending order number to be described later.
Next, a blackout-free live view display example based on control of the output timing will be described in detail. In Embodiment 1, the frame rate during exposure of the imaging element 101 is 120 fps, and the frame rate during thinning for display in the display unit 103 is 60 fps.
The numbers #(in
The imaging element 101 repeatedly outputs the LV data at 120 fps by exposure. In Embodiment 1, the LV data is the frame A to the frame O. In an LV imaging state where a shutter release signal is not issued even once during the ascending order numbers 0-14, the LV data processing unit 121 thins frames A to O outputted at 120 fps from the imaging element 101 down to 60 fps, and outputs the non-thinned frames A, C, E, G, I, K, M, and O to the display unit 103. That is, the frames B, D, F, H, J, L, and N are thinned. The display unit 103 sequentially displays the non-thinned frames A, C, E, G, I, K, M, and O for 1/60 second.
Here, the imaging trigger is the shutter release button being depressed at the timing of the ascending order number 5, for example and the read circuit 112 receiving the shutter release signal. In this case, the imaging element 101 outputs the frame G at the subsequent ascending order number 6 as still image data to the still image data processing unit 122.
The still image data processing unit 122 performs still image processing on the frame G and records the same in the recording medium 104. That the graphic for the frame G in the still image processing and recording is elongated in the horizontal direction indicates the length of time taken for still image processing and recording.
Thus, the frame G is the still image data, and therefore, the imaging element 101 outputs the LV data with the frame G omitted, or in other words, the frames A-F and H-O to the LV data processing unit 121.
The LV data processing unit 121 thins the LV data of the frames A-F and H-O, with the frame G removed, at 60 fps, and outputs the frames A, C, E, I, K, M, and O to the display unit 103. The display unit 103 displays the frames A, C, E, I, K, M, and O in the stated order, but while the display time for the frames A, C, I, K, M, and O is 1/60 second, the display time for the frame E is 1/30 second. This is because the frame G is dropped as LV data due to being acquired as the still image data, and thus, the time to display the frame E is extended until the timing that the frame I is displayed.
That is, if the frame rate during thinning is ½ the frame rate during exposure of the imaging element 101, then upon receiving input of the shutter release signal at the timing of the ascending order number 2n+1 (n being an integer of 0 or greater), the imaging trigger output control unit 105 outputs the shutter release signal to the read circuit 112 at the timing of the ascending order number 2n+2. As a result, the read circuit 112 reads the still image data from the pixel region 111 at the timing of the ascending order number 2n+3.
The still image data processing unit 122 performs still image processing on the frame H and records the same in the recording medium 104. That the graphic for the frame H in the still image processing and recording is elongated in the horizontal direction indicates the length of time taken for still image processing and recording.
Thus, the frame H is the still image data, and therefore, the imaging element 101 outputs the LV data with the frame H omitted, or in other words, the frames A-G and I-O to the LV data processing unit 121.
The LV data processing unit 121 thins the LV data of the frames A-G and I-O, with the removal of the frame H that was thinned during the thinning process, at 60 fps, and outputs the frames A, C, E, G, K, M, and O, which were not thinned, to the display unit 103. That is, the frames B, D, F, J, L, and N are thinned.
The display unit 103 sequentially displays the non-thinned frames A, C, E, G, K, M, and O for 1/60 second. In this manner, by controlling the read timing of the read circuit 112 such that the thinned frame H is the still image data, the display unit 103 can display the non-thinned frames A, C, E, G, K, M, and O at the same time interval ( 1/60 second).
That is, if the frame rate during thinning is ½ the frame rate during exposure of the imaging element 101, then upon receiving input of the shutter release signal at the timing of the ascending order number 2n, the imaging trigger output control unit 105 outputs the shutter release signal to the read circuit 112 at the timing of the ascending order number 2n. As a result, the read circuit 112 reads the still image data from the pixel region 111 at the timing of the ascending order number 2n+1.
Thus, similar to
In Embodiment 1, the imaging device 100 may have a first selection unit 500 that can select, by user designation, between the operation indicated in
Embodiment 2 is a configuration of Embodiment 1 in which the frame rate during exposure of the imaging element 101 is 120 fps, and the frame rate during thinning for display in the display unit 103 is 40 fps. Descriptions of content that is the same as Embodiment 1 will be omitted.
The imaging element 101 repeatedly outputs the LV data at 120 fps by exposure. In an LV imaging state where a shutter release signal is not issued even once during the ascending order numbers 0-14, the LV data processing unit 121 thins frames A to O outputted at 120 fps from the imaging element 101 down to 40 fps, and outputs the frames A, D, E, G, J, and N to the display unit 103. The display unit 103 sequentially displays the frames A, D, E, G, J, and N for 1/40 second.
Here, the imaging trigger is the shutter release button being depressed at the timing of the ascending order number 5, for example and the read circuit 112 receiving the shutter release signal. In this case, the imaging element 101 outputs the frame G at the subsequent ascending order number 6 as still image data to the still image data processing unit 122.
The still image data processing unit 122 performs still image processing on the frame G and records the same in the recording medium 104. Thus, the frame G is the still image data, and therefore, the imaging element 101 outputs the LV data with the frame G omitted, or in other words, the frames A-F and H-O to the LV data processing unit 121.
The LV data processing unit 121 thins the LV data of the frames A-F and H-O, with the frame G removed, to 40 fps, and outputs the non-thinned frames A, D, J, and M to the display unit 103. That is, the frames B, C, E, F, H, I, K, L, N, and O are thinned.
The display unit 103 displays the non-thinned frames A, D, J, and M in the stated order, but while the display time for the frames A, J, and M is 1/40 second, the display time for the frame D is 1/20 second. This is because the frame G is dropped as LV data due to being acquired as the still image data, and thus, the time to display the frame E is extended until the timing that the frame J is displayed.
That is, if the frame rate during thinning is ⅓ the frame rate during exposure of the imaging element 101, then upon receiving input of the shutter release signal at the timing of the ascending order number {2+3(n−1)} (n being an integer of 1 or greater), the imaging trigger output control unit 105 outputs the shutter release signal to the read circuit 112 at the timing of the ascending order number {3+3(n−1)}. As a result, the read circuit 112 reads the still image data from the pixel region 111 at the timing of the ascending order number {4+3(n−1)}.
The still image data processing unit 122 performs still image processing on the frame H and records the same in the recording medium 104. The LV data processing unit 121 thins the LV data of the frames A-G and I-O, with the removal of the frame H that was thinned during the thinning process, to 40 fps, and outputs the non-thinned frames A, D, G, J, and M, which were not thinned, to the display unit 103.
That is, the frames B, C, E, F, I, K, L, N, and O are thinned. The display unit 103 sequentially displays the non-thinned frames A, D, G, J, and M for 1/40 second. In this manner, by controlling the read timing of the read circuit 112 such that the thinned frame H is the still image data, the display unit 103 can display the frames A, D, G, J, and M at the same time interval ( 1/40 second).
That is, if the frame rate during thinning is ⅓ the frame rate during exposure of the imaging element 101, then upon receiving input of the shutter release signal at the timing of the ascending order number {2+3(n−1)} (n being an integer of 1 or greater), the imaging trigger output control unit 105 outputs the shutter release signal to the read circuit 112 at the timing of the ascending order number {4+3(n−1)}. As a result, the read circuit 112 reads the still image data from the pixel region 111 at the timing of the ascending order number {5+3(n−1)}.
The still image data processing unit 122 performs still image processing on the frame I and records the same in the recording medium 104. The LV data processing unit 121 thins the LV data of the frames A-H and J-O, with the removal of the frame I that was thinned during the thinning process, to 40 fps, and outputs the non-thinned frames A, D, G, J, and M, which were not thinned, to the display unit 103.
That is, the frames B, C, E, F, H, K, L, N, and O are thinned. The display unit 103 sequentially displays the non-thinned frames A, D, G, J, and M for 1/40 second. In this manner, by controlling the read timing of the read circuit 112 such that the thinned frame H is the still image data, the display unit 103 can display the frames A, D, G, J, and M at the same time interval ( 1/40 second).
As described above, upon receiving the shutter release signal at the timing of the ascending order number 5, the imaging trigger output control unit 105 can output at the timing of either the ascending order number 6 or the ascending order number 7, but if reducing the offset between the shutter release timing and the read timing of the still image data to the greatest extent possible, the imaging trigger output control unit 105 should delay from the ascending order number 5 to the ascending order number 6 in outputting to the read circuit 112 as indicated in
Also, in Embodiment 2, the imaging device 100 may have a first selection unit 500 that can select, by user designation, between the operation indicated in
Next, Embodiment 3 will be described. Embodiment 3 is an example in which extension is avoided in the live view display by a method differing from Embodiments 1 and 2. Descriptions of content that is the same as Embodiments 1 and 2 will be omitted.
In
The still image data processing unit 122 performs still image processing on the frame G and records the same in the recording medium 104. Thus, the frame G is the still image data, and therefore, the imaging element 101 outputs the LV data with the frame G omitted, or in other words, the frames A-F and H-O to the LV data processing unit 121.
Without thinning the LV data of the frames A-F and H-O, with the frame G removed, the LV data processing unit 121 outputs the frames A-F and H-O to the display unit 103. The display unit 103 displays the frames A-F and H-O in the stated order, but while the display time for the frames A-E and H-O is 1/120 second, the display time for the frame F is 1/60 second. This is because the frame G is dropped as LV data due to being acquired as the still image data, and thus, the time to display the frame F is extended until the timing that the frame H is displayed.
The LV data processing unit 121 sequentially outputs the LV data starting with the frame A to the display unit 103, and after outputting the frame F, outputs the frame G generated by the LV data generation unit 801, and subsequently outputs the frames F-O. Thus, the display unit 103 can display the frames A-O at the same time interval ( 1/120 second).
Similar to
The LV data processing unit 121 sequentially outputs the LV data starting with the non-thinned frame A to the display unit 103, and after outputting the non-thinned frame E, outputs the frame G generated by the LV data generation unit 801, and subsequently outputs the non-thinned frames I, K, M, and O. Thus, the display unit 103 can display the frames A, C, E, G, I, K, M, and O at the same time interval ( 1/60 second).
Similar to
In Embodiment 3, the imaging device 800 may have a second selection unit 802 that can select, by user designation, between the operation indicated in
Also, the imaging device 800 of Embodiment 3 may be configured so as to be able to execute the operations of Embodiment 1 or 2. Specifically, the imaging device 800 of Embodiment 3 has the imaging trigger output control unit 105 and the first selection unit 500, for example. As a result, the imaging device 800 selects, using the above-mentioned selection unit, either the operation mode indicated in
Thus, according to Embodiments 1 to 3, it is possible to realize a live view display by which each piece of LV data can be displayed at the same time interval in a blackout-free manner. As a result, display is controlled such that the subject in the extended frame in the live view instantaneously moves in the subsequent frame.
In the imaging device 100 of Embodiments 1 and 2, the imaging trigger output control unit 105 outputs the shutter release signal to the read circuit 112 while controlling the output timing thereof on the basis of the frame rate during thinning, but the shutter release signal may be outputted to the read circuit 112 while controlling the output timing thereof on the basis of the frame rate during exposure of the imaging element 101.
Specifically, the imaging trigger output control unit 105 has the function of calculating the frame rate during thinning by multiplying the frame rate during exposure by 1/K (K can be set by the user to any integer value greater than or equal to 1), for example. Once the frame rate during exposure is set, the imaging trigger output control unit 105 calculates the frame rate during thinning by multiplying the frame rate during exposure by 1/K, and outputs the shutter release signal to the read circuit 112 by controlling the output timing thereof at the calculated frame rate.
In Embodiments 1 to 3, examples were given of the imaging devices 100 and 800 having incorporated therein the imaging element 101, but the imaging element 101 may be configured so as to be detachable from the imaging devices 100 and 800.
Remaining portions of the imaging devices 100 and 800 after removal of the imaging element 101 constitute the image processing device that can be connected to the imaging element 101 in a manner enabling communication therewith. In Embodiments 1 and 2, the imaging trigger output control unit 105 is provided outside of the imaging element 101 but may be installed in the imaging element 101.
The present invention is not limited to the content above, and the content above may be freely combined. Also, other aspects considered to be within the scope of the technical concept of the present invention are included in the scope of the present invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2021-090956 | May 2021 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2022/021993 | 5/30/2022 | WO |
