Patent Application
20190306481
Any and all applications for which a foreign or domestic priority claim is identified in the Application Data Sheet as filed with the present application are hereby incorporated by reference under 37 CFR 1.57.
The present disclosure relates to an underwater digital video camera, in particular such video camera applying a color correction that depends on the depth of the camera.
It is well-known that the light absorption spectrum of water has a significant impact and underwater photography, and that the deeper the more the warm (red) colors are suppressed, resulting in images dominated by colder (green and blue) colors.
U.S. Pat. No. 7,760,994 and EP 2 550 559 B1 describe digital underwater cameras involving a depth sensor, a focus finder and a color correction scheme applied to recorded images that depends on the total water path length of the light from the recorded object. The total water path length can be either the distance from an underwater light source to the object and then to the camera, or the distance from the water surface (light source is the sun) to the object and then to the camera. It is a disadvantage that the applied correction is only suited for still image recording
It is an object of the invention to provide an underwater digital video camera that automatically adjusts a color correction setting during recording.
It is another object of the invention to provide an underwater digital video camera that automatically adjusts a color correction in response to changing depths.
In a first aspect, the invention provides an underwater digital video camera having a control system for adjusting color correction settings for correcting color in video recorded by the video camera, the control system being adapted to receive a current depth of the video camera and repeatedly adjust the color correction settings based on a received current depth of the video camera during recording of the video, i.e. without interrupting video recording. Thereby, when a first part of a video is recorded at a first depth and a second part of the video is recorded at a second, different depth, the control system will set the color correction settings differently for the first and second parts without interrupting video recording. This color correction based on current depth is referred to as depth color correction or DCC.
In a second aspect, the invention provides a method for performing color correction during recording of an underwater video with an underwater, digital video camera, the method comprising the following steps to be carried out repeatedly during recording of video: determining a current depth of the video camera; and adjusting color correction settings based on a determined current depth.
In the present description, an underwater video camera is any camera suitable for recording a video underwater, including a camera placed in an underwater housing. The color correction based on current depth is also referred to as depth color correction, or DCC. The current depth of the video camera includes any data indicative of such depth, such as pressure data or similar. Hence, when referring to “depth” in relation to data from the depth gauge or data stored or processed by the control system, any such data indicative of a depth is meant. Also, the depth of the video camera is to be understood as a depth in the immediate vicinity of the video camera, as the size of the video camera itself causes it to cover a depth range and since the depth gauge may be external to the video camera.
The gist of the invention is that DCC settings are adjusted in response to changing current depths during recording of video and not simply set one time when the recording starts. This has the advantage that when the color correction will not become insufficient or exaggerated when the depth changes substantially during video recording.
Color correction, also sometimes referred to as color reproduction or saturation correction, is commonly provided using a 3×3 linear space color correction matrix (CCM) acting on the RGB values (or similar in other color spaces) will be used in the following. Simpler as well as more complex (higher order, non-linear color correction methods) are described in the prior art and may also be used. The CCM is typically written as:
where R, G, B are the starting values, A0-2, B0-2, C0-2 the gain coefficients and RDCC, GDCC, BDCC are the depth color corrected RGB values. The CCM is the digital version of a red colored filter sometimes used before the lens in underwater photography, but more versatile since it can be used attenuate or intensify any color component. The color correction is applied to image date of each frame of the video and will typically be the same for all frames.
A color correction matrix can be used to adjust the colors of an underwater video to compensate for the wavelength dependent attenuation of water as a function of the depth. Water attenuates longer wavelength light faster than shorter wavelength. This results in videos with a strong green/blue hue or in other words a lack of red color.
With depth color correction according to an embodiment of the invention using CCMs, the matrix coefficients will change according to the current depth of the video camera. Thus, for a certain depth, and water type there will be a corresponding CCM as shown in
White balance (WB) correction is a special case of color correction serving to make sure that white elements in a scene also appear white in the recorded image. The need for white balance correction is therefore determined by the color temperature of the light incident on the object as well as absorption of light between the object and the camera. In the present invention, only the effect of the color temperature of the light incident on the object as determined by the depth of the object (and the water type) will be taken into account.
White balance can also be described with a CCM:
Often the WB gain coefficients are calculated with reference to a white point (or grey point). At a white/grey point the RGB values will optimally be equal, R=G=B. When calculating the gain coefficients A1-3, one possibility is to use the green pixel of a white or grey reference point and calculate the red and blue gain with reference to the green pixel. In this case, the WB matrix may look like this:
In a preferred embodiment, the color correction is a white balance correction, and the color correction settings adjusted by the control system are white balance gain coefficients.
The WB gain coefficients are typically empirically determined by taking a photo (in raw format) of a white or grey reference object at different depths, perform white balance correction in post-processing and then determine the value of each gain coefficient as a function of depth. This determination of gain coefficient profiles can be used to determine color correction versus depth profiles for different underwater conditions such as the turbidity and the apparent color of the water. These different profiles can be stored in the camera and selected by the user or automatically.
Thus, in a preferred embodiment, the control system utilizes a stored color correction versus depth profile to adjust color correction settings based on the current depth, and the video camera holds multiple stored color correction versus depth profiles suitable for different underwater conditions.
In a first embodiment, the control system receives a depth or depth-related pressure data from an electric depth gauge 8 and determines (10) a current depth of the video camera. Based on the current depth, the control system then adjusts (12) the DCC settings of the camera. In further embodiments, the control system also performs DCC (13, 16) on the recorded image sensor data (15) before the video is stored (18) in a final digital video file format. This correction on the recorded image sensor data is carried out for each frame of image sensor data, and other processing (17) such as other corrections and compression will typically also be performed before storage.
The control system may retrieve (11) a relevant color correction versus depth profile for adjusting the DCC settings.
In a further embodiment, the control system stores (14) the depth on which the most recent adjustment of the color correction settings is based and/or a time at which the most recent adjustment of the color correction settings is made. This stored data can be used in the determination of when to repeat adjustment of the DCC settings. The interval with which DCC settings are adjusted during recording is a balance between desired video quality and processing/power usage.
Adjusting DCC settings for e.g. every 2 centimeters of depth change or every 0.1 second would generally be considered as a waste of processing power and electric power without visual improvement of the video quality. Also, it would likely exceed the precision of the depth gauge. On the other hand, if the DCC settings are adjusted with too large intervals, the changes in color correction may be visible in the video as sudden changes in color.
There are, as described previously, different approaches to the repetition of DCC settings adjustment, i.e. the how the adjustment or updating of the DCC settings to be applied is triggered. These approaches use the depth and/or time data stored for the most recent adjustment of DCC settings to determine when adjustment of DCC settings are to be repeated.
In one embodiment, the control system can adjust DCC settings based on the current depth at fixed time intervals, such as at least every 0.5 second or every 1 second or every 2 seconds. Preferably, the control system repeats the adjustment of the color correction settings based on the current depth when a difference between a current time and the stored time exceeds a second predetermined threshold. This time triggered adjustment of color correction settings is advantageous since the processing and power usage is predictable and does not fluctuate.
In another embodiment, the control system adjusts DCC settings based on the current depth when the current depth has changed a predetermined amount, such as 0.1 meter, 0.5 meter, 1 meter, or 2 meters. Preferably, the control system repeats the adjustment of the color correction settings based on a current depth when a difference between a current depth and the stored depth exceeds a first predetermined threshold. This is advantageous since the DCC settings are a function of depth and thus need to be adjusted when the depth changes substantially. This changed-depth triggered adjustment of color correction settings is particularly relevant in applications where the depth changes rapidly such as in freediving. The change in depth since the last adjustment can be determined in different ways. One way may be to continuously compare the current depth from the depth gauge to the stored depth and trigger the adjustment when the difference exceeds the predetermined threshold. Another way would be to compare the current depth from the depth gauge to the stored depth at regular time intervals, St, and trigger the adjustment when the difference exceeds the predetermined threshold. Since such comparison of two depths requires very little processing power, the time interval St used may be very short, such as fractions of a second.
In yet another embodiment, the control system applies a combination of time- and changed-depth triggered adjustment of color correction settings, so that the color correction settings are adjusted upon a predetermined change in depth or time passed since the last adjustment, whichever occurs first.
The means 6 to provide a current depth of the video camera 1 to the control system may be an electric depth gauge 8 comprised by the video camera or a communication link to an external electric depth gauge 8 positioned close to the video camera and being in communication with the camera, such as via a wired or wireless communication link. Such external depth gauge would measure the depth of the depth gauge and not of the video camera and should therefore be close to the video camera, typically carried by the user of the video camera, so that it would measure the depth of the video camera+/−max. 0.5 meter. In the present description, the depth measured by such close, external depth gauge is also referred to as the depth of the video camera.
A depth gauge is most commonly a pressure sensor measuring a pressure of the water surrounding the sensor. The pressure P as a function of depth in the is given by:
P=P
0
+ρg d
G
where P0 is the air pressure at the upper surface of the water, ρ is the water density (1000 kg/m3), g is the acceleration due to gravity (˜9.8 m/s2) and dG is the depth of the depth gauge. Preferably, the pressure sensor is calibrated to return the “gauge pressure” PG=P−P0, which is the pressure difference relative to the air pressure. The gauge pressure can be used to determine the depth of the depth gauge:
d
G
=P
G
/ρg.
Other types of depth gauges exist and may be used in the invention.
| Number | Date | Country | |
|---|---|---|---|
| 62649445 | Mar 2018 | US |
