Patent Application
20120293489
1. Field of the Invention
The present invention generally relates to digital image processing, and more particularly to a nonlinear depth remapping system and method for a three-dimensional (3D) image pair.
2. Description of Related Art
When three-dimensional (3D) objects are mapped onto a two-dimensional (2D) image plane by prospective projection, such as an image taken by a still camera or a video camera, a lot of information, particularly 3D depth information, disappears. A 3D imaging system, however, can convey 3D information to a viewer by recording 3D visual information or by re-creating the illusion of depth. Although the 3D imaging technique has been known for over a century, the 3D display becomes more practical and popular owing to availability of high-resolution and low-price displays such as liquid crystal displays (LCDs).
However, there are still some basic constraints in stereo videos, for example, there may be a discrepancy between the image which two-camera captured and the image that viewer saw. The visual percept of depth information felt by the two-camera and two-eye of viewer may be different as well. There could be some health issues occurring. People may feel dizzy after watching a long term 3D movie or someone has the problem to discriminate depth accurately. These phenomenons raise a new issue between depth information and human visual system.
In view of the foregoing, a need has arisen to propose a novel depth adjusting system and method for an image that could improve perceptual feeling and provide a much more comfortable viewing experience.
In view of the foregoing, it is an object of the embodiment of the present invention to provide a nonlinear depth remapping system and method for an image which could remap or adjust 3D depth information to improve perceptual feeling and provide a much more comfortable viewing experience.
According to one embodiment, a nonlinear depth remapping system which comprises a depth generator and a depth adjusting unit is disclosed. The depth generator creates an initial depth map associated with at least one image, wherein the image comprises a plurality of pixels, and the initial depth map carries an initial depth value of each pixel. The depth adjusting unit utilizes an exponential function to adjust the initial depth values, so as to generate an adjusted depth map.
According to another embodiment, a nonlinear depth remapping method is disclosed. The method comprises the following steps: firstly, an initial depth map associated with at least one image is received, wherein the image comprises a plurality of pixels, and the initial depth map carries an initial depth value of each pixel. Then, an exponential function is utilized to adjust the initial depth values, so as to generate an adjusted depth map.
In order to facilitate explaining, take a single depth map for example as follows. Please refer to
After obtaining the initial depth values of the initial depth map 33, the depth adjusting unit 22 adjusts the initial depth values by an exponential function as the equations (1), (2),
Wherein D(x,y) is the initial depth value. Dmax and Dmin are the maximum and minimum of the initial depth values, respectively. Davg is average of Dmax and Dmin. The exponent (γ) of the exponential function (equations (1)), which is not fixed, is calculated according to the difference between each initial depth value D(x,y) and the average depth value Davg. Therefore, each initial depth value D(x,y) may be adjusted according to the difference between each initial depth value D(x,y) and the average depth value Davg. Hence, the new depth values (adjusted depth values O(x,y)) are adjusted from the initial depth values D(x,y), so as to generate an adjusted depth map 35.
The adjusted depth map 35 from the depth adjusting unit 22 is fed to the depth-image-based rendering (DIBR) unit 23, which generates (or synthesizes) an adjusted left (L′) image 25A and an adjusted right (R′) image 25B for being displayed and viewed by viewer based on the adjusted depth map 35 and the original image. The DIBR unit 23 may be implemented by a suitable conventional technique, for example, disclosed in a disclosure entitled “A 3D-TV Approach. Using Depth-Image-Based Rendering (DIBR),” by Christoph Fehn, the disclosure of which is hereby incorporated, by reference. For another example, the DIBR further generates more than two images with different viewpoint for multi-view application.
It is noted that, after depth remapping processing as above, in the region of the displayed image that is far from the display plane such as LCD, the steps between disparities were enhanced. Whereas in the region of the displayed image that is near the display plane, the differences of disparities were compressed. Therefore, it increases disparity steps, both on the near and the far sides according to the proposed exponential function, so as to increase 3D feeling both on the foreground and the background objects. The nonlinear effect on stereo perception can be compensated.
Afterward, in step S405, the depth adjusting unit 22 calculates the exponential parameter, the exponent (γ) of the exponential function, according to the difference between each initial depth value D(x,y) and the average depth value Davg by equations (2). Then, in step S407, the depth adjusting unit 22 puts each initial depth value D(x,y) and its corresponding exponential parameter (γ) into the exponential function by equations (1) to remap the original depth values, so as to generate an adjusted depth map 35 with new depth value in step S409.
Finally, the DIBR unit 23 then generates an adjusted left (L′) image 25A and an adjusted right (R′) image 251B for being displayed and viewed by viewer based on the adjusted depth map 35 in step S411.
According to the foregoing embodiment, the present invention proposes a nonlinear depth remapping processing using an exponential function to adjust the depth information to be suitable for human visual system, which not only improves perceptual feeling, but also provides a much more comfortable viewing experience.
Although specific embodiments have been illustrated and described, it will be appreciated by those skilled in the art that various modifications may be made without departing from the scope of the present invention, which is intended to be limited solely by the appended claims.
