The present disclosure relates to unmanned aerial vehicle field and, more particularly, to an image processing method, a device, and a computer-readable storage medium.
A dynamic range of an image refers to the ratio of the highest brightness value to the lowest brightness value in a natural scene. With the development of sensor technologies, current cameras can obtain up to 16 bits of data, but most display devices can only display 8 bits of data. When a high dynamic range image needs to be displayed on a low dynamic range display device, tone mapping of the high dynamic range image is required.
In the existing technology, a method for performing tone mapping on a high dynamic range image includes a global tone mapping method and a local tone mapping method, however, both the global tone mapping method and the local tone mapping method perform tone mapping on the luminance information of the high dynamic range image, which lead to a difficulty in retaining the color information of the high dynamic range image in the low dynamic range image after tone mapping, and a offset in color in the low dynamic range image compared to the high dynamic range image.
In accordance with the disclosure, there is provided an image processing method including pre-processing an original image to obtain a pre-processed image, decomposing the pre-processed image to obtain a plurality of first sub-images, determining detail information, color information, and mean value information of the plurality of first sub-images, compressing the plurality of first sub-images to obtain a plurality of second sub-images according to the detail information, the color information, and the mean value information, and determining a target image according to the plurality of second sub-images.
In accordance with the disclosure, there is provided an image processing device including a computer-readable storage medium storing a computer program, and one or more processors individually or collectively configured to pre-process an original image to obtain a pre-processed image, decompose the pre-processed image into a plurality of first sub-images, determine detail information, color information, and mean value information of the first sub-images, compress the plurality of first sub-images according to the detail information, the color information, and the mean value information to generate a plurality of second sub-images, and determine a target image according to the plurality of second sub-images.
Technical solutions of the present disclosure will be described with reference to the drawings of the embodiments of the disclosure. The described embodiments are only some embodiments of the disclosure not all the embodiments. Based on the embodiments of the disclosure, all other embodiments obtained by one of ordinary skill in the art without any creative effort are within the scope of the present disclosure.
When a component is referred to as “fixed to” another component, the component may be directly on the another component or may have a component therebetween. When a component is referred to as “connected to” another component, the component may be directly connected to the another component or may have a component therebetween.
All technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art, unless otherwise defined. The terminology used in the specification of the present disclosure is for the purpose of describing specific embodiments and is not intended to limit the disclosure. The term “and/or” as used herein includes any and all combinations of one or more of the associated listed items.
Some embodiments of the disclosure are described in detail with reference to the drawings. When no conflict, the features of the embodiments and the embodiments described below can be combined with each other.
In accordance with the disclosure, there is provided an image processing method.
As shown in
In the embodiment, an image processing device first pre-processes an original image to obtain a first image. The original image is an image that needs image processing. In some embodiments, the original image may be an individual image frame captured by a photographing device, and may be one image frame of a plurality of continuous image frames in video data captured by a photographing device. In the embodiment, the source of the original image is not restricted.
In some embodiments, obtaining the first image by pre-processing the original image includes converting the original image into red, green, and blue (RGB) space (i.e., converting the original image from an original color space into RGB space) to obtain a second image (also referred to as an “RGB image”), and globally adjusting the second image to obtain the first image. The first image includes R channel data, G channel data, and B channel data.
In the embodiment, the original image is denoted as L, which is converted into the RGB space to obtain the second image. In some embodiments, the original image is a high dynamic range image and the second image obtained by converting the original image into the RGB space is also a high dynamic range image. The second image is denoted as Li, i∈r, g, b, where Lr denotes the R channel data of the second image, Lg denotes the G channel data of the second image, and Lb denotes the B channel data of the second image. The second image Li is globally adjusted to obtain the first image, and the first image is denoted as Li′, i∈r, g, b. The global adjustment method can include obtaining the first image Li′, i∈r, g, b by global adjustment of the high dynamic range image, i.e., the second image Li, using a log curve. The specific adjustment method is shown in formula (1):
L
i′=log(Li*106+1) (1)
where, i∈r, g, b, Lr denotes the R channel data of the second image before the global adjustment, Lg denotes the G channel data of the second image before the global adjustment, Lb denotes the B channel data of the second image before the global adjustment, Lr′ denotes the R channel data of the first image after the global adjustment, Tg′ denotes the G channel data of the first image after the global adjustment, and Lb′ denotes the B channel data of the first image after the global adjustment.
At S102, the first image is decomposed into multiple first sub-images.
In some embodiments, the first image Li′, i∈r, g, b is decomposed into the plurality of first sub-images. A first sub-image is denoted as Xi, i∈r, g, b, where xr denotes the R channel data of the first sub-image, Xg denotes the G channel data of the first sub-image, and Xb denotes the B channel data of the first sub-image. In some embodiments, the first image Li′, i∈r, g, b is decomposed into the plurality of first sub-images by a sliding window method. The mean value mr of the first sub-images in R channel can be calculated according to the R channel data xr of the first sub-images, the mean value mg of the first sub-images in the G channel can be calculated according to the G channel data Xg of the first sub-images, and the mean value mb of the first sub-images in the B channel can be calculated according to the B channel data Xb of the first sub-images. The mean values of the first sub-images in R channel, G channel, and B channel are denoted as mi, i∈r, g, b. The overall mean value of the first sub-images in the three channels of R channel, G channel, and B channel is denoted as m, and m can be determined by using following formula (2):
m=(mr+mg+mb)/3 (2)
At S103, detail information, color information, and mean value information of each first sub-image of the plurality of first sub-images are determined.
In the embodiment, each first sub-image of the plurality of first sub-images can be decomposed into 3 parts, and the 3 parts are detail part, color part, and mean value part. The detail part corresponds to the detail information of the first sub-images, the color part corresponds to the color information of the first sub-images, and the mean value part corresponds to the mean value information of the first sub-images. The detail information can be expressed as formula (3), the color information can be expressed as formula (4), and the mean value information is the overall mean value m of the first sub-images in three channels of the R channel, G channel, and B channel.
X
i
where
m
i
where
At S104, the first sub-images are compressed according to the detail information, the color information, and the mean value information to obtain the plurality of second sub-images.
In some embodiments, each of the first sub-images is compressed according to detail information
In some embodiments, compressing the plurality of first sub-images according to the detail information, the color information, and the mean value information to generate the plurality of second sub-images includes non-linearly compressing the color information and the detail information of each first sub-image and linearly compressing the mean value information of the first sub-image to obtain the corresponding second sub-image.
Specifically, when any one of the first sub-images is compressed, the color information and detail information of the first sub-image can be non-linearly compressed, and the mean value information of the first sub-image can be linearly compressed, so that the corresponding second sub-image of the first sub-image is obtained. In general, each first sub-image can be compressed in the same way so that the second sub-image corresponding to that first sub-image is obtained, i.e., the plurality of second sub-images are generated.
At S105, the target image is determined according to the plurality of second sub-images.
As shown in
Determining the target image according to the plurality of second sub-images includes forming a third image by arranging the second sub-images corresponding to the first sub-images according to the positions of the first sub-images in the first image, and mapping the pixel value of each pixel in the third image to the dynamic range of a display device to obtain the target image. In this disclosure, the third image is also referred to as a “composed image.”
As shown in
After the third image 200 is obtained, the pixel value of each pixel in the third image 200 can be mapped to the dynamic range of the display device. For example, the dynamic range of the display device is 8 bit, thus the pixel value of each pixel in the third image 200 can be stretched to the range of 0-255 to obtain the target image. That is, the pixel values in the range 0-1 are mapped to the range 0-255.
In the embodiment, the original image is a high dynamic range image and the target image is a low dynamic range image.
In some other embodiments, before mapping the pixel value of each pixel in the third image to the dynamic range of the display device, the process also includes adjusting the pixel values of the pixels in the third image to improve the contrast of the third image.
Since the pixel value of each pixel in the third image 200 is from 0 to 1, the pixel values of the pixels in the third image 200 can be adjusted to improve the contrast of the third image 200. The specific adjustment method can include compressing the pixel values of the brightest and the darkest pixels according to a pre-set compression ratio. For example, the pre-set ratio can be 10%. Since the pixel value of each pixel in the third image 200 is 1 at maximum, the pixels with the pixel values smaller than 1*10% is set to 0 and the pixels with the pixel values larger than 1*(1−10%) is set to 1.
In some embodiments, after the third image 200 is formed by arranging the second sub-image 210, the second sub-image 220, the second sub-image 230, and the second sub-image 240, weighting process is performed on the second sub-image 210, the second sub-image 220, the second sub-image 230, and the second sub-image 240. The reason for performing the weighting process is that the sliding window method is used to decompose the first image into a plurality of first sub-images. In certain sliding window method, adjacent first sub-images may overlap with each other. That is, two adjacent ones of the first sub-image 21, the first sub-image 22, the first sub-image 23, and the first sub-image 24 may overlap with each other. After the first sub-image 21, the first sub-image 22, the first sub-image 23, and the first sub-image 24 are compressed individually to obtain the second sub-image 210, the second sub-image 220, the second sub-image 230, and the second sub-image 240, the overlaps may still exist between two adjacent ones of the second sub-image 210, the second sub-image 220, the second sub-image 230, and the second sub-image 240. To prevent the overlaps from affecting the picture quality of the third image 200, the weighting process is implemented on the second sub-image 210, the second sub-image 220, the second sub-image 230, and the second sub-image 240 in the third image 200 to reduce or eliminate the overlaps between the two adjacent second sub-images.
In the embodiment, the first image is obtained by pre-processing the original image and the first image is decomposed into the plurality of first sub-images. Each first sub-image is compressed to generate a corresponding second sub-image according to the detail information, the color information, and the mean value information of the first sub-image and the target image is determined according to the plurality of second sub-images. When the tone mapping is performed on the high dynamic range image, not only tone mapping is performed on the brightness information, but also the detail information, the color information, and the mean value information are compressed at the same time to ensure that the low dynamic range image retains the color information of the high dynamic range image after tone mapping and to avoid the offset in color of the low dynamic range image as compared to the high dynamic range image.
In accordance with the disclosure, there is provided an image processing method.
At S301, the individual mean values of the first sub-images in R channel, G channel, and B channel are calculated.
In some embodiments, calculating the individual mean values of the first sub-images in R channel, G channel, and B channel includes calculating a first mean value of the first sub-images in R channel according to the R channel data of the first sub-images, calculating a second mean value of the first sub-images in G channel according to the G channel data of the first sub-images, and calculating a third mean value of the first sub-images in B channel according to the B channel data of the first sub-images.
The first sub-images are denoted as Xi, i∈r, g, b, where xr denotes the R channel data of the first sub-images, Xg denotes the G channel data of the first sub-images, and Xb denotes the B channel data of the first sub-images. When the individual mean values of the first sub-images in R channel, G channel, and B channel are calculated, in some embodiments, the first mean value mr of the first sub-images in R channel is calculated according to the R channel data xr of the first sub-images, the second mean value mg of the first sub-images in G channel is calculated according to the G channel data Xg of the first sub-images, and the third mean value mb of the first sub-images in B channel is calculated according to the B channel data Xb of the first sub-images.
At S302, the detail information, the color information, and the mean value information of the first sub-images are determined according to the individual mean values of the first sub-images in R channel, G channel, and B channel.
In some embodiments, determining the detail information, the color information, and the mean value information of the first sub-images according to the individual mean values of the first sub-images in R channel, G channel, and B channel includes the following aspects.
In one aspect, the detail information of the first sub-images in R channel, G channel, and B channel is determined according to the R channel data, the G channel data, and the B channel data of the first sub-images and the individual mean values of the first sub-images in R channel, G channel, and B channel.
In some embodiments, determining the detail information of the first sub-images in R channel, G channel, and B channel according to the R channel data, the G channel data, and the B channel data of the first sub-images and the individual mean values of the first sub-images in R channel, G channel, and B channel includes determining the detail information of the first sub-images in R channel according to the R channel data and the first mean value of the first sub-images, determining the detail information of the first sub-images in G channel according to the G channel data and the second mean value of the first sub-images, and determining the detail information of the first sub-images in B channel according to the B channel data and the third mean value of the first sub-images.
For example, the R channel data of the first sub-images is denoted as xr, the first mean value is denoted as mr, the detail information of the first sub-images in the R channel is denoted as
In another aspect, the color information of the first sub-images in each channel is determined according to the individual mean values of the first sub-images in R channel, G channel, and B channel and the mean value of the first sub-images.
In some embodiments, determining the color information of the first sub-images in R channel, G channel, and B channel according to the individual mean values of the first sub-images in R channel, G channel, and B channel and the mean value of the first sub-images includes determining the color information of the first sub-images in R channel according to the first mean value and the mean value of the first sub-images, determining the color information of the first sub-images in G channel according to the second mean value and the mean value of the first sub-images, and determining the color information of the first sub-images in B channel according to the third mean value and the mean value of the first sub-images.
In some embodiments, the mean value information of the first sub-images is a mean value of the first mean value, the second mean value, and the third mean value. In some embodiments, the individual mean values of the first sub-images in R channel, G channel, and B channel are denoted as mi, i∈r, g, b. The overall mean value of the first sub-images in R channel, G channel, and B channel is denoted as m, m can be determined according to formula (2) above, i.e., m=(mr+mg+mb)/3. The mean value of the first sub-images is the mean value of the first mean value, the second mean value, and the third mean value, i.e., the mean value of the first sub-images is m.
For example, the first mean value is denoted as mr, the mean value information of the first sub-images is denoted as m, the color information of the first sub-images in R channel is denoted as
In the embodiment, by calculating the individual mean values of the first sub-images in R channel, G channel, and B channel, the detail information, the color information, and the mean value information of the first sub-images are determined, and accurate calculations are implemented for the detail information, the color information, and the mean value information.
In accordance with the disclosure, there is provided an image processing method.
At S401, the detail information of the first sub-images is clustered to determine a group to which each first sub-image belongs.
As shown in
In the embodiment, by clustering the detail information of the first sub-images, the group of each first sub-image is determined. In some embodiments, the K-means method can be used for clustering. During clustering, the detail information of each first sub-image can be constructed as a column vector. For example, a first sub-image Xi, i∈r, g, b is a 5*5 block, and hence each of xr, Xg, Xb is a 5*5 block and each of
Assume that the first image is decomposed into M first sub-images, and N groups are obtained after clustering, where M and N are positive integers and N is less than or equal to M. The N groups are respectively denoted as G1, G2, G3 . . . GN, and the covariance matrices of the groups are Φ1, Φ2, Φ3 . . . . ΦN, where Φn denotes the covariance matrix of the nth group in the N groups. Formula (5) can be obtained with eigenvalue decomposition of the covariance matrix Φn:
Φn=QnΛnQn−1 (5)
where Qn is a square matrix composed of eigenvectors, and Λn is a diagonal matrix composed of eigenvalues. Correspondingly, the dictionary Pn corresponding to the nth group can be calculated as follows in formula (6):
P
n
=Q
n
T (6)
where QnT is the transpose matrix of Qn.
Dictionaries corresponding to other ones of the N groups other than the nth group can be calculated using the same method as above. Thus, the corresponding dictionaries for groups G1, G2, G3 . . . GN are P1, P2, P3 . . . PN, respectively.
At S402, the first sub-images are projected according to the detail information of the first sub-images and the groups of the first sub-images to obtain projection values of the detail information of the first sub-images.
In some embodiments, projecting the first sub-images according to the groups of the first sub-images and the detail information of the first sub-images includes determining the covariance matrices of the groups of the first sub-images, decomposing the covariance matrices to determine the corresponding dictionaries of the groups for the first sub-images, and projecting the detail information of the first sub-images into the corresponding dictionaries of the groups of the first sub-images.
As shown in
In some embodiments, the detail information of each first sub-image is projected into the corresponding dictionary of the group that the first sub-image belongs to. For example, the detail information of the first sub-image 21 is projected into the corresponding dictionary of the group to which the first sub-image 21 belongs, the detail information of the first sub-image 22 is projected into the corresponding dictionary of the group to which the first sub-image 22 belongs, the detail information of the first sub-image 23 is projected into the corresponding dictionary of the group to which the first sub-image 23 belongs, and the detail information of the first sub-image 24 is projected into the corresponding dictionary of the group to which the first sub-image 24 belongs. The projection process will be described in more detail taking the projection of the detail information of the first sub-image 21 into the corresponding dictionary of the group to which the first sub-image 21 belongs as an example. The projection processes for other first sub-images are similar.
In some embodiments, the detail information of the first sub-image 21 is
At S403, the projection values of the detail information of the first sub-images are non-linearly compressed to obtain first compression data.
In some embodiments, the projection value P1
α=w1(γ(P1
where w1 denotes the S-type curve, γ denotes a threshold function. For an unknown number z, the threshold function γ can be expressed as formula (8) below, and the S-type curve can be expressed as formula (9) below:
where max denotes the maximum value of the projection value, TN denotes a threshold.
w
1(z)=(2/π)*arctan(z*b) (9)
where b is a parameter used to control a shape of the S-type curve.
The first compression data obtained by non-linearly compressing the projection value P1
At S404, the color information of the first sub-images is non-linearly compressed to obtain second compression data.
The color information
At S405, the mean value information of the first sub-images is linearly compressed to obtain third compression data.
The mean value information of the first sub-image 21 is the overall mean value of the first sub-image 21 in R channel, G channel, and B channel, and the third compression data obtained by linearly compressing the mean value information of the first sub-image 21 is denoted as w2*m, where w1 is a number from 0 to 1.
At S406, image reconstruction is performed according to the first compression data, the second compression data, and the third compression data to obtain the second sub-images.
Image reconstruction is performed according to the first compression data P1Tα, the second compression data w3(
y
i
=P
1
T
α+w
2
*m+w
3(
The calculation principles of the second sub-image 220, the second sub-image 230, and the second sub-image 240 are the same as the calculation principles of the second sub-image 210, which are not described here.
In the embodiment, the first image is obtained by pre-processing the original image and the first image is decomposed into the plurality of first sub-images. The first sub-images are compressed to generate the plurality of second sub-images according to the detail information, the color information, and the mean value information of the first sub-images. The target image is determined according to the plurality of second sub-images. When tone mapping is performed on the high dynamic range image, not only tone mapping is performed on the brightness information, but also the detail information, the color information, and the mean value information are compressed at the same time, so as to ensure that the low dynamic range image retains the color information of the high dynamic range image after tone mapping and to avoid an offset in color of the low dynamic range image as compared to the high dynamic range image.
In accordance with the disclosure, there is provided an image processing method.
In the embodiment, the original image is pre-processed to obtain the first image, the first image is decomposed into the plurality of first sub-images, and the first sub-images are compressed according to the detail information, the color information, and the mean value information of the first sub-images to generate the plurality of second sub-images. The target image is determined according to the plurality of second sub-images. When the high dynamic range image is tone mapped, not only the tone mapping is performed on the brightness information, but also the detail information, the color information, and the mean value information are compressed at the same time to ensure that the low dynamic range image retains the color information of the high dynamic range image to avoid an offset in color of the low dynamic range image as compared to the high dynamic range image.
In accordance with the disclosure, there is provided an image processing device.
In some embodiments, to compress the plurality of first sub-images according to the detail information, the color information, and the mean value information to generate the plurality of second sub-images, the one or more processors 71 non-linearly compress the color information and the detail information of the first sub-images and linearly compress the mean value information of the first sub-images to obtain the second sub-images.
In some embodiments, the original image is the high dynamic range image, the target image is the low dynamic range image.
In some embodiments, to pre-process the original image to obtain the first image, the one or more processors 71 convert the original image into RGB space to obtain the second image, and globally adjust the second image to obtain the first image. The first image includes the R channel data, the G channel data, and the B channel data.
In some embodiments, to determine the target image according to the plurality of second sub-images, the one or more processors 71 arrange the second sub-images corresponding to the first sub-images to construct the third image according to the positions of the first sub-images, and map the pixel values of the pixels of the third image into the dynamic range of the display device to obtain the target image.
In some embodiments, before the one or more processors 71 map the pixel values of the pixels of the third image into the dynamic range of the display device, the one or more processors 71 also adjust the pixel values of the pixels of the third image to improve the contrast of the third image.
The principles and implementation of the image processing device are similar to those of the methods described above in connection with
In the embodiment, the original image is pre-processed to obtain the first image, the first image is decomposed into the plurality of first sub-images, the first sub-images are compressed according to the detail information, the color information, and the mean value information of each first sub-image to generate the plurality of second sub-images, and the target image is determined according to the plurality of second sub-images. When the high dynamic range image is tone mapped, not only the tone mapping is performed on the brightness information, but also the detail information, the color information, and the mean value information are compressed at the same time to ensure that the low dynamic range image retains the color information of the high dynamic range image to avoid an offset in color of the low dynamic range image as compared to the high dynamic range image.
In accordance with the disclosure, there is provided an image processing device. Based on the technical solution of the embodiment descried above in connection with
In some embodiments, to determine the detail information, the color information, and the mean value information of the first sub-images according to the individual mean value of the first sub-images in R channel, G channel, and B channel, the one or more processors 71 determine the detail information of the first sub-images in R channel, G channel, and B channel according to the R channel data, the G channel data, and the B channel data of the first sub-images and the individual mean values of the first sub-images in R channel, G channel, and B channel, and determine the color information of the first sub-images in each channel according to the individual mean values of the first sub-images in R channel, G channel, and B channel and the mean value information of the first sub-images.
To calculate the individual mean values of the first sub-images in R channel, G channel, and B channel, the one or more processors 71 calculate the first mean value of the first sub-images in R channel according to the R channel data of the first sub-images, calculate the second mean value of the first sub-images in G channel according to the G channel data of the first sub-images, and calculate the third mean value of the first sub-images in B channel according to the B channel data of the first sub-images.
To determine the detail information of the first sub-images in R channel, G channel, and B channel according to R channel data, G channel data, and B channel data of the first sub-images and the individual mean values of the first sub-images in R channel, G channel, and B channel, the one or more processors 71 determine the detail information of the first sub-images in R channel according to the first mean value of the first sub-images in R channel, determine the detail information of the first sub-images in G channel according to the second mean value of the first sub-images in G channel, and determine the detail information of the first sub-images in B channel according to the third mean value of the first sub-images in B channel.
To determine the color information of the first sub-images in R channel, G channel, and B channel according to the individual mean values of the first sub-images in R channel, G channel, and B channel and the mean value information of the first sub-images, the one or more processors 71 determine the color information of the first sub-images in R channel according to the first mean value and the mean value information of the first sub-images, determine the color information of the first sub-images in G channel according to the second mean value and the mean value information of the first sub-images, and determine the color information of the first sub-images in B channel according to the third mean value and the mean value information of the first sub-images.
In some embodiments, the mean value information of the first sub-images is the mean value of the first mean value, the second mean value, and the third mean value.
The principles and implementation of the image processing device are similar to those of the methods described above in connection with
In the embodiment, by calculating the individual mean values of the first sub-images in R channel, G channel, and B channel, the detail information, the color information, and the mean value information of the first sub-images are determined and the accurate calculations are implemented for the detail information, the color information, and the mean value information.
In accordance with the disclosure, there is provided an image processing device. Based on the technical solution of the embodiments descried above in connection with
In some embodiments, before the first sub-images are projected according to the detail information and the groups to which the first sub-images belong, the one or more processors 71 cluster the detail information of the first sub-images to determine the groups to which the first sub-images belong.
In some embodiments, to project the first sub-images according to the detail information and the groups to which the first sub-images belong, the one or more processors 71 determine the covariance matrices of the groups to which the first sub-images belong, decompose the covariance matrices to determine the corresponding dictionaries of the groups to which the first sub-images belong, and project the detail information of the first sub-images in the corresponding dictionaries of the groups to which the first sub-images belong.
The principles and implementation of the image processing device are similar to those of the methods described above in connection with
In the embodiment, the original image is pre-processed to obtain the first image, the first image is decomposed into the plurality of first sub-images, and the first sub-images are compressed according to the detail information, the color information, and the mean value information of the first sub-images to generate the plurality of second sub-images. The target image is determined according to the plurality of second sub-images. When the tone mapping is performed on the high dynamic range image, not only the tone mapping is performed on the brightness information, but also the detail information, the color information, and the mean value information are compressed at the same time to ensure that the low dynamic range image retains the color information of the high dynamic range image to avoid an offset in color of the low dynamic range image as compared to the high dynamic range image.
In accordance with the disclosure, there is provided an unmanned aerial vehicle (UAV).
In some embodiments, as shown in
The image processing device 109 can perform image processing to the image captured by the photographing device 104. The image processing method is similar to those of the methods described above, the principles and the implementation of image processing device 109 are similar to those of the methods described above, which are thus not described here in detail.
In the embodiment, the original image is pre-processed to obtain the first image, the first image is decomposed into the plurality of first sub-images, and the first sub-images are compressed to generate the plurality of second sub-images according to the detail information, the color information, and the mean value information of the first sub-images. The target image is determined according to the plurality of second sub-images. When the tone mapping is performed on the high dynamic range image, not only the tone mapping is performed on the brightness information, but also the detail information, the color information, and the mean value information are compressed at the same time to ensure that the low dynamic range image retains the color information of the high dynamic range image to avoid an offset in color of the low dynamic range image as compared to the high dynamic range image.
In accordance with the disclosure, there is provided a computer-readable storage medium, in which the computer program is stored. The computer program is executed by the one or more processors to pre-process the original image to obtain the first image, decompose the first image into the plurality of first sub-images, determine the detail information, the color information, and the mean value information of the first sub-images, compress the plurality of first sub-images to generate the plurality of second sub-images according to the detail information, the color information, and the mean value information, and determine the target image according to the plurality of second sub-images.
In some embodiments, the plurality of first sub-images are compressed to generate the plurality of second sub-images according to the detail information, the color information, and the mean value information, and the process includes non-linearly compressing the color information and the detail information of each first sub-image and linearly compressing the mean value information of the first sub-images to obtain the second sub-images.
In some embodiments, the original image is the high dynamic range image, and the target image is the low dynamic range image.
In some embodiments, pre-processing the original image to obtain the first image includes converting the original image into RGB space to obtain the second image, and globally adjusting the second image to obtain the first image. The first image includes the R channel data, the G channel data, or the B channel data.
In some embodiments, determining the detail information, the color information, and the mean value information of the first sub-images includes calculating the individual mean values of the first sub-images in R channel, G channel, and B channel, and determining the detail information, the color information, and the mean value information of the first sub-images according to the individual mean values of the first sub-images in R channel, G channel, and B channel.
In some embodiments, determining the detail information, the color information, and the mean value information of the first sub-images according to the individual mean values of the first sub-images in R channel, G channel, and B channel includes determining the detail information of the first sub-images in R channel, G channel, and B channel according to the R channel data, the G channel data, and the B channel data and the individual mean values of the first sub-images in R channel, G channel, and B channel, and determining the color information of the first sub-images in R channel, G channel, and B channel according to the individual mean values of the first sub-images in R channel, G channel, and B channel and the mean value information of the first sub-images.
In some embodiments, calculating the individual mean values of the first sub-images in R channel, G channel, and B channel includes calculating the first mean value of the first sub-images in R channel according to the R channel data of the first sub-images, calculating the second mean value of the first sub-images in G channel according to the G channel data of the first sub-images, and calculating the third mean value of the first sub-images in B channel according to the B channel data of the first sub-images.
In some embodiments, determining the detail information of the first sub-images in R channel, G channel, or B channel according to the R channel data, the G channel data, and the B channel data of the first sub-image and the individual mean values of the first sub-images in R channel, G channel, and B channel includes determining the detail information of the first sub-images in R channel according to the R channel data of the first sub-images and the first mean value, determining the detail information of the first sub-images in G channel according to the G channel data of the first sub-images and the second mean value, and determining the detail information of the first sub-images in B channel according to the B channel data of the first sub-images and the third mean value.
In some embodiments, according to the individual mean values of the first sub-images in R channel, G channel, and B channel and the mean value information of the first sub-images, determining the color information of the first sun-images in R channel, G channel, and B channel includes determining the color information of the first sub-images in R channel according to the first mean value and the mean value information of the first sub-images, determining the color information of the first sub-images in G channel according to the second mean value and the mean value information of the first sub-images, and determining the color information of the first sub-images in B channel according to the third mean value and the mean value information of the first sub-images.
In some embodiments, the mean value information of the first sub-images is the mean value of the first mean value, the second mean value, and the third mean value.
In some embodiments, non-linearly compressing the color information and the detail information of the first sub-images and linearly compressing the mean value information of the first sub-images to obtain the second sub-images includes projecting the first sub-images according to the detail information of the first sub-images and the groups to which the first sub-images belong to obtain the projection value of the detail information of the first sub-images, non-linearly compressing the projection value of the detail information of the first sub-images to obtain the first compression data, non-linearly compressing the color information of the first sub-images to obtain the second compression data, linearly compressing the mean value information of the first sub-images to obtain the third compression data, and reconstructing the image according to the first compression data, the second compression data, and the third compression data to obtain the second image.
In some embodiments, before projecting the first sub-images according to the detail information of the first sub-images and the groups to which the first sub-images belong includes clustering the detail information of the first images to determine the groups to which the first sub-images belong.
In some embodiments, projecting the first sub-images according to the detail information of the first sub-images and the groups to which the first sub-images belong includes determining covariance matrices of the groups to which the first sub-images belong, decomposing the covariance matrices to determine the corresponding dictionaries of the groups to which the first images belong, and projecting the detail information of the first sub-images into the corresponding dictionaries of the groups to which the first sub-images belong.
In some embodiments, determining the target image according to a plurality of second images includes arranging the corresponding second sub-images of the first sub-images according to the positions of the sub-images in the first image to construct the third image, and mapping the pixel values of the pixels of the third images into the dynamic range of the display device to obtain the target image.
In some embodiments, before projecting the pixel values of the pixels in the third image into the dynamic range of the display device includes adjusting the pixel values of the pixels of the third image to improve the contrast of the third image.
In some embodiments of the disclosure, the devices and methods disclosed can be implemented in other forms. For example, the device embodiments described above are merely illustrative. the division of the units is only a logical function division, and the actual implementation may be according to another division method. For example, a plurality of units or components can be combined or integrated in another system, or some features can be omitted or not be executed. Further, the displayed or discussed mutual coupling or direct coupling or communicative connection can be through some interfaces, the indirect coupling or communicative connection of the devices or units can be electronically, mechanically, or in other forms.
The units described as separate instructions may be or may not be physically separated, the components displayed as units may be or may not be physical units, which can be in one place or be distributed to a plurality of network units. Some or all of the units can be chosen to implement the purpose of the embodiment according to the actual needs.
In the embodiments of the disclosure, individual functional units can be integrated in one processing unit, or can be individual units physically separated, or two or more units can be integrated in one unit. The integrated units above can be implemented by hardware or can be implemented by hardware and software functional unit.
The integrated units implemented by software functional units can be stored in a computer-readable storage medium. The above software functional units stored in a storage medium includes a plurality of instructions for a computing device (such as a personal computer, a server, or network device, etc.) or a processor to execute some of the operations in the embodiments of the disclosure. The storage medium includes USB drive, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, or another medium that can store program codes.
Those of ordinary skilled in the art can understand that, for convenient and simple description, the division of individual functional units are described as an example. In actual applications, the functions above can be assigned to different functional units for implementation, i.e., the internal structure of the device can be divided into different functional units to implement all or some of the functions described above. For the specific operation process of the device described above, reference can be to the corresponding process in the method embodiments, which will not be described in detail here.
The individual embodiments are merely used to describe the technical solution of the disclosure but not used to limit the disclosure. Although the disclosure is described in detail referring to the individual embodiments, one of ordinary skill in the art should understand that it is still possible to modify the technical solutions in the embodiments, or to replace some or all of the technical features. However, these modifications or substitutions do not cause the essence of the corresponding technical solution to depart from the scope of the technical solutions in the individual embodiments of the disclosure.
This application is a continuation of International Application No. PCT/CN2017/096627, filed Aug. 9, 2017, the entire content of which is incorporated herein by reference.
Number | Date | Country | |
---|---|---|---|
Parent | PCT/CN2017/096627 | Aug 2017 | US |
Child | 16731026 | US |