Patent Grant
10003826
This application claims priority to Korean Patent Application No. 10-2015-0151245 filed on Oct. 29, 2015 in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference in its entirety.
1. Field of the Invention
The invention relates to a method of reducing noise of a video signal, and more particularly, to a method of reducing noise of a video signal, which is capable of detecting and reducing noise generated in a quantization step of an encoding process.
2. Description of the Related Art
Noise reduction in a conventional video transcoding process is performed by detecting noise using the intra-picture coding mode or the motion vector of a block and filtering the noise.
However, the conventional noise reduction method cannot detect noise resulting from variations in Quantization Parameters (QPs) that may be encountered during an encoding process.
Referring to
The transcoder 20 receives the converted bitstream, decodes the received bitstream using a first codec decoding unit 21, and converts the decoded bitstream to a second codec using a second codec encoding unit 22.
However, a transcoding process performed by the conventional video transcoding system may include pickup noise 30, which is generated during the pickup of an image by the image pickup unit 11, first quantization noise 40, which is generated during the encoding of the image by the first codec encoding unit 12, and second quantization noise 50, which is generated during the encoding of the image by the second codec encoding unit 22.
Accordingly, the necessity of a method to reduce quantization noise generated during the conversion of a bitstream from an arbitrary first codec to an arbitrary second codec has arisen.
Exemplary embodiments of the invention provide a method of reducing noise of a video signal, which is capable of reducing noise generated in a quantization process.
Exemplary embodiments of the invention also provide a method of reducing noise of a video signal, which is capable of improving the similarities between frames and the compression rate of the frames by extracting a frame with large noise so as to reduce noise.
However, exemplary embodiments of the invention are not restricted to those set forth herein. The above and other exemplary embodiments of the invention will become more apparent to one of ordinary skill in the art to which the invention pertains by referencing the detailed description of the invention given below.
According to an exemplary embodiment of the invention, there is provided a method for reducing noise, the method comprising decoding a bitstream encoded with a first codec, acquiring Quantization Parameter (QP) values determined in a process of encoding the bitstream with the first codec, detecting a frame with noise by using the QP values and inputting the frame with noise to a predefined noise filter.
In an embodiment of the present invention, wherein the detecting the frame with noise, comprises determining a frame having a QP value that exceeds a predefined threshold value as the frame with noise.
In an embodiment of the present invention, wherein the acquiring the QP values, comprises acquiring QP values respectively applied to frames that are sequentially displayed.
In an embodiment of the present invention, wherein the detecting the frame with noise, comprises: calculating a first difference, which is the difference between a QP value of a first frame and a QP value of a second frame, calculating a second difference, which is the difference between the QP value of the second frame and a QP value of a third frame and determining the second frame as the frame with noise if the first and second differences exceed the predefined threshold value.
In an embodiment of the present invention, wherein the detecting the frame with noise, further comprises: calculating a third difference, which is the difference between the QP value of the first frame and the QP value of the third frame and determining the second frame as the frame with noise if the first or second difference exceeds the third difference.
In an embodiment of the present invention, the method further comprising: encoding the bitstream with a second codec.
According to another aspect of the present invention, there is provided a noise reduction apparatus comprising a decoding unit decoding a bitstream encoded with a first codec, a QP acquisition unit acquiring QP values determined in a process of encoding the bitstream with the first codec, a noise extraction unit detecting a frame with noise by using the QP values and a noise filtering unit reducing noise from the frame with noise.
In an embodiment of the present invention, wherein the noise extraction unit determines a frame having a QP value that exceeds a predefined threshold value as the frame with noise.
In an embodiment of the present invention, wherein the QP acquisition unit acquires QP values respectively applied to frames that are sequentially displayed.
In an embodiment of the present invention, wherein the noise extraction unit calculates a first difference, which is the difference between a QP value of a first frame and a QP value of a second frame, and a second difference, which is the difference between the QP value of the second frame and a QP value of a third frame, and determines the second frame as the frame with noise if the first and second differences exceed the predefined threshold value.
In an embodiment of the present invention, wherein the noise extraction unit further calculates a third difference, which is the difference between the QP value of the first frame and the QP value of the third frame, and determines the second frame as the frame with noise if the first or second difference exceeds the third difference.
In an embodiment of the present invention, further comprising: an encoding unit encoding the bitstream with a second codec.
According to another aspect of the present invention, there is provided a noise reduction apparatus comprising one or more processors, a memory loading a computer program that is executed by the processors and a storage storing a noise reduction computer program capable of reducing noise from a video signal, wherein the noise reduction computer program comprises: an operation decoding a bitstream encoded with a first codec, an operation acquiring QP values determined in a process of encoding the bitstream with the first codec, an operation detecting a frame with noise by using the QP values and an operation inputting the frame with noise to a predefined noise filter.
According to another aspect of the present invention, there is provided a computer program stored in a computer-readable recording medium stored in a computer-readable recording medium, for executing, in connection with a computing device, the steps of: decoding a bitstream encoded with a first codec, acquiring QP values determined in a process of encoding the bitstream with the first codec, detecting a frame with noise by using the QP values and inputting the frame with noise to a predefined noise filter.
According to the exemplary embodiments, noise generated in a quantization process can be reduced.
In addition, the similarities between frames and the compression rate of the frames can be improved by extracting a frame with large noise so as to reduce noise.
Other features and aspects will be apparent from the following detailed description, the drawings, and the claims.
Advantages and features of the present invention and methods of accomplishing the same may be understood more readily by reference to the following detailed description of preferred embodiments and the accompanying drawings. The present invention may, however, be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concept of the invention to those skilled in the art, and the present invention will only be defined by the appended claims. Like numbers refer to like elements throughout.
Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
In addition, it will be understood that the singular forms are intended to include the plural forms as well. It will be further understood that the terms “comprises” and/or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, operations, elements, and/or components thereof.
The exemplary embodiment of
In response to a bitstream encoded with the first codec being received, the bitstream is decoded (S210). The decoding of the bitstream may be performed by performing steps of the encoding of the bitstream in reverse order. More specifically, video information of a spatial domain may be generated by {circle around (1)} converting compressed encoded values into symbols, {circle around (2)} performing inverse quantization so as to acquire Discrete Cosine Transform (DCT) coefficients, and {circle around (3)} performing inverse DCT.
A noise reduction apparatus according to an exemplary embodiment of the invention acquires Quantization Parameter (QP) values (S220) determined in the process of encoding the bitstream with the first codec.
A QP is needed to inversely quantize symbol-type encoded values and thus to acquire DCT coefficients. Accordingly, QP information regarding QP values set in the process of compressing a video signal with the first codec is included in the bitstream encoded with the first codec.
An encoder may acquire DCT coefficients by performing DCT on an N×N pixel block of an arbitrary frame. A process of dividing the DCT coefficients by a predetermined value to remove high-frequency components that are of less importance is referred to as quantization, and the predetermined value is referred to as the QP.
Accordingly, the noise reduction apparatus may acquire QP values from the bitstream encoded with the first codec. More specifically, the aforementioned encoding process is performed in units of frames of a video signal, and the noise reduction apparatus may acquire the QP value of each of the frames.
Different frames may have different QP values. For example, a relatively large QP value may be applied to a frame whose compression rate needs to be increased according to the size of bandwidth for transmitting a bitstream, and a relatively small QP value may be applied to a frame whose compression rate does not need to be increased.
Accordingly, a first frame may be quantized with a first QP value, and a second frame, which is different from the first frame, may be quantized with a second QP value, which is different from the first QP value.
Once the QP value of each frame is acquired, a frame with noise is extracted using the QP value of each frame. More specifically, a frame having a QP value larger than a predefined threshold value may be extracted as a frame with noise (S230).
In the case of a frame with a relatively large QP value, a large amount of data is likely to be lost from the frame in the process of increasing the compression rate of the frame, and thus, a large amount of noise may be generated in the frame after the decoding of the frame due to the data loss.
Thus, a frame with a QP value that exceeds the predefined threshold value may be extracted as a frame with noise.
Once the frame with noise is extracted, noise is reduced by inputting the extracted frame to a noise filter (S240). The noise filter may be a Low Pass Filter (LPF), but the invention is not limited thereto. That is, various other general-purpose filters capable of reducing noise may also be used.
In short, the bitstream encoded with the first codec may be decoded, noise caused by the QP may be reduced during the decoding of the bitstream encoded with the first codec, and the decoded bitstream is encoded back with a second codec, thereby performing transcoding.
According to the exemplary embodiment of
A process of increasing the compression rate of a video signal by reducing noise will be described later with reference to
Referring to
An entropy decoder 320 receives the compressed data 310 and converts the compressed data into original symbol values. The symbol values are values acquired by quantizing DCT coefficients during an encoding process.
An inverse quantizer 330 inversely quantizes the symbol values and thus outputs DCT coefficients. In order for the inverse quantizer 330 to perform inverse quantization, QP values used in a quantization process are needed.
The compressed data 310 includes QP information regarding a QP value applied to each frame of the video signal. Accordingly, the noise reduction apparatus may acquire the QP value of each frame from the compressed data 310, i.e., the bitstream.
Thereafter, the inverse quantizer 330 outputs DCT coefficients yet to be quantized, and an inverse discrete cosine transformer 340 outputs a video signal 350 including video information of a spatial domain.
Referring to
The noise reduction apparatus may acquire a QP value applied to each frame by performing the decoding process described above with reference to
If the QP value of a particular frame exceeds a predefined threshold value, the particular frame may be determined as a frame with noise. As mentioned above, the larger the QP value of the particular frame, the larger the amount of high-frequency components removed from the particular frame. That is, the larger the QP value of the particular frame, the larger the amount of data lost during a compression process. Accordingly, if the particular frame is decoded, a large amount of noise may be included in the particular frame. Therefore, a frame having a QP value that exceeds the predefined threshold value may be determined as a frame with noise.
For example, if a QP value QPcur of the second frame 420 exceeds the predefined threshold value, the second frame 420 may be determined as a frame with noise.
However, the invention is not limited to the exemplary embodiment of
Referring to
The noise reduction apparatus calculate a first difference, which is the difference between the QP value of the first frame 410 and the QP value of the second frame 420, and a second difference, which is the difference between the QP value of the second frame 420 and the QP value of the third frame 430.
More specifically, the first and second differences may be calculated by Equations (1) and (2), respectively, as illustrated in
dQPcur-prev=|QPcur−QPprev| (1); and
dQPcur-next=|QPcur−QPnext| (2).
where dQPcur-prev denotes the first difference, QPcur denotes the QP value of the second frame 420, QPprev denotes the QP value of the first frame 410, dQPcur-next denotes the second difference, and QPnext denotes the QP value of the third frame 430.
Thereafter, if the first and second differences exceed a predefined threshold value, the second frame 420 may be determined as a frame with noise.
That is, in a case where the first and second differences both exceed the predefined threshold value, the QP value of the second frame 420 is much larger than the QP value of the first frame 410 and the QP value of the third frame 430.
In this case, a bitstream corresponding to the second frame 420 may be in a state of having a large amount of high-frequency components removed therefrom. Thus, if the bitstream is decoded, a large amount of noise may be included in the second frame 420.
According to the exemplary embodiment of
Referring to
The QP value of the third frame 430 may be calculated by Equation (3), as illustrated in
dQPprev-next=|QPprev−QPnext| (3)
where dQPprev-next denotes the third difference, QPprev denotes the QP value of the first frame 410, and QPnext denotes the QP value of the third frame 430.
In a case where the QP value of a second frame 420 is much larger than the QP value of the first frame 410 and the QP value of the third frame 430, the first or second difference may exceed the third difference.
As already mentioned above, since a frame with a large QP value includes a large amount of noise, the second frame 420 may be determined as a frame with noise.
Once a frame with noise is detected using at least one of the methods of
A method of reducing noise by inputting a frame with noise to a filter is already well known and obvious to a person skilled in the art to which the invention pertains, and thus, a detailed description thereof will be omitted.
In short, by detecting a frame with noise and reducing noise, the similarities between consecutive frames of a video signal may be enhanced. As a result, the compression rate of the video signal may be improved when transcoding a decoded bitstream with a second codec.
There is a method of compressing a video signal using inter-frame temporal correlations. This method reduces the size of a video based on the fact that adjacent frames are generally very similar.
A 24 Frames Per Second (FPS) video signal includes 24 frames for a one-second period. That is, since a motion video is played by exposing similar frames for a short period of time, pixels from temporally adjacent frames are highly likely to have similar values.
That is, the more similar the temporally adjacent frames of a video signal are to each other, the higher the compression rate of the video signal becomes. On the other hand, if too much noise is included in an arbitrary frame, the similarities between the arbitrary frame and its previous and subsequent frames may be lowered, and as a result, the compression rate of the video signal may also be lowered.
Referring to
A fifth frame 720 has a large size in bits than the other frames, and this may be due to an excessive quantization of a fourth frame 710 that is temporally adjacent to the fifth frame 720.
That is, if the fourth frame 710 is quantized by applying a large QP value to the fourth frame 710, a large amount of high-frequency components may be removed from the fourth frame 710, and as a result, a large amount of data may be lost from the fourth frame 710. Then, if the fourth frame 710 is decoded, an excessive number of noise may be generated, and due to this noise, the similarity between the fourth frame 710 and the fifth frame 720 may be lowered.
Accordingly, the compression rate of the fifth frame 720 may decrease, and thus, the fifth frame 720 may have a larger size in bits than the other frames. If the noise of the fourth frame 710 is reduced by the method according to the exemplary embodiment of
More specifically, referring to
Referring to
The decoding unit 910 decodes a bitstream encoded with a first codec. More specifically, the decoding unit 910 generates video information of a spatial domain by {circle around (1)} converting compressed encoded values into symbols, {circle around (2)} performing inverse quantization so as to acquire DCT coefficients, and {circle around (3)} performing inverse DCT.
The QP acquisition unit 920 acquires QP values that have been determined in the process of encoding the bitstream with the first codec. The QP values are included in the bitstream encoded with the first codec. The QP values may be set to differ from one frame to another frame.
The noise extraction unit 930 extracts a frame with noise using the QP values. The noise extraction unit 930 may determine a frame having a QP value that exceeds a predefined threshold value as the frame with noise. Alternatively, the noise extraction unit 930 may detect the frame with noise by calculating the differences between the QP values of consecutive frames.
A method of detecting the frame with noise by calculating the differences between the QP values of consecutive frames has already been described above with reference to
Once the noise extraction unit 930 detects the frame with noise, the noise filtering unit 940 inputs the frame with noise to a filter so as to filter the noise.
The noise filtering unit 940 may be provided as an LPF, but the invention is not limited thereto. That is, the noise filtering unit 940 may also be provided as a general-purpose filter, other than an LPF, capable of reducing noise.
The noise reduction apparatus 900 may also include an encoding unit (not illustrated), which decodes the bitstream encoded with the first codec for transcoding and then encodes the resulting video signal with noise reduced therefrom back with a second codec.
The noise reduction apparatus 900 may reduce noise that may be generated in a quantization process. Also, by reducing the noise, the noise reduction apparatus 900 may enhance the similarities between frames and may thus improve the compression rate of the frames.
Referring to
The processor 1010 executes a noise reduction program capable of reducing noise from a video signal. However, a program that can be executed by the processor 1010 is not particularly limited. That is, various other general-purpose programs may also be executed by the processor 1010.
The storage 1020 stores the noise reduction program. The storage 1020 may also store in advance information regarding a predefined threshold value for detecting a frame with noise. More specifically, a frame with noise may be detected by comparing the QP value of the frame with the predefined threshold value, and thus, the predefined threshold value may be stored in advance in the storage 1020.
The noise reduction program executes the steps of: decoding a bitstream encoded with a first codec; acquiring QP values determined in the process of encoding the bitstream with the first codec; detecting a frame with noise by using the QP values; and inputting the frame with noise to a predefined noise filter.
The memory 1030 loads the noise reduction program and thus allows the noise reduction program to be executed in the processor 1010.
Various computing devices may be connected to the network interface 1040, and the bus 1050 serves as a data transfer path to which the processor 1010, the storage 1020, the memory 1030, and the network interface 1040 are all connected.
The method of reducing noise according to the present invention can be recorded in programs that can be executed on a computer and be implemented through general purpose digital computers. In addition, the data format used in the method for generating the web page according to the present invention may be recorded in a computer-readable recording medium using various means. Examples of the computer-readable recording medium may include recording media such as magnetic storage media (e.g., ROMs, floppy disks, hard disks, etc.) and optical recording media (e.g., CD-ROMs or DVDs).
While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the present invention as defined by the following claims. It is therefore desired that the present embodiments be considered in all respects as illustrative and not restrictive, reference being made to the appended claims
| Number | Date | Country | Kind |
|---|---|---|---|
| 10-2015-0151245 | Oct 2015 | KR | national |
| Number | Name | Date | Kind |
|---|---|---|---|
| 8331438 | Chang et al. | Dec 2012 | B2 |
| 8879856 | Raveendran | Nov 2014 | B2 |
| 20040076271 | Koistinen | Apr 2004 | A1 |
| 20070081587 | Raveendran | Apr 2007 | A1 |
| 20080151993 | Mitsuya | Jun 2008 | A1 |
| 20080240250 | Lin | Oct 2008 | A1 |
| 20100104027 | Youn | Apr 2010 | A1 |
| 20120269266 | Lin | Oct 2012 | A1 |
| 20130114725 | Lou | May 2013 | A1 |
| 20130177084 | Wang | Jul 2013 | A1 |
| 20130271566 | Chen | Oct 2013 | A1 |
| 20140161189 | Zhang | Jun 2014 | A1 |
| 20140301463 | Rusanovskyy | Oct 2014 | A1 |
| 20150172714 | Wu | Jun 2015 | A1 |
| Number | Date | Country |
|---|---|---|
| 4129006 | Jul 2008 | JP |
| 10-0960742 | Jun 2010 | KR |
| 10-2012-0015495 | Feb 2012 | KR |
| Entry |
|---|
| Farrugia et al, Enhancing the error detection capabilities of DCT based codecs using compressed domain dissimilarity metrics (Year: 2007). |
| Communication dated Feb. 27, 2017, issued by the Korean Intellectual Property Office in counterpart Korean Application No. 10-2015-0151245. |
| Communication issued by the Korean Intellectual Property Office dated Sep. 8, 2017 in counterpart Korean Patent Application No. 10-2015-0151245. |
| Number | Date | Country | |
|---|---|---|---|
| 20170127088 A1 | May 2017 | US |
