One or more exemplary embodiments relate to quantization and inverse quantization of a linear prediction coefficient, and more particularly, to a method and apparatus for efficiently quantizing a linear prediction coefficient with low complexity and a method and apparatus for inverse quantization.
In a system for encoding a sound such as speech or audio, a linear predictive coding (LPC) coefficient is used to represent a short-term frequency characteristic of the sound. The LPC coefficient is obtained in a form of dividing an input sound in frame units and minimizing energy of a prediction error for each frame. However, the LPC coefficient has a large dynamic range, and a characteristic of a used LPC filter is very sensitive to a quantization error of the LPC coefficient, and thus stability of the filter is not guaranteed.
Therefore, an LPC coefficient is quantized by converting the LPC coefficient into another coefficient in which stability of the filter is easily confirmed, interpolation is advantageous, and a quantization characteristic is good. It is mostly preferred that an LPC coefficient is quantized by converting the LPC coefficient into a line spectral frequency (LSF) or an immittance spectral frequency (ISF). Particularly, a scheme of quantizing an LSF coefficient may use a high inter-frame correlation of the LSF coefficient in a frequency domain and a time domain, thereby increasing a quantization gain.
An LSF coefficient exhibits a frequency characteristic of a short-term sound, and in a case of frame in which a frequency characteristic of an input sound sharply varies, an LSF coefficient of a corresponding frame also sharply varies. However, a quantizer including an inter-frame predictor using a high inter-frame correlation of an LSF coefficient cannot perform proper prediction for a sharply varying frame, and thus, quantization performance decreases. Therefore, it is necessary to select an optimized quantizer in correspondence with a signal characteristic of each frame of an input sound.
One or more exemplary embodiments include a method and apparatus for efficiently quantizing a linear predictive coding (LPC) coefficient with low complexity and a method and apparatus for inverse quantization.
According to one or more exemplary embodiments, a quantization apparatus includes: a first quantization module for performing quantization without an inter-frame prediction; and a second quantization module for performing quantization with an inter-frame prediction, wherein the first quantization module includes: a first quantization part for quantizing an input signal; and a third quantization part for quantizing a first quantization error signal, the second quantization module includes: a second quantization part for quantizing a prediction error; and a fourth quantization part for quantizing a second quantization error signal, and the first quantization part and the second quantization part include a vector quantizer of a trellis structure.
According to one or more exemplary embodiments, a quantization method includes: selecting, in an open-loop manner, one of a first quantization module for performing quantization without an inter-frame prediction and a second quantization module for performing quantization with an inter-frame prediction; and quantizing an input signal by using the selected quantization module, wherein the first quantization module includes: a first quantization part for quantizing the input signal; and a third quantization part for quantizing a first quantization error signal, the second quantization module includes: a second quantization part for quantizing a prediction error; and a fourth quantization part for quantizing a second quantization error signal, and the third quantization part and the fourth quantization part share a codebook.
According to one or more exemplary embodiments, an inverse quantization apparatus includes: a first inverse quantization module for performing inverse quantization without an inter-frame prediction; and a second inverse quantization module for performing inverse quantization with an inter-frame prediction, wherein the first inverse quantization module includes: a first inverse quantization part for inverse-quantizing an input signal; and a third inverse quantization part disposed in parallel to the first inverse quantization part, the second inverse quantization module includes: a second inverse quantization part for inverse-quantizing the input signal; and a fourth inverse quantization part disposed in parallel to the second inverse quantization part, and the first inverse quantization part and the second inverse quantization part include an inverse vector quantizer of a trellis structure.
According to one or more exemplary embodiments, an inverse quantization method includes: selecting one of a first inverse quantization module for performing inverse quantization without an inter-frame prediction and a second inverse quantization module for performing inverse quantization with an inter-frame prediction; and inverse-quantizing an input signal by using the selected inverse quantization module, wherein the first inverse quantization module includes: a first inverse quantization part for quantizing the input signal; and a third inverse quantization part disposed in parallel to the first inverse quantization part, the second inverse quantization module includes: a second inverse quantization part for inverse-quantizing the input signal; and a fourth inverse quantization part disposed in parallel to the second inverse quantization part, and the third quantization part and the fourth quantization part share a codebook.
According to an exemplary embodiment, when a speech or audio signal is quantized by classifying the speech or audio signal into a plurality of coding modes according to a signal characteristic of speech or audio and allocating a various number of bits according to a compression ratio applied to each coding mode, the speech or audio signal may be more efficiently quantized by designing a quantizer having good performance at a low bit rate.
In addition, a used amount of a memory may be minimized by sharing a codebook of some quantizers when a quantization device for providing various bit rates is designed.
These and/or other aspects will become apparent and more readily appreciated from the following description of the exemplary embodiments, taken in conjunction with the accompanying drawings in which:
The inventive concept may allow various kinds of change or modification and various changes in form, and specific embodiments will be illustrated in drawings and described in detail in the specification. However, it should be understood that the specific embodiments do not limit the inventive concept to a specific disclosing form but include every modified, equivalent, or replaced one within the spirit and technical scope of the inventive concept. In the description of the inventive concept, when it is determined that a specific description of relevant well-known features may obscure the essentials of the inventive concept, a detailed description thereof is omitted.
Although terms, such as ‘first’ and ‘second’, can be used to describe various elements, the elements cannot be limited by the terms. The terms can be used to classify a certain element from another element.
The terminology used in the application is used only to describe specific embodiments and does not have any intention to limit the inventive concept. The terms used in this specification are those general terms currently widely used in the art, but the terms may vary according to the intention of those of ordinary skill in the art, precedents, or new technology in the art. Also, specified terms may be selected by the applicant, and in this case, the detailed meaning thereof will be described in the detailed description. Thus, the terms used in the specification should be understood not as simple names but based on the meaning of the terms and the overall description.
An expression in the singular includes an expression in the plural unless they are clearly different from each other in context. In the application, it should be understood that terms, such as ‘include’ and ‘have’, are used to indicate the existence of an implemented feature, number, step, operation, element, part, or a combination thereof without excluding in advance the possibility of the existence or addition of one or more other features, numbers, steps, operations, elements, parts, or combinations thereof.
Hereinafter, embodiments of the inventive concept will be described in detail with reference to the accompanying drawings, and like reference numerals in the drawings denote like elements, and thus their repetitive description will be omitted.
In general, a trellis coded quantizer (TCQ) quantizes an input vector by allocating one element to each TCQ stage, whereas a trellis coded vector quantizer (TCVQ) uses a structure of generating sub-vectors by dividing an entire input vector into sub-vectors and then allocating each sub-vector to a TCQ stage. When a quantizer is formed using one element, a TCQ is formed, and when a quantizer is formed using a sub-vector by combining a plurality of elements, a TCVQ is formed. Therefore, when a two-dimensional (2D) sub-vector is used, a total number of TCQ stages are the same size as obtained by dividing a size of an input vector by 2. Commonly, a speech/audio codec encodes an input signal in a frame unit, and a line spectral frequency (LSF) coefficient is extracted for each frame. An LSF coefficient has a vector form, and a dimension of 10 or 16 is used for the LSF coefficient. In this case, when considering a 2D TCVQ, the number of sub-vectors is 5 or 8.
A sound coding apparatus 100 shown in
Referring to
The LPC coefficient quantization unit 130 may quantize an LPC coefficient by using a quantizer corresponding to the selected coding mode and determine a quantization index representing the quantized LPC coefficient. The LPC coefficient quantization unit 130 may perform quantization by converting the LPC coefficient into another coefficient suitable for the quantization.
The excitation signal coding unit 150 may perform excitation signal coding according to the selected coding mode. For the excitation signal coding, a code-excited linear prediction (CELP) or algebraic CELP (ACELP) algorithm may be used. Representative parameters for encoding an LPC coefficient by a CELP scheme are an adaptive codebook index, an adaptive codebook gain, a fixed codebook index, a fixed codebook gain, and the like. The excitation signal coding may be carried out based on a coding mode corresponding to a characteristic of an input signal. For example, four coding modes, i.e., an unvoiced coding (UC) mode, a voiced coding (VC) mode, a generic coding (GC) mode, and a transition coding (TC) mode, may be used. The UC mode may be selected when a speech signal is an unvoiced sound or noise having a characteristic that is similar to that of the unvoiced sound. The VC mode may be selected when a speech signal is a voiced sound. The TC mode may be used when a signal of a transition period in which a characteristic of a speech signal sharply varies is encoded. The GC mode may be used to encode the other signals. The UC mode, the VC mode, the TC mode, and the GC mode follow the definition and classification criterion drafted in ITU-T G.718 but is not limited thereto. The excitation signal coding unit 150 may include an open-loop pitch search unit (not shown), a fixed codebook search unit (not shown), or a gain quantization unit (not shown), but components may be added to or omitted from the excitation signal coding unit 150 according to a coding mode. For example, in the VC mode, all the components described above are included, and in the UC mode, the open-loop pitch search unit is not used. The excitation signal coding unit 150 may be simplified in the GC mode and the VC mode when the number of bits allocated to quantization is large, i.e., in the case of a high bit rate. That is, by including the UC mode and the TC mode in the GC mode, the GC mode may be used for the UC mode and the TC mode. In the case of a high bit rate, an inactive coding (IC) mode and an audio coding (AC) mode may be further included. The excitation signal coding unit 150 may classify a coding mode into the GC mode, the UC mode, the VC mode, and the TC mode when the number of bits allocated to quantization is small, i.e., in the case of a low bit rate. In the case of a low bit rate, the IC mode and the AC mode may be further included. The IC mode may be selected for mute, and the AC mode may be selected when a characteristic of a speech signal is close to audio.
The coding mode may be further subdivided according to a bandwidth of a speech signal. The bandwidth of a speech signal may be classified into, for example, a narrowband (NB), a wideband (WB), a super wideband (SWB), and a full band (FB). The NB may have a bandwidth of 300-3400 Hz or 50-4000 Hz, the WB may have a bandwidth of 50-7000 Hz or 50-8000 Hz, the SWB may have a bandwidth of 50-14000 Hz or 50-16000 Hz, and the FB may have a bandwidth up to 20000 Hz. Herein, the numeric values related to the bandwidths are set for convenience and are not limited thereto. In addition, the classification of the bandwidth may also be set to be simpler or more complex.
When the types and number of coding modes are determined, it is necessary that a codebook is trained again using a speech signal corresponding to a determined coding mode.
The excitation signal coding unit 150 may additionally use a transform coding algorithm according to a coding mode. An excitation signal may be encoded in a frame or subframe unit.
A sound coding apparatus 200 shown in
Referring to
The LP analysis unit 220 may extract an LPC coefficient by performing an LP analysis on the pre-processed speech signal. In general, one LP analysis per frame is performed, but two or more LP analyses per frame may be performed for additional sound quality enhancement. In this case, one analysis is an LP for a frame-end, which is an existing LP analysis, and the other analyses may be LPs for a mid-subframe to enhance sound quality. Herein, a frame-end of a current frame indicates the last subframe among subframes constituting the current frame, and a frame-end of a previous frame indicates the last subframe among subframes constituting the previous frame. The mid-subframe indicates one or more subframes among subframes existing between the last subframe which is the frame-end of the previous frame and the last subframe which is the frame-end of the current frame. For example, one frame may consist of four subframes. A dimension of 10 is used for an LPC coefficient when an input signal is an NB, and a dimension of 16-20 is used for an LPC coefficient when an input signal is a WB, but the embodiment is not limited thereto.
The weighted-signal calculation unit 230 may receive the pre-processed speech signal and the extracted LPC coefficient and calculate a perceptual weighting filtered signal based on a perceptual weighting filter. The perceptual weighting filter may reduce quantization noise of the pre-processed speech signal within a masking range in order to use a masking effect of a human auditory structure.
The open-loop pitch search unit 240 may search an open-loop pitch by using the perceptual weighting filtered signal.
The signal analysis and VAD unit 250 may determine whether the input signal is an active speech signal by analyzing various characteristics including the frequency characteristic of the input signal.
The encoding unit 260 may determine a coding mode of the current frame by using a signal characteristic, VAD information or a coding mode of the previous frame, quantize an LPC coefficient by using a quantizer corresponding to the selected coding mode, and encode an excitation signal according to the selected coding mode. The encoding unit 260 may include the components shown in
The memory update unit 270 may store the encoded current frame and parameters used during encoding for encoding of a subsequent frame.
The parameter coding unit 280 may encode parameters to be used for decoding at a decoding end and include the encoded parameters in a bitstream. Preferably, parameters corresponding to a coding mode may be encoded. The bitstream generated by the parameter coding unit 280 may be used for the purpose of storage or transmission.
Table 1 below shows an example of a quantization scheme and structure for four coding modes. A scheme of performing quantization without an inter-frame prediction can be named a safety-net scheme, and a scheme of performing quantization with an inter-frame prediction can be named a predictive scheme. In addition, a VQ stands for a vector quantizer, and a BC-TCQ stands for a block-constrained trellis coded quantizer.
A BC-TCVQ stands for a block-constrained trellis coded vector quantizer. A TCVQ allows a vector codebook and a branch label by generalizing a TCQ. Main features of the TCVQ are to partition VQ symbols of an expanded set into subsets and to label trellis branches with these subsets. The TCVQ is based on a rate ½ convolution code, which has N=2v trellis states, and has two branches entering and leaving each trellis state. When M source vectors are given, a minimum distortion path is searched for using a Viterbi algorithm. As a result, a best trellis path may begin in any of N initial states and end in any of N terminal states. A codebook in the TCVQ has 2(R+R′)L vector codewords. Herein, since the codebook has 2R′L times as many codewords as a nominal rate R VQ, R′ may be a codebook expansion factor. An encoding operation is simply described as follows. First, for each input vector, distortion corresponding to the closest codeword in each subset is searched for, and a minimum distortion path through a trellis is searched for using the Viterbi algorithm by putting, as searched distortion, a branch metric for a branch labeled to a subset S. Since the BC-TCVQ requires one bit for each source sample to designate a trellis path, the BC-TCVQ has low complexity. A BC-TCVQ structure may have 2k initial trellis states and 2v-k terminal states for each allowed initial trellis state when 0≤k≤v. Single Viterbi encoding starts from an allowed initial trellis state and ends at a vector stage m-k. To specify an initial state, k bits are required, and to designate a path to the vector stage m-k, m-k bits are required. The unique terminating path depending on an initial trellis state is pre-specified for each trellis state at the vector stage m-k through a vector stage m. Regardless of a value of k, m bits are required to specify an initial trellis state and a path through a trellis.
A BC-TCVQ for the VC mode at an internal sampling frequency of 16 KHz may use 16-state and 8-stage TCVQ having a 2D vector. LSF sub-vectors having two elements may be allocated to each stage. Table 2 below shows initial states and terminal states for a 16-state BC-TCVQ. Herein, k and v denotes 2 and 4, respectively, and four bits for an initial state and a terminal state are used.
A coding mode may vary according to an applied bit rate. As described above, to quantize an LPC coefficient at a high bit rate using two coding modes, 40 or 41 bits for each frame may be used in the GC mode, and 46 bits for each frame may be used in the TC mode.
An LPC coefficient quantization unit 300 shown in
Referring to
The weighting function determination unit 330 may determine a weighting function for the ISF/LSF quantization unit 350 by using the ISF coefficient or the LSF coefficient converted from the LPC coefficient. The determined weighting function may be used in an operation of selecting a quantization path or a quantization scheme or searching for a codebook index with which a weighted error is minimized in quantization. For example, the weighting function determination unit 330 may determine a final weighting function by combining a magnitude weighting function, a frequency weighting function and a weighting function based on a position of the ISF/LSF coefficient.
In addition, the weighting function determination unit 330 may determine a weighting function by taking into account at least one of a frequency bandwidth, a coding mode, and spectrum analysis information. For example, the weighting function determination unit 330 may derive an optimal weighting function for each coding mode. Alternatively, the weighting function determination unit 330 may derive an optimal weighting function according to a frequency bandwidth of a speech signal. Alternatively, the weighting function determination unit 330 may derive an optimal weighting function according to frequency analysis information of a speech signal. In this case, the frequency analysis information may include spectral tilt information. The weighting function determination unit 330 is described in detail below.
The ISF/LSF quantization unit 350 may obtain an optimal quantization index according to an input coding mode. In detail, the ISF/LSF quantization unit 350 may quantize the ISF coefficient or the LSF coefficient converted from the LPC coefficient of the frame-end of the current frame. When an input signal is the UC mode or the TC mode corresponding to a non-stationary signal, the ISF/LSF quantization unit 350 may quantize the input signal by only using the safety-net scheme without an inter-frame prediction, and when an input signal is the VC mode or the GC mode corresponding to a stationary signal, the ISF/LSF quantization unit 350 may determine an optimal quantization scheme in consideration of a frame error by switching the predictive scheme and the safety-net scheme.
The ISF/LSF quantization unit 350 may quantize the ISF coefficient or the LSF coefficient by using the weighting function determined by the weighting function determination unit 330. The ISF/LSF quantization unit 350 may quantize the ISF coefficient or the LSF coefficient by using the weighting function determined by the weighting function determination unit 330 to select one of a plurality of quantization paths. An index obtained as a result of the quantization may be used to obtain the quantized ISF (QISF) coefficient or the quantized LSF (QLSF) coefficient through an inverse quantization operation.
The second coefficient conversion unit 370 may convert the QISF coefficient or the QLSF coefficient into a quantized LPC (QLPC) coefficient.
Hereinafter, a relationship between vector quantization of LPC coefficients and a weighting function is described.
The vector quantization indicates an operation of selecting a codebook index having the least error by using a squared error distance measure based on the consideration that all entries in a vector have the same importance. However, for the LPC coefficients, since all the coefficients have different importance, when errors of important coefficients are reduced, perceptual quality of a finally synthesized signal may be improved. Therefore, when the LSF coefficients are quantized, a decoding apparatus may select an optimal codebook index by applying a weighting function representing the importance of each LPC coefficient to a squared error distance measure, thereby improving the performance of a synthesized signal.
According to an embodiment, a magnitude weighting function about what is actually affected to a spectral envelope by each ISF or LSF may be determined using frequency information of the ISF and the LSF and an actual spectral magnitude. According to an embodiment, additional quantization efficiency may be obtained by combining a frequency weighting function in which a perceptual characteristic of a frequency domain and a formant distribution are considered and the magnitude weighting function. In this case, since an actual magnitude in the frequency domain is used, envelope information of whole frequencies may be well reflected, and a weight of each ISF or LSF coefficient may be accurately derived. According to an embodiment, additional quantization efficiency may be obtained by combining a weighting function based on position information of LSF coefficients or ISF coefficients with the magnitude weighting function and the frequency weighting function.
According to an embodiment, when an ISF or an LSF converted from an LPC coefficient is vector-quantized, if the importance of each coefficient is different, a weighting function indicating which entry is relatively more important in a vector may be determined. In addition, by determining a weighting function capable of assigning a higher weight to a higher-energy portion by analyzing a spectrum of a frame to be encoded, accuracy of the encoding may be improved. High energy in a spectrum indicates a high correlation in a time domain.
In Table 1, an optimal quantization index for a VQ applied to all modes may be determined as an index for minimizing Ewerr(p) of Equation 1.
In Equation 1, w(i) denotes a weighting function, r(i) denotes an input of a quantizer, and c(i) denotes an output of the quantizer and is to obtain an index for minimizing weighted distortion between two values.
Next, a distortion measure used by a BC-TCQ basically follows a method disclosed in U.S. Pat. No. 7,630,890. In this case, a distortion measure d(x, y) may be represented by Equation 2.
According to an embodiment, a weighting function may be applied to the distortion measure d(x, y). Weighted distortion may be obtained by extending a distortion measure used for a BC-TCQ in U.S. Pat. No. 7,630,890 to a measure for a vector and then applying a weighting function to the extended measure. That is, an optimal index may be determined by obtaining weighted distortion as represented in Equation 3 below at all stages of a BC-TCVQ.
The ISF/LSF quantization unit 350 may perform quantization according to an input coding mode, for example, by switching a lattice vector quantizer (LVQ) and a BC-TCVQ. If a coding mode is the GC mode, the LVQ may be used, and if the coding mode is the VC mode, the BC-TCVQ may be used. An operation of selecting a quantizer when the LVQ and the BC-TCVQ are mixed is described as follows. First, bit rates for encoding may be selected. After selecting the bit rates for encoding, bits for an LPC quantizer corresponding to each bit rate may be determined. Thereafter, a bandwidth of an input signal may be determined. A quantization scheme may vary according to whether the input signal is an NB or a WB. In addition, when the input signal is a WB, it is necessary that it is additionally determined whether an upper limit of a bandwidth to be actually encoded is 6.4 KHz or 8 KHz. That is, since a quantization scheme may vary according to whether an internal sampling frequency is 12.8 KHz or 16 KHz, it is necessary to check a bandwidth. Next, an optimal coding mode within a limit of usable coding modes may be determined according to the determined bandwidth. For example, four coding modes (the UC, the VC, the GC, and the TC) are usable, but only three modes (the VC, the GC, and the TC) may be used at a high bit rate (for example, 9.6 Kbit/s or above). A quantization scheme, e.g., one of the LVQ and the BC-TCVQ, is selected based on a bit rate for encoding, a bandwidth of an input signal, and a coding mode, and an index quantized based on the selected quantization scheme is output.
According to an embodiment, it is determined whether a bit rate corresponds to between 24.4 Kbps and 65 Kbps, and if the bit rate does not correspond to between 24.4 Kbps and 65 Kbps, the LVQ may be selected. Otherwise, if the bit rate corresponds to between 24.4 Kbps and 65 Kbps, it is determined whether a bandwidth of an input signal is an NB, and if the bandwidth of the input signal is an NB, the LVQ may be selected. Otherwise, if the bandwidth of the input signal is not an NB, it is determined whether a coding mode is the VC mode, and if the coding mode is the VC mode, the BC-TCVQ may be used, and if the coding mode is not the VC mode, the LVQ may be used.
According to another embodiment, it is determined whether a bit rate corresponds to between 13.2 Kbps and 32 Kbps, and if the bit rate does not correspond to between 13.2 Kbps and 32 Kbps, the LVQ may be selected. Otherwise, if the bit rate corresponds to between 13.2 Kbps and 32 Kbps, it is determined whether a bandwidth of an input signal is a WB, and if the bandwidth of the input signal is not a WB, the LVQ may be selected. Otherwise, if the bandwidth of the input signal is a WB, it is determined whether a coding mode is the VC mode, and if the coding mode is the VC mode, the BC-TCVQ may be used, and if the coding mode is not the VC mode, the LVQ may be used.
According to an embodiment, an encoding apparatus may determine an optimal weighting function by combining a magnitude weighting function using a spectral magnitude corresponding to a frequency of an ISF coefficient or an LSF coefficient converted from an LPC coefficient, a frequency weighting function in which a perceptual characteristic of an input signal and a formant distribution are considered, a weighting function based on positions of LSF coefficients or ISF coefficients.
A weighting function determination unit 400 shown in
Referring to
The LP analysis unit 430 may generate an LPC coefficient by LP-analyzing the input signal. The LP analysis unit 430 may generate an ISF or LSF coefficient from the LPC coefficient.
The first weighting function generation unit 450 may obtain a magnitude weighting function and a frequency weighting function based on spectrum analysis information of the ISF or LSF coefficient and generate a first weighting function by combining the magnitude weighting function and the frequency weighting function. The first weighting function may be obtained based on FFT, and a large weight may be allocated as a spectral magnitude is large. For example, the first weighting function may be determined by normalizing the spectrum analysis information, i.e., spectral magnitudes, so as to meet an ISF or LSF band and then using a magnitude of a frequency corresponding to each ISF or LSF coefficient.
The second weighting function generation unit 470 may determine a second weighting function based on interval or position information of adjacent ISF or LSF coefficients. According to an embodiment, the second weighting function related to spectrum sensitivity may be generated from two ISF or LSF coefficients adjacent to each ISF or LSF coefficient. Commonly, ISF or LSF coefficients are located on a unit circle of a Z-domain and are characterized in that when an interval between adjacent ISF or LSF coefficients is narrower than that of the surroundings, a spectral peak appears. As a result, the second weighting function may be used to approximate spectrum sensitivity of LSF coefficients based on positions of adjacent LSF coefficients. That is, by measuring how close adjacent LSF coefficients are located, a density of the LSF coefficients may be predicted, and since a signal spectrum may have a peak value near a frequency at which dense LSF coefficients exist, a large weight may be allocated. Herein, to increase accuracy when the spectrum sensitivity is approximated, various parameters for the LSF coefficients may be additionally used when the second weighting function is determined.
As described above, an interval between ISF or LSF coefficients and a weighting function may have an inverse proportional relationship. Various embodiments may be carried out using this relationship between an interval and a weighting function. For example, an interval may be represented by a negative value or represented as a denominator. As another example, to further emphasis an obtained weight, each element of a weighting function may be multiplied by a constant or represented as a square of the element. As another example, a weighting function secondarily obtained by performing an additional computation, e.g., a square or a cube, of a primarily obtained weighting function may be further reflected.
An example of deriving a weighting function by using an interval between ISF or LSF coefficients is as follows.
According to an embodiment, a second weighting function Ws(n) may be obtained by Equation 4 below.
In Equation 4, Isfi−1 and Isfi+1 denote LSF coefficients adjacent to a current LSF coefficient.
According to another embodiment, the second weighting function Ws(n) may be obtained by Equation 5 below.
In Equation 5, lsfn denotes a current LSF coefficient, Isfn−1 and Isfn+1 denote adjacent LSF coefficients, and M is a dimension of an LP model and may be 16. For example, since LSF coefficients span between 0 and π, first and last weights may be calculated based on Isf0=0 and IsfM=π.
The combination unit 490 may determine a final weighting function to be used to quantize an LSF coefficient by combining the first weighting function and the second weighting function. In this case, as a combination scheme, various schemes, such as a scheme of multiplying the first weighting function and the second weighting function, a scheme of multiplying each weighting function by a proper ratio and then adding the multiplication results, and a scheme of multiplying each weight by a value predetermined using a lookup table or the like and then adding the multiplication results, may be used.
A first weighting function generation unit 500 shown in
Referring to
The magnitude weighting function generation unit 530 may generate a magnitude weighting function W1(n) based on spectrum analysis information for the normalized LSF coefficient. According to an embodiment, the magnitude weighting function may be determined based on a spectral magnitude of the normalized LSF coefficient.
In detail, the magnitude weighting function may be determined using a spectral bin corresponding to a frequency of the normalized LSF coefficient and two neighboring spectral bins located at the left and the right of, e.g., one previous or subsequent to, a corresponding spectral bin. Each magnitude weighting function W1(n) related to a spectral envelope may be determined based on Equation 6 below by extracting a maximum value among magnitudes of three spectral bins.
W
1(n)=(√{square root over (wj(n)−Min)})+2, for n=0, . . . ,M−1 [Equation 6]
In Equation 6, Min denotes a minimum value of wf(n), and wf(n) may be defined by 10log(Emax(n)) (herein, n=0, . . . , M−1). Herein, M denotes 16, and Emax(n) denotes a maximum value among magnitudes of three spectral bins for each LSF coefficient.
The frequency weighting function generation unit 550 may generate a frequency weighting function W2(n) based on frequency information for the normalized LSF coefficient. According to an embodiment, the frequency weighting function may be determined using a perceptual characteristic of an input signal and a formant distribution. The frequency weighting function generation unit 550 may extract the perceptual characteristic of the input signal according to a bark scale. In addition, the frequency weighting function generation unit 550 may determine a weighting function for each frequency based on a first formant of a distribution of formants. The frequency weighting function may exhibit a relatively low weight at a very low frequency and a high frequency and exhibit the same sized weight in a certain frequency period, e.g., a period corresponding to a first formant, at a low frequency. The frequency weighting function generation unit 550 may determine the frequency weighting function according to an input bandwidth and a coding mode.
The combination unit 570 may determine an FFT-based weighting function Wf(n) by combining the magnitude weighting function W1(n) and the frequency weighting function W2(n). The combination unit 570 may determine a final weighting function by multiplying or adding the magnitude weighting function and the frequency weighting function. For example, the FFT-based weighting function Wf(n) for frame-end LSF quantization may be calculated based on Equation 7 below.
W
f(n)
=W
1(n)·W2(n), for n=0, . . . ,M−1 [Equation 7]
An LPC coefficient quantization unit 600 shown in
Referring to
The first quantization module 630 may quantize an input signal provided through the selection unit 610 when the quantization without an inter-frame prediction is selected.
The second quantization module 650 may quantize an input signal provided through the selection unit 610 when the quantization with an inter-frame prediction is selected.
The first quantization module 630 may perform quantization without an inter-frame prediction and may be named the safety-net scheme. The second quantization module 650 may perform quantization with an inter-frame prediction and may be named the predictive scheme.
Accordingly, an optimal quantizer may be selected in correspondence with various bit rates from a low bit rate for a highly efficient interactive voice service to a high bit rate for providing a service of differentiated quality.
A selection unit 700 shown in
Referring to
First, a weighted AR prediction error using a quantized signal z(n) of a previous frame may be represented by Equation 8 below.
Second, an AR prediction error using the quantized signal z(n) of the previous frame may be represented by Equation 9 below.
Third, a weighted AR prediction error using a signal z(n) of the previous frame may be represented by Equation 10 below.
Fourth, an AR prediction error using the signal z(n) of the previous frame may be represented by Equation 11 below.
Herein, M denotes a dimension of an LSF, and when a bandwidth of an input speech signal is a WB, 16 is commonly used for M, and p(i) denotes a predicted coefficient of the AR method. As described above, a case in which information about an immediately previous frame is used is usual, and a quantization scheme may be determined using a prediction error obtained as described above.
If a prediction error is greater than a predetermined threshold, this may suggest that a current frame tends to be non-stationary. In this case, the safety-net scheme may be used. Otherwise, the predictive scheme is used, and in this case, it may be restrained such that the predictive scheme is not continuously selected.
According to an embodiment, to prepare for a case in which information about a previous frame does not exist due to the occurrence of a frame error on the previous frame, a second prediction error may be obtained using a previous frame of the previous frame, and a quantization scheme may be determined using the second prediction error. In this case, compared with the first case described above, the second prediction error may be represented by Equation 12 below.
The quantization scheme selection unit 730 may determine a quantization scheme for a current frame by using the prediction error obtained by the prediction error calculation unit 710. In this case, the coding mode obtained by the coding mode determination unit (110 of
Referring to
Otherwise, as a result of the determination in operation 810, if the prediction mode is not 0, one of the safety-net scheme and the predictive scheme may be determined as a quantization scheme in consideration of a prediction error. To this end, in operation 830, it is determined whether the prediction error is greater than a predetermined threshold. Herein the threshold may be determined in advance through experiments or simulations. For example, for a WB of which a dimension is 16, the threshold may be determined as, for example, 3,784,536.3. However, it may be restrained such that the predictive scheme is not continuously selected.
As a result of the determination in operation 830, if the prediction error is greater than or equal to the threshold, the safety-net scheme may be selected in operation 850. Otherwise, as a result of the determination in operation 830, if the prediction error is less than the threshold, the predictive scheme may be selected in operation 870.
A first quantization module 900 shown in
An operation of the first quantizer 911 and the second quantizer 913 is as follows.
First, a signal z(n) may be obtained by removing a previously defined mean value from a un-quantized LSF coefficient. The first quantizer 911 may quantize or inverse-quantize an entire vector of the signal z(n). A quantizer used herein may be, for example, a BC-TCQ or a BC-TCVQ. To obtain a quantization error signal, a signal r(n) may be obtained using a difference value between the signal z(n) and an inverse-quantized signal. The signal r(n) may be provided as an input of the second quantizer 913. The second quantizer 913 may be implemented using an SVQ, an MSVQ, or the like. A signal quantized by the second quantizer 913 becomes a quantized value z(n) after being inverse-quantized and then added to a result inverse-quantized by the first quantizer 911, and a quantized LSF value may be obtained by adding the mean value to the quantized value z(n).
The first quantization module 900 shown in
An intra-frame prediction operation of the TCQ is as follows. An input signal tj(n) of the first quantizer 931, i.e., a first TCQ, may be obtained by Equation 13 below.
t
j(n)=rj(n)−ρj{circumflex over (r)}j-1(n),j=1, . . . ,M−1
{circumflex over (r)}
j-1(n)={circumflex over (t)}j-1(n)+ρj{circumflex over (r)}j-2(n),j=2, . . . ,M [Equation 13]
However, an intra-frame prediction operation of the TCVQ using 2D is as follows. The input signal tj(n) of the first quantizer 931, i.e., the first TCQ, may be obtained by Equation 14 below.
t
j(n)=rj(n)−Aj{circumflex over (r)}j-1(n),j=1, . . . ,M/2−1
{circumflex over (r)}
j-1(n)={circumflex over (t)}j-1(n)+Aj{circumflex over (r)}j-2(n),j=2, . . . ,M/2 [Equation 14]
Herein, M denotes a dimension of an LSF coefficient and uses 10 for an NB and 16 for a WB, ρj denotes a 1D prediction coefficient, and Aj denotes a 2×2 prediction coefficient.
The first quantizer 931 may quantize a prediction error vector t(n). According to an embodiment, the first quantizer 931 may be implemented using a TCQ, in detail, a BC-TCQ, a BC-TCVQ, a TCQ, or a TCVQ. The intra-frame predictor 932 used together with the first quantizer 931 may repeat a quantization operation and a prediction operation in an element unit or a sub-vector unit of an input vector. An operation of the second quantizer 933 is the same as that of the second quantizer 913 of
A second quantization module 10000 shown in
The second quantization module 10000 shown in
A quantizer of a switching structure mat be implemented by combining the quantizer forms of various structures, which have been described with reference to
The selection unit 1210 may select one of the safety-net scheme and the predictive scheme as a quantization scheme based on a prediction error.
The first quantization module 1230 performs quantization without an inter-frame prediction when the safety-net scheme is selected and may include a first quantizer 1231 and a first intra-frame predictor 1232. In detail, an LSF vector may be quantized to 30 bits by the first quantizer 1231 and the first intra-frame predictor 1232.
The second quantization module 1250 performs quantization with an inter-frame prediction when the predictive scheme is selected and may include a second quantizer 1251, a second intra-frame predictor 1252, and an inter-frame predictor 1253. In detail, a prediction error corresponding to a difference between an LSF vector from which a mean value has been removed and a prediction vector may be quantized to 30 bits by the second quantizer 1251 and the second intra-frame predictor 1252.
The quantization apparatus shown in
When the safety-net scheme is selected by the selection unit 1210, an entire input vector of an LSF coefficient z(n) from which the mean value has been removed may be quantized through the first intra-frame predictor 1232 and using the first quantizer 1231 using 30 bits. However, when the predictive scheme is selected by the selection unit 1210, a prediction error signal obtained using the inter-frame predictor 1253 from the LSF coefficient z(n) from which the mean value has been removed may be quantized through the second intra-frame predictor 1252 and using the second quantizer 1251 using 30 bits. The first and second quantizers 1231 and 1251 may be, for example, quantizers having a form of a TCQ or a TCVQ. In detail, a BC-TCQ, a BC-TCVQ, or the like may be used. In this case, a quantizer uses a total of 31 bits. A quantized result is used as an output of a quantizer of a low rate, and main outputs of the quantizer are a quantized LSF vector and a quantization index.
The selection unit 1310 may select one of the safety-net scheme and the predictive scheme as a quantization scheme based on a prediction error.
The first quantization module 1330 may perform quantization without an inter-frame prediction when the safety-net scheme is selected and may include the first quantizer 1331, the first intra-frame predictor 1332, and the third quantizer 1333.
The second quantization module 1350 may perform quantization with an inter-frame prediction when the predictive scheme is selected and may include the second quantizer 1351, a second intra-frame predictor 1352, the fourth quantizer 1353, and an inter-frame predictor 1354.
The quantization apparatus shown in
When the safety-net scheme is selected by the selection unit 1310, an entire input vector of an LSF coefficient z(n) from which the mean value has been removed may be quantized and inverse-quantized through the first intra-frame predictor 1332 and the first quantizer 1331 using 30 bits. A second error vector indicating a difference between an original signal and the inverse-quantized result may be provided as an input of the third quantizer 1333. The third quantizer 1333 may quantize the second error vector by using 10 bits. The third quantizer 1333 may be, for example, an SQ, a VQ, an SVQ, or an MSVQ. After the quantization and the inverse quantization, a finally quantized vector may be stored for a subsequent frame.
However, when the predictive scheme is selected by the selection unit 1310, a prediction error signal obtained by subtracting p(n) of the inter-frame predictor 1354 from the LSF coefficient z(n) from which the mean value has been removed may be quantized or inverse-quantized by the second quantizer 1351 using 30 bits and the second intra-frame predictor 1352. The first and second quantizers 1331 and 1351 may be, for example, quantizers having a form of a TCQ or a TCVQ. In detail, a BC-TCQ, a BC-TCVQ, or the like may be used. A second error vector indicating a difference between an original signal and the inverse-quantized result may be provided as an input of the fourth quantizer 1353. The fourth quantizer 1353 may quantize the second error vector by using 10 bits. Herein, the second error vector may be divided into two 8×8-dimension sub-vectors and then quantized by the fourth quantizer 1353. Since a low band is more important that a high band in terms of perception, the second error vector may be encoded by allocating a different number of bits to a first VQ and a second VQ. The fourth quantizer 1353 may be, for example, an SQ, a VQ, an SVQ, or an MSVQ. After the quantization and the inverse quantization, a finally quantized vector may be stored for a subsequent frame.
In this case, a quantizer uses a total of 41 bits. A quantized result is used as an output of a quantizer of a high rate, and main outputs of the quantizer are a quantized LSF vector and a quantization index.
As a result, when both
When the safety-net scheme is selected by the selection unit 1410, an LSF coefficient z(n) from which the mean value has been removed may be quantized by the first quantizer 1431. The first quantizer 1431 may use an intra-frame prediction for high performance or may not use the intra-frame prediction for low complexity as described with reference to
When the predictive scheme is selected by the selection unit 1410, the LSF coefficient z(n) from which the mean value has been removed may be provided to the second quantizer 1451 for quantizing a prediction error signal, which is obtained using inter-frame prediction, by using a TCQ or a TCVQ through the intra-frame prediction. The first and second quantizers 1431 and 1451 may be, for example, quantizers having a form of a TCQ or a TCVQ. In detail, a BC-TCQ, a BC-TCVQ, or the like may be used. A quantized result is used as an output of a quantizer of a low rate.
However, when the predictive scheme is selected by the selection unit 1510, the second quantizer 1551 performs quantization and inverse quantization, and a second error vector indicating a difference between an original signal and an inverse-quantized result may be provided as an input of the fourth quantizer 1552. The fourth quantizer 1552 may quantize the second error vector. The fourth quantizer 1552 may be, for example, an SQ, a VQ, an SVQ, or an MSVQ. After the quantization and inverse quantization, a finally quantized vector may be stored for a subsequent frame.
An LPC coefficient quantization unit 1600 shown in
Referring to
In the second quantization module 1730, the second quantizer 1731 may quantize a prediction error signal by using a BC-TCVQ or a BC-TCQ through the second intra-frame predictor 1732. The fourth quantizer 1733 may quantize a quantization error signal by using a VQ.
The selection unit 1750 may select one of an output of the first quantization module 1710 and an output of the second quantization module 1730.
In
Referring to
Herein, when a prediction mode (predmode) is 0, this indicates a mode in which the safety-net scheme is always used, and when the prediction mode (predmode) is not 0, this indicates that the safety-net scheme and the predictive scheme are switched and used. An example of a mode in which the safety-net scheme is always used may be the TC or UC mode. In addition, WDist[0] denotes weighted distortion of the safety-net scheme, and WDist[1] denotes weighted distortion of the predictive scheme. In addition, abs_threshold denotes a preset threshold. When the prediction mode is not 0, an optimal quantization scheme may be selected by giving a higher priority to the weighted distortion of the safety-net scheme in consideration of a frame error. That is, basically, if a value of WDist[0] is less than the pre-defined threshold, the safety-net scheme may be selected regardless of a value of WDist[1]. Even in the other cases, instead of simply selecting less weighted distortion, for the same weighted distortion, the safety-net scheme may be selected because the safety-net scheme is more robust against a frame error. Therefore, only when WDist[0] is greater than PREFERSFNET*WDist[1], the predictive scheme may be selected. Herein, usable PREFERSFNET=1.15 but is not limited thereto. By doing this, when a quantization scheme is selected, bit information indicating the selected quantization scheme and a quantization index obtained by performing quantization using the selected quantization scheme may be transmitted.
An inverse quantization apparatus 1900 shown in
Referring to
The first inverse quantization module 1930 may inverse-quantize the encoded LPC parameter without an inter-frame prediction.
The second inverse quantization module 1950 may inverse-quantize the encoded LPC parameter with an inter-frame prediction.
The first inverse quantization module 1930 and the second inverse quantization module 1950 may be implemented based on inverse processing of the first and second quantization modules of each of the various embodiments described above according to an encoding apparatus corresponding to a decoding apparatus.
The inverse quantization apparatus of
The VC mode in a 16-KHz internal sampling frequency may have two decoding rates of, for example, 31 bits per frame or 40 or 41 bits per frame. The VC mode may be decoded by a 16-state 8-stage BC TCVQ.
Referring to
When the quantization scheme information indicates the safety-net scheme, the first inverse quantizer 2031 of the first inverse quantization module 2030 may perform inverse quantization by using a BC-TCVQ. A quantized LSF coefficient may be obtained through the first inverse quantizer 2031 and the first intra-frame predictor 2032. A finally decoded LSF coefficient is generated by adding a mean value that is a predetermined DC value to the quantized LSF coefficient.
However, when the quantization scheme information indicates the predictive scheme, the second inverse quantizer 2051 of the second inverse quantization module 2050 may perform inverse quantization by using a BC-TCVQ. An inverse quantization operation starts from the lowest vector among LSF vectors, and the intra-frame predictor 2052 generates a prediction value for a vector element of a next order by using a decoded vector. The inter-frame predictor 2053 generates a prediction value through a prediction between frames by using an LSF coefficient decoded in a previous frame. A finally decoded LSF coefficient is generated by adding an inter-frame prediction value obtained by the inter-frame predictor 2053 to a quantized LSF coefficient obtained through the second inverse quantizer 2051 and the intra-frame predictor 2052 and then adding a mean value that is a predetermined DC value to the addition result.
Referring to
When the quantization scheme information indicates the safety-net scheme, the first inverse quantizer 2131 of the first inverse quantization module 2130 may perform inverse quantization by using a BC-TCVQ. The third inverse quantizer 2133 may perform inverse quantization by using an SVQ. A quantized LSF coefficient may be obtained through the first inverse quantizer 2131 and the first intra-frame predictor 2132. A finally decoded LSF coefficient is generated by adding a quantized LSF coefficient obtained by the third inverse quantizer 2133 to the quantized LSF coefficient and then adding a mean value that is a predetermined DC value to the addition result.
However, when the quantization scheme information indicates the predictive scheme, the second inverse quantizer 2151 of the second inverse quantization module 2150 may perform inverse quantization by using a BC-TCVQ. An inverse quantization operation starts from the lowest vector among LSF vectors, and the second intra-frame predictor 2152 generates a prediction value for a vector element of a next order by using a decoded vector. The fourth inverse quantizer 2153 may perform inverse quantization by using an SVQ. A quantized LSF coefficient provided from the fourth inverse quantizer 2153 may be added to a quantized LSF coefficient obtained through the second inverse quantizer 2151 and the second intra-frame predictor 2152. The inter-frame predictor 2154 may generate a prediction value through a prediction between frames by using an LSF coefficient decoded in a previous frame. A finally decoded LSF coefficient is generated by adding an inter-frame prediction value obtained by the inter-frame predictor 2153 to the addition result and then adding a mean value that is a predetermined DC value thereto.
Herein, the third inverse quantizer 2133 and the fourth inverse quantizer 2153 may share a codebook.
Although not shown, the inverse quantization apparatuses of
The contents related to a BC-TCVQ employed in association with LPC coefficient quantization/inverse quantization are described in detail in “Block Constrained Trellis Coded Vector Quantization of LSF Parameters for Wideband Speech Codecs” (Jungeun Park and Sangwon Kang, ETRI Journal, Volume 30, Number 5, October 2008). In addition, the contents related to a TCVQ are described in detail in “Trellis Coded Vector Quantization” (Thomas R. Fischer et al, IEEE Transactions on Information Theory, Vol. 37, No. 6, November 1991).
The methods according to the embodiments may be edited by computer-executable programs and implemented in a general-use digital computer for executing the programs by using a computer-readable recording medium. In addition, data structures, program commands, or data files usable in the embodiments of the present invention may be recorded in the computer-readable recording medium through various means. The computer-readable recording medium may include all types of storage devices for storing data readable by a computer system. Examples of the computer-readable recording medium include magnetic media such as hard discs, floppy discs, or magnetic tapes, optical media such as compact disc-read only memories (CD-ROMs), or digital versatile discs (DVDs), magneto-optical media such as floptical discs, and hardware devices that are specially configured to store and carry out program commands, such as ROMs, RAMs, or flash memories. In addition, the computer-readable recording medium may be a transmission medium for transmitting a signal for designating program commands, data structures, or the like. Examples of the program commands include a high-level language code that may be executed by a computer using an interpreter as well as a machine language code made by a compiler.
Although the embodiments of the present invention have been described with reference to the limited embodiments and drawings, the embodiments of the present invention are not limited to the embodiments described above, and their updates and modifications could be variously carried out by those of ordinary skill in the art from the disclosure. Therefore, the scope of the present invention is defined not by the above description but by the claims, and all their uniform or equivalent modifications would belong to the scope of the technical idea of the present invention.
This application is a continuation application of U.S. application Ser. No. 16/688,482, filed on Nov. 19, 2019, which is a continuation application of U.S. application Ser. No. 15/300,173, filed on Sep. 28, 2016, now U.S. Pat. No. 10,515,646, issued on Dec. 24, 2019, which is a National Stage application of International Application No. PCT/IB2015/001152 filed Mar. 30, 2015, which claims the benefit of U.S. Provisional Application No. 61/971,638, filed on Mar. 28, 2014 and U.S. Provisional Application No. 62/029,687, filed on Jul. 28, 2014, in the U.S. Patent and Trademark Office, the disclosures of which are incorporated herein in their entireties by reference.
Number | Date | Country | |
---|---|---|---|
62029687 | Jul 2014 | US | |
61971638 | Mar 2014 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 16688482 | Nov 2019 | US |
Child | 17947249 | US | |
Parent | 15300173 | Sep 2016 | US |
Child | 16688482 | US |