The present disclosure relates to audio decoding technologies, and in particular, to a speech/audio bitstream decoding method and apparatus.
In a system based on Voice over Internet Protocol (VoIP), a packet may need to pass through multiple routers in a transmission process, but because these routers may change in a call process, a transmission delay in the call process may change. In addition, when two or more users attempt to enter a network using a same gateway, a routing delay may change, and such a delay change is called a delay jitter. Similarly, a delay jitter may also be caused when a receiver, a transmitter, a gateway, and the like use a non-real-time operating system, and in a severe situation, a data packet loss occurs, resulting in speech/audio distortion and deterioration of VoIP quality.
Currently, many technologies have been used at different layers of a communication system to reduce a delay, smooth a delay jitter, and perform packet loss compensation. A receiver may use a high-efficiency jitter buffer processing (e.g., Jitter Buffer Management (JBM)) algorithm to compensate for a network delay jitter to some extent. However, in a case of a relatively high packet loss rate, a high-quality communication requirement cannot be met only using the JBM technology.
To help avoid the quality deterioration problem caused by a delay jitter of a speech/audio frame, a redundancy coding algorithm is introduced. That is, in addition to encoding current speech/audio frame information at a particular bit rate, an encoder encodes other speech/audio frame information than the current speech/audio frame at a lower bit rate, and transmits a relatively low bit rate bitstream of the other speech/audio frame information, as redundancy information, to a decoder together with a bitstream of the current speech/audio frame information. When a speech/audio frame is lost, if a jitter buffer buffers or a received bitstream includes redundancy information of the lost speech/audio frame, the decoder recovers the lost speech/audio frame according to the redundancy information, thereby improving speech/audio quality.
In an existing redundancy coding algorithm, in addition to including speech/audio frame information of the Nth frame, a bitstream of the Nth frame includes speech/audio frame information of the (N-M)th frame at lower bit rate. In a transmission process, if the (N-M)th frame is lost, decoding processing is performed according to the speech/audio frame information that is of the (N-M)th frame and is included in the bitstream of the Nth frame, to recover a speech/audio signal of the (N-M)th frame.
It can be learned from the foregoing description that, in the existing redundancy coding algorithm, redundancy bitstream information is obtained by means of encoding at a lower bit rate, which is therefore highly likely to cause signal instability and further cause low quality of an output speech/audio signal.
Embodiments of the present disclosure provide a speech/audio bitstream decoding method and apparatus, which help improve quality of an output speech/audio signal.
A first aspect of the embodiments of the present disclosure provides a speech/audio bitstream decoding method, which may include acquiring a speech/audio decoding parameter of a current speech/audio frame, where the current speech/audio frame is a redundant decoded frame or a speech/audio frame previous to the current speech/audio frame is a redundant decoded frame, performing post processing on the speech/audio decoding parameter of the current speech/audio frame according to speech/audio parameters of X speech/audio frames to obtain a post-processed speech/audio decoding parameter of the current speech/audio frame, where the X speech/audio frames include M speech/audio frames previous to the current speech/audio frame and/or N speech/audio frames next to the current speech/audio frame, and M and N are positive integers, and recovering a speech/audio signal of the current speech/audio frame using the post-processed speech/audio decoding parameter of the current speech/audio frame.
A second aspect of the embodiments of the present disclosure provides a decoder for decoding a speech/audio bitstream, including a parameter acquiring unit configured to acquire a speech/audio decoding parameter of a current speech/audio frame, where the current speech/audio frame is a redundant decoded frame or a speech/audio frame previous to the current speech/audio frame is a redundant decoded frame, a post processing unit configured to perform post processing on the speech/audio decoding parameter of the current speech/audio frame according to speech/audio parameters of X speech/audio frames to obtain a post-processed speech/audio decoding parameter of the current speech/audio frame, where the X speech/audio frames include M speech/audio frames previous to the current speech/audio frame and/or N speech/audio frames next to the current speech/audio frame, and M and N are positive integers, and a recovery unit configured to recover a speech/audio signal of the current speech/audio frame using the post-processed speech/audio decoding parameter of the current speech/audio frame.
A third aspect of the embodiments of the present disclosure provides a computer storage medium, where the computer storage medium may store a program, and when being executed, the program includes some or all steps of any speech/audio bitstream decoding method described in the embodiments of the present disclosure.
It can be learned that in some embodiments of the present disclosure, in a scenario in which a current speech/audio frame is a redundant decoded frame or a speech/audio frame previous to the current speech/audio frame is a redundant decoded frame, after obtaining a speech/audio decoding parameter of the current speech/audio frame, a decoder performs post processing on the speech/audio decoding parameter of the current speech/audio frame according to speech/audio parameters of X speech/audio frames to obtain a post-processed speech/audio decoding parameter of the current speech/audio frame, where the foregoing X speech/audio frames include M speech/audio frames previous to the foregoing current speech/audio frame and/or N speech/audio frames next to the foregoing current speech/audio frame, and recovers a speech/audio signal of the current speech/audio frame using the post-processed speech/audio decoding parameter of the current speech/audio frame, which ensures stable quality of a decoded signal during transition between a redundant decoded frame and a normal decoded frame or between a redundant decoded frame and a frame erasure concealment (FEC) recovered frame, thereby improving quality of an output speech/audio signal.
To describe the technical solutions in some of the embodiments of the present disclosure more clearly, the following briefly describes the accompanying drawings describing some of the embodiments. The accompanying drawings in the following description show merely some embodiments of the present disclosure, and persons of ordinary skill in the art may still derive other drawings from these accompanying drawings without creative efforts.
Embodiments of the present disclosure provide a speech/audio bitstream decoding method and apparatus, which help improve quality of an output speech/audio signal.
To make the disclosure objectives, features, and advantages of the present disclosure clearer and more comprehensible, the following clearly describes the technical solutions in the embodiments of the present disclosure with reference to the accompanying drawings in the embodiments of the present disclosure. The embodiments described in the following are merely a part rather than all of the embodiments of the present disclosure. All other embodiments obtained by persons of ordinary skill in the art based on the embodiments of the present disclosure without creative efforts shall fall within the protection scope of the present disclosure.
In the specification, claims, and accompanying drawings of the present disclosure, the terms “first,” “second,” “third,” “fourth,” and so on are intended to distinguish between different objects but not to indicate a particular order. In addition, the terms “including,” “including,” or any other variant thereof, are intended to cover a non-exclusive inclusion. For example, a process, a method, a system, a product, or a device including a series of steps or units is not limited to the listed steps or units, and may include steps or units that are not listed.
The following gives respective descriptions in details.
The speech/audio bitstream decoding method provided in the embodiments of the present disclosure is first described. The speech/audio bitstream decoding method provided in the embodiments of the present disclosure is executed by a decoder, where the decoder may be any apparatus that needs to output speeches, for example, a device such as a mobile phone, a notebook computer, a tablet computer, or a personal computer.
In an embodiment of the speech/audio bitstream decoding method in the present disclosure, the speech/audio bitstream decoding method may include acquiring a speech/audio decoding parameter of a current speech/audio frame, where the foregoing current speech/audio frame is a redundant decoded frame or a speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame, performing post processing on the speech/audio decoding parameter of the foregoing current speech/audio frame according to speech/audio parameters of X speech/audio frames, to obtain a post-processed speech/audio decoding parameter of the foregoing current speech/audio frame, where the foregoing X speech/audio frames include M speech/audio frames previous to the foregoing current speech/audio frame and/or N speech/audio frames next to the foregoing current speech/audio frame, and M and N are positive integers, and recovering a speech/audio signal of the foregoing current speech/audio frame using the post-processed speech/audio decoding parameter of the foregoing current speech/audio frame.
Step 101. Acquire a speech/audio decoding parameter of a current speech/audio frame.
The foregoing current speech/audio frame is a redundant decoded frame or a speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame.
When the speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame, the current speech/audio frame may be a normal decoded frame, an FEC recovered frame, or a redundant decoded frame, where if the current speech/audio frame is an FEC recovered frame, the speech/audio decoding parameter of the current speech/audio frame may be predicated based on an FEC algorithm.
Step 102. Perform post processing on the speech/audio decoding parameter of the foregoing current speech/audio frame according to speech/audio parameters of X speech/audio frames to obtain a post-processed speech/audio decoding parameter of the foregoing current speech/audio frame.
The foregoing X speech/audio frames include M speech/audio frames previous to the foregoing current speech/audio frame and/or N speech/audio frames next to the foregoing current speech/audio frame, and M and N are positive integers.
That a speech/audio frame (for example, the current speech/audio frame or the speech/audio frame previous to the current speech/audio frame) is a normal decoded frame means that a speech/audio parameter of the foregoing speech/audio frame can be directly obtained from a bitstream of the speech/audio frame by means of decoding.
That a speech/audio frame (for example, a current speech/audio frame or a speech/audio frame previous to a current speech/audio frame) is a redundant decoded frame means that a speech/audio parameter of the speech/audio frame cannot be directly obtained from a bitstream of the speech/audio frame by means of decoding, but redundant bitstream information of the speech/audio frame can be obtained from a bitstream of another speech/audio frame.
The M speech/audio frames previous to the current speech/audio frame refer to M speech/audio frames preceding the current speech/audio frame and immediately adjacent to the current speech/audio frame in a time domain.
For example, M may be equal to 1, 2, 3, or another value. When M=1, the M speech/audio frames previous to the current speech/audio frame are the speech/audio frame previous to the current speech/audio frame, and the speech/audio frame previous to the current speech/audio frame and the current speech/audio frame are two immediately adjacent speech/audio frames, when M=2, the M speech/audio frames previous to the current speech/audio frame are the speech/audio frame previous to the current speech/audio frame and a speech/audio frame previous to the speech/audio frame previous to the current speech/audio frame, and the speech/audio frame previous to the current speech/audio frame, the speech/audio frame previous to the speech/audio frame previous to the current speech/audio frame, and the current speech/audio frame are three immediately adjacent speech/audio frames, and so on.
The N speech/audio frames next to the current speech/audio frame refer to N speech/audio frames following the current speech/audio frame and immediately adjacent to the current speech/audio frame in a time domain.
For example, N may be equal to 1, 2, 3, 4, or another value. When N=1, the N speech/audio frames next to the current speech/audio frame are a speech/audio frame next to the current speech/audio frame, and the speech/audio frame next to the current speech/audio frame and the current speech/audio frame are two immediately adjacent speech/audio frames, when N=2, the N speech/audio frames next to the current speech/audio frame are a speech/audio frame next to the current speech/audio frame and a speech/audio frame next to the speech/audio frame next to the current speech/audio frame, and the speech/audio frame next to the current speech/audio frame, the speech/audio frame next to the speech/audio frame next to the current speech/audio frame, and the current speech/audio frame are three immediately adjacent speech/audio frames, and so on.
The speech/audio decoding parameter may include at least one of the following parameters a bandwidth extension envelope, an adaptive codebook gain (gain_pit), an algebraic codebook, a pitch period, a spectrum tilt factor, a spectral pair parameter, and the like.
The speech/audio parameter may include a speech/audio decoding parameter, a signal class, and the like.
A signal class of a speech/audio frame may be unvoiced, voiced, generic, transient, inactive, or the like.
The spectral pair parameter may be, for example, at least one of a line spectral pair (LSP) parameter or an immittance spectral pair (ISP) parameter.
It may be understood that in this embodiment of the present disclosure, post processing may be performed on at least one speech/audio decoding parameter of a bandwidth extension envelope, an adaptive codebook gain, an algebraic codebook, a pitch period, or a spectral pair parameter of the current speech/audio frame.
Further, how many parameters are selected and which parameters are selected for post processing may be determined according to an application scenario and an application environment, which is not limited in this embodiment of the present disclosure.
Different post processing may be performed on different speech/audio decoding parameters. For example, post processing performed on the spectral pair parameter of the current speech/audio frame may be adaptive weighting performed using the spectral pair parameter of the current speech/audio frame and a spectral pair parameter of the speech/audio frame previous to the current speech/audio frame to obtain a post-processed spectral pair parameter of the current speech/audio frame, and post processing performed on the adaptive codebook gain of the current speech/audio frame may be adjustment such as attenuation performed on the adaptive codebook gain.
A specific post processing manner is not limited in this embodiment of the present disclosure, and specific post processing may be set according to a requirement or according to an application environment and an application scenario.
Step 103. Recover a speech/audio signal of the foregoing current speech/audio frame using the post-processed speech/audio decoding parameter of the foregoing current speech/audio frame.
It can be learned from the foregoing description that in this embodiment, in a scenario in which a current speech/audio frame is a redundant decoded frame or a speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame, after obtaining a speech/audio decoding parameter of the current speech/audio frame, a decoder performs post processing on the speech/audio decoding parameter of the current speech/audio frame according to speech/audio parameters of X speech/audio frames to obtain a post-processed speech/audio decoding parameter of the foregoing current speech/audio frame, where the foregoing X speech/audio frames include M speech/audio frames previous to the foregoing current speech/audio frame and/or N speech/audio frames next to the foregoing current speech/audio frame, and recovers a speech/audio signal of the current speech/audio frame using the post-processed speech/audio decoding parameter of the current speech/audio frame, which ensures stable quality of a decoded signal during transition between a redundant decoded frame and a normal decoded frame or between a redundant decoded frame and an FEC recovered frame, thereby improving quality of an output speech/audio signal.
In some embodiments of the present disclosure, the speech/audio decoding parameter of the foregoing current speech/audio frame includes the spectral pair parameter of the foregoing current speech/audio frame, and performing post processing on the speech/audio decoding parameter of the foregoing current speech/audio frame according to speech/audio parameters of X speech/audio frames to obtain a post-processed speech/audio decoding parameter of the foregoing current speech/audio frame, for example, may include performing post processing on the spectral pair parameter of the foregoing current speech/audio frame according to at least one of a signal class, a spectrum tilt factor, an adaptive codebook gain, or a spectral pair parameter of the X speech/audio frames to obtain a post-processed spectral pair parameter of the foregoing current speech/audio frame.
For example, performing post processing on the spectral pair parameter of the foregoing current speech/audio frame according to at least one of a signal class, a spectrum tilt factor, an adaptive codebook gain, or a spectral pair parameter of the X speech/audio frames to obtain a post-processed spectral pair parameter of the foregoing current speech/audio frame may include, if the foregoing current speech/audio frame is a normal decoded frame, the speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame, a signal class of the foregoing current speech/audio frame is unvoiced, and a signal class of the speech/audio frame previous to the foregoing current speech/audio frame is not unvoiced, using the spectral pair parameter of the foregoing current speech/audio frame as the post-processed spectral pair parameter of the foregoing current speech/audio frame, or obtaining the post-processed spectral pair parameter of the foregoing current speech/audio frame based on the spectral pair parameter of the foregoing current speech/audio frame.
If the foregoing current speech/audio frame is a normal decoded frame, the speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame, a signal class of the foregoing current speech/audio frame is unvoiced, and a signal class of the speech/audio frame previous to the foregoing current speech/audio frame is not unvoiced, obtaining the post-processed spectral pair parameter of the foregoing current speech/audio frame based on the spectral pair parameter of the current speech/audio frame and a spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame.
If the foregoing current speech/audio frame is a redundant decoded frame, a signal class of the foregoing current speech/audio frame is not unvoiced, and a signal class of a speech/audio frame next to the foregoing current speech/audio frame is unvoiced, using a spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame as the post-processed spectral pair parameter of the foregoing current speech/audio frame, or obtaining the post-processed spectral pair parameter of the foregoing current speech/audio frame based on a spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame.
If the foregoing current speech/audio frame is a redundant decoded frame, a signal class of the foregoing current speech/audio frame is not unvoiced, and a signal class of a speech/audio frame next to the foregoing current speech/audio frame is unvoiced, obtaining the post-processed spectral pair parameter of the foregoing current speech/audio frame based on the spectral pair parameter of the foregoing current speech/audio frame and a spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame.
If the foregoing current speech/audio frame is a redundant decoded frame, a signal class of the foregoing current speech/audio frame is not unvoiced, a maximum value of an adaptive codebook gain of a subframe in a speech/audio frame next to the foregoing current speech/audio frame is less than or equal to a first threshold, and a spectrum tilt factor of the speech/audio frame previous to the foregoing current speech/audio frame is less than or equal to a second threshold, using a spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame as the post-processed spectral pair parameter of the foregoing current speech/audio frame, or obtaining the post-processed spectral pair parameter of the foregoing current speech/audio frame based on a spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame.
If the foregoing current speech/audio frame is a redundant decoded frame, a signal class of the foregoing current speech/audio frame is not unvoiced, a maximum value of an adaptive codebook gain of a subframe in a speech/audio frame next to the foregoing current speech/audio frame is less than or equal to a first threshold, and a spectrum tilt factor of the speech/audio frame previous to the foregoing current speech/audio frame is less than or equal to a second threshold, obtaining the post-processed spectral pair parameter of the foregoing current speech/audio frame based on the spectral pair parameter of the current speech/audio frame and a spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame.
If the foregoing current speech/audio frame is a redundant decoded frame, a signal class of the foregoing current speech/audio frame is not unvoiced, a speech/audio frame next to the foregoing current speech/audio frame is unvoiced, a maximum value of an adaptive codebook gain of a subframe in the speech/audio frame next to the foregoing current speech/audio frame is less than or equal to a third threshold, and a spectrum tilt factor of the speech/audio frame previous to the foregoing current speech/audio frame is less than or equal to a fourth threshold, using a spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame as the post-processed spectral pair parameter of the foregoing current speech/audio frame, or obtaining the post-processed spectral pair parameter of the foregoing current speech/audio frame based on a spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame.
If the foregoing current speech/audio frame is a redundant decoded frame, a signal class of the foregoing current speech/audio frame is not unvoiced, a signal class of a speech/audio frame next to the foregoing current speech/audio frame is unvoiced, a maximum value of an adaptive codebook gain of a subframe in the speech/audio frame next to the foregoing current speech/audio frame is less than or equal to a third threshold, and a spectrum tilt factor of the speech/audio frame previous to the foregoing current speech/audio frame is less than or equal to a fourth threshold, obtaining the post-processed spectral pair parameter of the foregoing current speech/audio frame based on the spectral pair parameter of the foregoing current speech/audio frame and a spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame.
There may be various manners for obtaining the post-processed spectral pair parameter of the foregoing current speech/audio frame based on the spectral pair parameter of the foregoing current speech/audio frame and a spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame.
For example, obtaining the post-processed spectral pair parameter of the foregoing current speech/audio frame based on the spectral pair parameter of the foregoing current speech/audio frame and a spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame may include obtaining the post-processed spectral pair parameter of the foregoing current speech/audio frame based on the spectral pair parameter of the foregoing current speech/audio frame and the spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame and using the following formula:
lsp[k]=α*lsp_old[k]+β*lsp_mid[k]+δ*lsp_new[k] 0≤k≤L,
where lsp[k] is the post-processed spectral pair parameter of the foregoing current speech/audio frame, lsp_old[k] is the spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame, lsp_mid[k] is a middle value of the spectral pair parameter of the foregoing current speech/audio frame, lsp_new[k] is the spectral pair parameter of the foregoing current speech/audio frame, L is an order of a spectral pair parameter, α is a weight of the spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame, β is a weight of the middle value of the spectral pair parameter of the foregoing current speech/audio frame, δ is a weight of the spectral pair parameter of the foregoing current speech/audio frame, α≥0, β≥0, and α+β+δ=1, where if the foregoing current speech/audio frame is a normal decoded frame, and the speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame, α is equal to 0 or α is less than or equal to a fifth threshold, if the foregoing current speech/audio frame is a redundant decoded frame, β is equal to 0 or β is less than or equal to a sixth threshold, if the foregoing current speech/audio frame is a redundant decoded frame, δ is equal to 0 or δ is less than or equal to a seventh threshold, or if the foregoing current speech/audio frame is a redundant decoded frame, β is equal to 0 or β is less than or equal to a sixth threshold, and δ is equal to 0 or δ is less than or equal to a seventh threshold.
For another example, obtaining the post-processed spectral pair parameter of the foregoing current speech/audio frame based on the spectral pair parameter of the foregoing current speech/audio frame and a spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame may include obtaining the post-processed spectral pair parameter of the foregoing current speech/audio frame based on the spectral pair parameter of the foregoing current speech/audio frame and the spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame and using the following formula:
lsp[k]=α*lsp_old[k]+δ*lsp_new[k] 0≤k≤L,
where lsp[k] is the post-processed spectral pair parameter of the foregoing current speech/audio frame, lsp_old[k] is the spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame, lsp_new[k] is the spectral pair parameter of the foregoing current speech/audio frame, L is an order of a spectral pair parameter, α is a weight of the spectral pair parameter of the speech/audio frame previous to the foregoing current speech/audio frame, δ is a weight of the spectral pair parameter of the foregoing current speech/audio frame, α≥0, δ≥0, and α+δ=1, where if the foregoing current speech/audio frame is a normal decoded frame, and the speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame, a is equal to 0 or a is less than or equal to a fifth threshold, or if the foregoing current speech/audio frame is a redundant decoded frame, δ is equal to 0 or δ is less than or equal to a seventh threshold.
The fifth threshold, the sixth threshold, and the seventh threshold each may be set to different values according to different application environments or scenarios. For example, a value of the fifth threshold may be close to 0.
For example, the fifth threshold may be equal to 0.001, 0.002, 0.01, 0.1, or another value close to 0, a value of the sixth threshold may be close to 0, where for example, the sixth threshold may be equal to 0.001, 0.002, 0.01, 0.1, or another value close to 0, and a value of the seventh threshold may be close to 0, where for example, the seventh threshold may be equal to 0.001, 0.002, 0.01, 0.1, or another value close to 0.
The first threshold, the second threshold, the third threshold, and the fourth threshold each may be set to different values according to different application environments or scenarios.
For example, the first threshold may be set to 0.9, 0.8, 0.85, 0.7, 0.89, or 0.91.
For example, the second threshold may be set to 0.16, 0.15, 0.165, 0.1, 0.161, or 0.159.
For example, the third threshold may be set to 0.9, 0.8, 0.85, 0.7, 0.89, or 0.91.
For example, the fourth threshold may be set to 0.16, 0.15, 0.165, 0.1, 0.161, or 0.159.
The first threshold may be equal to or not equal to the third threshold, and the second threshold may be equal to or not equal to the fourth threshold.
In other embodiments of the present disclosure, the speech/audio decoding parameter of the foregoing current speech/audio frame includes the adaptive codebook gain of the foregoing current speech/audio frame, and performing post processing on the speech/audio decoding parameter of the foregoing current speech/audio frame according to speech/audio parameters of X speech/audio frames to obtain a post-processed speech/audio decoding parameter of the foregoing current speech/audio frame may include performing post processing on the adaptive codebook gain of the foregoing current speech/audio frame according to at least one of the signal class, an algebraic codebook gain, or the adaptive codebook gain of the X speech/audio frames, to obtain a post-processed adaptive codebook gain of the foregoing current speech/audio frame.
For example, performing post processing on the adaptive codebook gain of the foregoing current speech/audio frame according to at least one of the signal class, an algebraic codebook gain, or the adaptive codebook gain of the X speech/audio frames may include, if the foregoing current speech/audio frame is a redundant decoded frame, the signal class of the foregoing current speech/audio frame is not unvoiced, a signal class of at least one of two speech/audio frames next to the foregoing current speech/audio frame is unvoiced, and an algebraic codebook gain of a current subframe of the foregoing current speech/audio frame is greater than or equal to an algebraic codebook gain of the speech/audio frame previous to the foregoing current speech/audio frame (for example, the algebraic codebook gain of the current subframe of the foregoing current speech/audio frame is 1 or more than 1 time, for example, 1, 1.5, 2, 2.5, 3, 3.4, or 4 times, the algebraic codebook gain of the speech/audio frame previous to the foregoing current speech/audio frame, attenuating an adaptive codebook gain of the foregoing current subframe.
If the foregoing current speech/audio frame is a redundant decoded frame, the signal class of the foregoing current speech/audio frame is not unvoiced, a signal class of at least one of the speech/audio frame next to the foregoing current speech/audio frame or a speech/audio frame next to the next speech/audio frame is unvoiced, and an algebraic codebook gain of a current subframe of the foregoing current speech/audio frame is greater than or equal to an algebraic codebook gain of a subframe previous to the foregoing current subframe (for example, the algebraic codebook gain of the current subframe of the foregoing current speech/audio frame is 1 or more than 1 time, for example, 1, 1.5, 2, 2.5, 3, 3.4, or 4 times, the algebraic codebook gain of the subframe previous to the foregoing current subframe), attenuating an adaptive codebook gain of the foregoing current subframe.
If the foregoing current speech/audio frame is a redundant decoded frame, or the foregoing current speech/audio frame is a normal decoded frame, and the speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame, and if the signal class of the foregoing current speech/audio frame is generic, the signal class of the speech/audio frame next to the foregoing current speech/audio frame is voiced, and an algebraic codebook gain of a subframe of the foregoing current speech/audio frame is greater than or equal to an algebraic codebook gain of a subframe previous to the foregoing subframe (for example, the algebraic codebook gain of the subframe of the foregoing current speech/audio frame may be 1 or more than 1 time, for example, 1, 1.5, 2, 2.5, 3, 3.4, or 4 times, the algebraic codebook gain of the subframe previous to the foregoing subframe), adjusting (for example, augmenting or attenuating) an adaptive codebook gain of a current subframe of the foregoing current speech/audio frame based on at least one of a ratio of an algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of a subframe adjacent to the foregoing current subframe, a ratio of the adaptive codebook gain of the current subframe of the foregoing current speech/audio frame to that of the subframe adjacent to the foregoing current subframe, or a ratio of the algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of the speech/audio frame previous to the foregoing current speech/audio frame (for example, if the ratio of the algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of the subframe adjacent to the foregoing current subframe is greater than or equal to an eleventh threshold (where the eleventh threshold may be equal to, for example, 2, 2.1, 2.5, 3, or another value), the ratio of the adaptive codebook gain of the current subframe of the foregoing current speech/audio frame to that of the subframe adjacent to the foregoing current subframe is greater than or equal to a twelfth threshold (where the twelfth threshold may be equal to, for example, 1, 1.1, 1.5, 2, 2.1, or another value), and the ratio of the algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of the speech/audio frame previous to the foregoing current speech/audio frame is less than or equal to a thirteenth threshold (where the thirteenth threshold may be equal to, for example, 1, 1.1, 1.5, 2, or another value), the adaptive codebook gain of the current subframe of the foregoing current speech/audio frame may be augmented).
If the foregoing current speech/audio frame is a redundant decoded frame, or the foregoing current speech/audio frame is a normal decoded frame, and the speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame, and if the signal class of the foregoing current speech/audio frame is generic, the signal class of the speech/audio frame next to the foregoing current speech/audio frame is voiced, and an algebraic codebook gain of a subframe of the foregoing current speech/audio frame is greater than or equal to an algebraic codebook gain of the speech/audio frame previous to the foregoing current speech/audio frame (where the algebraic codebook gain of the subframe of the foregoing current speech/audio frame is 1 or more than 1 time, for example, 1, 1.5, 2, 2.5, 3, 3.4, or 4 times, the algebraic codebook gain of the speech/audio frame previous to the foregoing current speech/audio frame), adjusting (attenuating or augmenting) an adaptive codebook gain of a current subframe of the foregoing current speech/audio frame based on at least one of a ratio of an algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of a subframe adjacent to the foregoing current subframe, a ratio of the adaptive codebook gain of the current subframe of the foregoing current speech/audio frame to that of the subframe adjacent to the foregoing current subframe, or a ratio of the algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of the speech/audio frame previous to the foregoing current speech/audio frame (for example, if the ratio of the algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of the subframe adjacent to the foregoing current subframe is greater than or equal to an eleventh threshold (where the eleventh threshold may be equal to, for example, 2, 2.1, 2.5, 3, or another value), the ratio of the adaptive codebook gain of the current subframe of the foregoing current speech/audio frame to that of the subframe adjacent to the foregoing current subframe is greater than or equal to a twelfth threshold (where the twelfth threshold may be equal to, for example, 1, 1.1, 1.5, 2, 2.1, or another value), and the ratio of the algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of the speech/audio frame previous to the foregoing current speech/audio frame is less than or equal to a thirteenth threshold (where the thirteenth threshold may be equal to, for example, 1, 1.1, 1.5, 2, or another value), the adaptive codebook gain of the current subframe of the foregoing current speech/audio frame may be augmented).
If the foregoing current speech/audio frame is a redundant decoded frame, or the foregoing current speech/audio frame is a normal decoded frame, and the speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame, and if the foregoing current speech/audio frame is voiced, the signal class of the speech/audio frame previous to the foregoing current speech/audio frame is generic, and an algebraic codebook gain of a subframe of the foregoing current speech/audio frame is greater than or equal to an algebraic codebook gain of a subframe previous to the foregoing subframe (for example, the algebraic codebook gain of the subframe of the foregoing current speech/audio frame may be 1 or more than 1 time, for example, 1, 1.5, 2, 2.5, 3, 3.4, or 4 times, the algebraic codebook gain of the subframe previous to the foregoing subframe), adjusting (attenuating or augmenting) an adaptive codebook gain of a current subframe of the foregoing current speech/audio frame based on at least one of a ratio of an algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of a subframe adjacent to the foregoing current subframe, a ratio of the adaptive codebook gain of the current subframe of the foregoing current speech/audio frame to that of the subframe adjacent to the foregoing current subframe, or a ratio of the algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of the speech/audio frame previous to the foregoing current speech/audio frame (for example, if the ratio of the algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of the subframe adjacent to the foregoing current subframe is greater than or equal to an eleventh threshold (where the eleventh threshold is equal to, for example, 2, 2.1, 2.5, 3, or another value), the ratio of the adaptive codebook gain of the current subframe of the foregoing current speech/audio frame to that of the subframe adjacent to the foregoing current subframe is greater than or equal to a twelfth threshold (where the twelfth threshold is equal to, for example, 1, 1.1, 1.5, 2, 2.1, or another value), and the ratio of the algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of the speech/audio frame previous to the foregoing current speech/audio frame is less than or equal to a thirteenth threshold (where the thirteenth threshold may be equal to, for example, 1, 1.1, 1.5, 2, or another value), the adaptive codebook gain of the current subframe of the foregoing current speech/audio frame may be augmented.
If the foregoing current speech/audio frame is a redundant decoded frame, or the foregoing current speech/audio frame is a normal decoded frame, and the speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame, and if the signal class of the foregoing current speech/audio frame is voiced, the signal class of the speech/audio frame previous to the foregoing current speech/audio frame is generic, and an algebraic codebook gain of a subframe of the foregoing current speech/audio frame is greater than or equal to an algebraic codebook gain of the speech/audio frame previous to the foregoing current speech/audio frame (for example, the algebraic codebook gain of the subframe of the foregoing current speech/audio frame is 1 or more than 1 time, for example, 1, 1.5, 2, 2.5, 3, 3.4, or 4 times, the algebraic codebook gain of the speech/audio frame previous to the foregoing current speech/audio frame), adjusting (attenuating or augmenting) an adaptive codebook gain of a current subframe of the foregoing current speech/audio frame based on at least one of a ratio of an algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of a subframe adjacent to the foregoing current subframe, a ratio of the adaptive codebook gain of the current subframe of the foregoing current speech/audio frame to that of the subframe adjacent to the foregoing current subframe, or a ratio of the algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of the speech/audio frame previous to the foregoing current speech/audio frame (for example, if the ratio of the algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of the subframe adjacent to the foregoing current subframe is greater than or equal to an eleventh threshold (where the eleventh threshold may be equal to, for example, 2, 2.1, 2.5, 3, or another value), the ratio of the adaptive codebook gain of the current subframe of the foregoing current speech/audio frame to that of the subframe adjacent to the foregoing current subframe is greater than or equal to a twelfth threshold (where the twelfth threshold may be equal to, for example, 1, 1.1, 1.5, 2, 2.1, or another value), and the ratio of the algebraic codebook gain of the current subframe of the foregoing current speech/audio frame to that of the speech/audio frame previous to the foregoing current speech/audio frame is less than or equal to a thirteenth threshold (where the thirteenth threshold is equal to, for example, 1, 1.1, 1.5, 2, or another value), the adaptive codebook gain of the current subframe of the foregoing current speech/audio frame may be augmented.
In other embodiments of the present disclosure, the speech/audio decoding parameter of the foregoing current speech/audio frame includes the algebraic codebook of the foregoing current speech/audio frame, and the performing post processing on the speech/audio decoding parameter of the foregoing current speech/audio frame according to speech/audio parameters of X speech/audio frames to obtain a post-processed speech/audio decoding parameter of the foregoing current speech/audio frame may include performing post processing on the algebraic codebook of the foregoing current speech/audio frame according to at least one of the signal class, an algebraic codebook, or the spectrum tilt factor of the X speech/audio frames to obtain a post-processed algebraic codebook of the foregoing current speech/audio frame.
For example, the performing post processing on the algebraic codebook of the foregoing current speech/audio frame according to at least one of the signal class, an algebraic codebook, or the spectrum tilt factor of the X speech/audio frames may include, if the foregoing current speech/audio frame is a redundant decoded frame, the signal class of the speech/audio frame next to the foregoing current speech/audio frame is unvoiced, the spectrum tilt factor of the speech/audio frame previous to the foregoing current speech/audio frame is less than or equal to an eighth threshold, and an algebraic codebook of a subframe of the foregoing current speech/audio frame is 0 or is less than or equal to a ninth threshold, using an algebraic codebook or a random noise of a subframe previous to the foregoing current speech/audio frame as an algebraic codebook of the foregoing current subframe.
The eighth threshold and the ninth threshold each may be set to different values according to different application environments or scenarios.
For example, the eighth threshold may be set to 0.16, 0.15, 0.165, 0.1, 0.161, or 0.159.
For example, the ninth threshold may be set to 0.1, 0.09, 0.11, 0.07, 0.101, 0.099, or another value close to 0.
The eighth threshold may be equal to or not equal to the second threshold.
In other embodiments of the present disclosure, the speech/audio decoding parameter of the foregoing current speech/audio frame includes a bandwidth extension envelope of the foregoing current speech/audio frame, and the performing post processing on the speech/audio decoding parameter of the foregoing current speech/audio frame according to speech/audio parameters of X speech/audio frames to obtain a post-processed speech/audio decoding parameter of the foregoing current speech/audio frame may include performing post processing on the bandwidth extension envelope of the foregoing current speech/audio frame according to at least one of the signal class, a bandwidth extension envelope, or the spectrum tilt factor of the X speech/audio frames to obtain a post-processed bandwidth extension envelope of the foregoing current speech/audio frame.
For example, the performing post processing on the bandwidth extension envelope of the foregoing current speech/audio frame according to at least one of the signal class, a bandwidth extension envelope, or the spectrum tilt factor of the X speech/audio frames to obtain a post-processed bandwidth extension envelope of the foregoing current speech/audio frame may include, if the speech/audio frame previous to the foregoing current speech/audio frame is a normal decoded frame, and the signal class of the speech/audio frame previous to the foregoing current speech/audio frame is the same as that of the speech/audio frame next to the current speech/audio frame, obtaining the post-processed bandwidth extension envelope of the foregoing current speech/audio frame based on a bandwidth extension envelope of the speech/audio frame previous to the foregoing current speech/audio frame and the bandwidth extension envelope of the foregoing current speech/audio frame.
If the foregoing current speech/audio frame is a prediction form of redundancy decoding, obtaining the post-processed bandwidth extension envelope of the foregoing current speech/audio frame based on a bandwidth extension envelope of the speech/audio frame previous to the foregoing current speech/audio frame and the bandwidth extension envelope of the foregoing current speech/audio frame.
If the signal class of the foregoing current speech/audio frame is not unvoiced, the signal class of the speech/audio frame next to the foregoing current speech/audio frame is unvoiced, the spectrum tilt factor of the speech/audio frame previous to the foregoing current speech/audio frame is less than or equal to a tenth threshold, modifying the bandwidth extension envelope of the foregoing current speech/audio frame according to a bandwidth extension envelope or the spectrum tilt factor of the speech/audio frame previous to the foregoing current speech/audio frame to obtain the post-processed bandwidth extension envelope of the foregoing current speech/audio frame.
The tenth threshold may be set to different values according to different application environments or scenarios. For example, the tenth threshold may be set to 0.16, 0.15, 0.165, 0.1, 0.161, or 0.159.
For example, the obtaining the post-processed bandwidth extension envelope of the foregoing current speech/audio frame based on a bandwidth extension envelope of the speech/audio frame previous to the foregoing current speech/audio frame and the bandwidth extension envelope of the foregoing current speech/audio frame may include obtaining the post-processed bandwidth extension envelope of the foregoing current speech/audio frame based on the bandwidth extension envelope of the speech/audio frame previous to the foregoing current speech/audio frame and the bandwidth extension envelope of the foregoing current speech/audio frame and using the following formula:
GainFrame=fac1*GainFrame_old+fac2*GainFrame_new,
where GainFrame is the post-processed bandwidth extension envelope of the foregoing current speech/audio frame, GainFrame_old is the bandwidth extension envelope of the speech/audio frame previous to the foregoing current speech/audio frame, GainFrame_new is the bandwidth extension envelope of the foregoing current speech/audio frame, fac1 is a weight of the bandwidth extension envelope of the speech/audio frame previous to the foregoing current speech/audio frame, fac2 is a weight of the bandwidth extension envelope of the foregoing current speech/audio frame, and fac1≥0, fac2≥0, and fac1+fac2=1.
For another example, a modification factor for modifying the bandwidth extension envelope of the foregoing current speech/audio frame is inversely proportional to the spectrum tilt factor of the speech/audio frame previous to the foregoing current speech/audio frame, and is proportional to a ratio of the bandwidth extension envelope of the speech/audio frame previous to the foregoing current speech/audio frame to the bandwidth extension envelope of the foregoing current speech/audio frame.
In other embodiments of the present disclosure, the speech/audio decoding parameter of the foregoing current speech/audio frame includes a pitch period of the foregoing current speech/audio frame, and performing post processing on the speech/audio decoding parameter of the foregoing current speech/audio frame according to speech/audio parameters of X speech/audio frames to obtain a post-processed speech/audio decoding parameter of the foregoing current speech/audio frame may include performing post processing on the pitch period of the foregoing current speech/audio frame according to the signal classes and/or pitch periods of the X speech/audio frames (for example, post processing such as augmentation or attenuation may be performed on the pitch period of the foregoing current speech/audio frame according to the signal classes and/or the pitch periods of the X speech/audio frames) to obtain a post-processed pitch period of the foregoing current speech/audio frame.
It can be learned from the foregoing description that in some embodiments of the present disclosure, during transition between an unvoiced speech/audio frame and a non-unvoiced speech/audio frame (for example, when a current speech/audio frame is of an unvoiced signal class and is a redundant decoded frame, and a speech/audio frame previous or next to the current speech/audio frame is of a non unvoiced signal type and is a normal decoded frame, or when a current speech/audio frame is of a non unvoiced signal class and is a normal decoded frame, and a speech/audio frame previous or next to the current speech/audio frame is of an unvoiced signal class and is a redundant decoded frame), post processing is performed on a speech/audio decoding parameter of the current speech/audio frame, which helps avoid a click phenomenon caused during the interframe transition between the unvoiced speech/audio frame and the non-unvoiced speech/audio frame, thereby improving quality of an output speech/audio signal.
In other embodiments of the present disclosure, during transition between a generic speech/audio frame and a voiced speech/audio frame (when a current speech/audio frame is a generic frame and is a redundant decoded frame, and a speech/audio frame previous or next to the current speech/audio frame is of a voiced signal class and is a normal decoded frame, or when a current speech/audio frame is of a voiced signal class and is a normal decoded frame, and a speech/audio frame previous or next to the current speech/audio frame is of a generic signal class and is a redundant decoded frame), post processing is performed on a speech/audio decoding parameter of the current speech/audio frame, which helps rectify an energy instability phenomenon caused during the transition between a generic frame and a voiced frame, thereby improving quality of an output speech/audio signal.
In still other embodiments of the present disclosure, when a current speech/audio frame is a redundant decoded frame, a signal class of the current speech/audio frame is not unvoiced, and a signal class of a speech/audio frame next to the current speech/audio frame is unvoiced, a bandwidth extension envelope of the current frame is adjusted, to rectify an energy instability phenomenon in time-domain bandwidth extension, and improve quality of an output speech/audio signal.
To help better understand and implement the foregoing solution in this embodiment of the present disclosure, some specific application scenarios are used as examples in the following description.
Referring to
Step 201. Determine a decoding status of a current speech/audio frame.
Further, for example, it may be determined, based on a JBM algorithm or another algorithm, that the current speech/audio frame is a normal decoded frame, a redundant decoded frame, or an FEC recovered frame.
If the current speech/audio frame is a normal decoded frame, and a speech/audio frame previous to the current speech/audio frame is a redundant decoded frame, step 202 is executed.
If the current speech/audio frame is a redundant decoded frame, step 203 is executed.
If the current speech/audio frame is an FEC recovered frame, and a speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame, step 204 is executed.
Step 202. Obtain a speech/audio decoding parameter of the current speech/audio frame based on a bitstream of the current speech/audio frame, and jump to step 205.
Step 203. Obtain a speech/audio decoding parameter of the foregoing current speech/audio frame based on a redundant bitstream of the current speech/audio frame, and jump to step 205.
Step 204. Obtain a speech/audio decoding parameter of the current speech/audio frame by means of prediction based on an FEC algorithm, and jump to step 205.
Step 205. Perform post processing on the speech/audio decoding parameter of the foregoing current speech/audio frame according to speech/audio parameters of X speech/audio frames to obtain a post-processed speech/audio decoding parameter of the foregoing current speech/audio frame, where the foregoing X speech/audio frames include M speech/audio frames previous to the foregoing current speech/audio frame and/or N speech/audio frames next to the foregoing current speech/audio frame, and M and N are positive integers.
Step 206. Recover a speech/audio signal of the foregoing current speech/audio frame using the post-processed speech/audio decoding parameter of the foregoing current speech/audio frame.
Different post processing may be performed on different speech/audio decoding parameters. For example, post processing performed on a spectral pair parameter of the current speech/audio frame may be adaptive weighting performed using the spectral pair parameter of the current speech/audio frame and a spectral pair parameter of the speech/audio frame previous to the current speech/audio frame, to obtain a post-processed spectral pair parameter of the current speech/audio frame, and post processing performed on an adaptive codebook gain of the current speech/audio frame may be adjustment such as attenuation performed on the adaptive codebook gain.
It may be understood that the details about performing post processing on the speech/audio decoding parameter in this embodiment may refer to related descriptions of the foregoing method embodiments, and details are not described herein.
It can be learned from the foregoing description that in this embodiment, in a scenario in which a current speech/audio frame is a redundant decoded frame or a speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame, after obtaining a speech/audio decoding parameter of the current speech/audio frame, a decoder performs post processing on the speech/audio decoding parameter of the current speech/audio frame according to speech/audio parameters of X speech/audio frames to obtain a post-processed speech/audio decoding parameter of the foregoing current speech/audio frame, where the foregoing X speech/audio frames include M speech/audio frames previous to the foregoing current speech/audio frame and/or N speech/audio frames next to the foregoing current speech/audio frame, and recovers a speech/audio signal of the current speech/audio frame using the post-processed speech/audio decoding parameter of the current speech/audio frame, which ensures stable quality of a decoded signal during transition between a redundant decoded frame and a normal decoded frame or between a redundant decoded frame and an FEC recovered frame, thereby improving quality of an output speech/audio signal.
It can be learned from the foregoing description that in some embodiments of the present disclosure, during transition between an unvoiced speech/audio frame and a non-unvoiced speech/audio frame (for example, when a current speech/audio frame is of an unvoiced signal class and is a redundant decoded frame, and a speech/audio frame previous or next to the current speech/audio frame is of a non unvoiced signal type and is a normal decoded frame, or when a current speech/audio frame is of a non unvoiced signal class and is a normal decoded frame, and a speech/audio frame previous or next to the current speech/audio frame is of an unvoiced signal class and is a redundant decoded frame), post processing is performed on a speech/audio decoding parameter of the current speech/audio frame, which helps avoid a click phenomenon caused during the interframe transition between the unvoiced speech/audio frame and the non-unvoiced speech/audio frame, thereby improving quality of an output speech/audio signal.
In other embodiments of the present disclosure, during transition between a generic speech/audio frame and a voiced speech/audio frame (when a current speech/audio frame is a generic frame and is a redundant decoded frame, and a speech/audio frame previous or next to the current speech/audio frame is of a voiced signal class and is a normal decoded frame, or when a current speech/audio frame is of a voiced signal class and is a normal decoded frame, and a speech/audio frame previous or next to the current speech/audio frame is of a generic signal class and is a redundant decoded frame), post processing is performed on a speech/audio decoding parameter of the current speech/audio frame, which helps rectify an energy instability phenomenon caused during the transition between a generic frame and a voiced frame, thereby improving quality of an output speech/audio signal.
In still other embodiments of the present disclosure, when a current speech/audio frame is a redundant decoded frame, a signal class of the current speech/audio frame is not unvoiced, and a signal class of a speech/audio frame next to the current speech/audio frame is unvoiced, a bandwidth extension envelope of the current frame is adjusted, to rectify an energy instability phenomenon in time-domain bandwidth extension, and improve quality of an output speech/audio signal.
An embodiment of the present disclosure further provides a related apparatus for implementing the foregoing solution.
Referring to
The parameter acquiring unit 310 is configured to acquire a speech/audio decoding parameter of a current speech/audio frame, where the foregoing current speech/audio frame is a redundant decoded frame or a speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame.
When the speech/audio frame previous to the foregoing current speech/audio frame is a redundant decoded frame, the current speech/audio frame may be a normal decoded frame, a redundant decoded frame, or an FEC recovery frame.
The post processing unit 320 is configured to perform post processing on the speech/audio decoding parameter of the foregoing current speech/audio frame according to speech/audio parameters of X speech/audio frames to obtain a post-processed speech/audio decoding parameter of the foregoing current speech/audio frame, where the foregoing X speech/audio frames include M speech/audio frames previous to the foregoing current speech/audio frame and/or N speech/audio frames next to the foregoing current speech/audio frame, and M and N are positive integers.
The recovery unit 330 is configured to recover a speech/audio signal of the foregoing current speech/audio frame using the post-processed speech/audio decoding parameter of the foregoing current speech/audio frame.
That a speech/audio frame (for example, the current speech/audio frame or the speech/audio frame previous to the current speech/audio frame) is a normal decoded frame means that a speech/audio parameter, and the like of the foregoing speech/audio frame can be directly obtained from a bitstream of the speech/audio frame by means of decoding. That a speech/audio frame (for example, the current speech/audio frame or the speech/audio frame previous to the current speech/audio frame) is a redundant decoded frame means that a speech/audio parameter, and the like of the speech/audio frame cannot be directly obtained from a bitstream of the speech/audio frame by means of decoding, but redundant bitstream information of the speech/audio frame can be obtained from a bitstream of another speech/audio frame.
The M speech/audio frames previous to the current speech/audio frame refer to M speech/audio frames preceding the current speech/audio frame and immediately adjacent to the current speech/audio frame in a time domain.
For example, M may be equal to 1, 2, 3, or another value. When M=1, the M speech/audio frames previous to the current speech/audio frame are the speech/audio frame previous to the current speech/audio frame, and the speech/audio frame previous to the current speech/audio frame and the current speech/audio frame are two immediately adjacent speech/audio frames, when M=2, the M speech/audio frames previous to the current speech/audio frame are the speech/audio frame previous to the current speech/audio frame and a speech/audio frame previous to the speech/audio frame previous to the current speech/audio frame, and the speech/audio frame previous to the current speech/audio frame, the speech/audio frame previous to the speech/audio frame previous to the current speech/audio frame, and the current speech/audio frame are three immediately adjacent speech/audio frames, and so on.
The N speech/audio frames next to the current speech/audio frame refer to N speech/audio frames following the current speech/audio frame and immediately adjacent to the current speech/audio frame in a time domain.
For example, N may be equal to 1, 2, 3, 4, or another value. When N=1, the N speech/audio frames next to the current speech/audio frame are a speech/audio frame next to the current speech/audio frame, and the speech/audio frame next to the current speech/audio frame and the current speech/audio frame are two immediately adjacent speech/audio frames, when N=2, the N speech/audio frames next to the current speech/audio frame are a speech/audio frame next to the current speech/audio frame and a speech/audio frame next to the speech/audio frame next to the current speech/audio frame, and the speech/audio frame next to the current speech/audio frame, the speech/audio frame next to the speech/audio frame next to the current speech/audio frame, and the current speech/audio frame are three immediately adjacent speech/audio frames, and so on.
The speech/audio decoding parameter may include at least one of a bandwidth extension envelope, an adaptive codebook gain, an algebraic codebook, a pitch period, a spectrum tilt factor, a spectral pair parameter, and the like.
The speech/audio parameter may include a speech/audio decoding parameter, a signal class, and the like.
A signal class of a speech/audio frame may be unvoiced, voiced, generic, transient, inactive, or the like.
The spectral pair parameter may be, for example, at least one of an LSP parameter or an ISP parameter.
It may be understood that in this embodiment of the present disclosure, the post processing unit 320 may perform post processing on at least one speech/audio decoding parameter of a bandwidth extension envelope, an adaptive codebook gain, an algebraic codebook, a pitch period, or a spectral pair parameter of the current speech/audio frame. Further, how many parameters are selected and which parameters are selected for post processing may be determined according to an application scenario and an application environment, which is not limited in this embodiment of the present disclosure.
The post processing unit 320 may perform different post processing on different speech/audio decoding parameters. For example, post processing performed by the post processing unit 320 on the spectral pair parameter of the current speech/audio frame may be adaptive weighting performed using the spectral pair parameter of the current speech/audio frame and a spectral pair parameter of the speech/audio frame previous to the current speech/audio frame, to obtain a post-processed spectral pair parameter of the current speech/audio frame, and post processing performed by the post processing unit 320 on the adaptive codebook gain of the current speech/audio frame may be adjustment such as attenuation performed on the adaptive codebook gain.
It may be understood that functions of function modules of the decoder 300 in this embodiment may be further implemented according to the method in the foregoing method embodiment. For a specific implementation process, refer to related descriptions of the foregoing method embodiment. Details are not described herein. The decoder 300 may be any apparatus that needs to output speeches, for example, a device such as a notebook computer, a tablet computer, or a personal computer, or a mobile phone.
By invoking, using the bus 401, code stored in the memory 403, the processor 402 is configured to perform the steps as described in the previous method embodiments, and the specific implementation process of the processor 402 can refer to related descriptions of the foregoing method embodiments. Details are not described herein.
It may be understood that in this embodiment of the present disclosure, by invoking the code stored in the memory 403, the processor 402 may be configured to perform post processing on at least one speech/audio decoding parameter of a bandwidth extension envelope, an adaptive codebook gain, an algebraic codebook, a pitch period, or a spectral pair parameter of the current speech/audio frame. Further, how many parameters are selected and which parameters are selected for post processing may be determined according to an application scenario and an application environment, which is not limited in this embodiment of the present disclosure.
Different post processing may be performed on different speech/audio decoding parameters. For example, post processing performed on the spectral pair parameter of the current speech/audio frame may be adaptive weighting performed using the spectral pair parameter of the current speech/audio frame and a spectral pair parameter of the speech/audio frame previous to the current speech/audio frame, to obtain a post-processed spectral pair parameter of the current speech/audio frame, and post processing performed on the adaptive codebook gain of the current speech/audio frame may be adjustment such as attenuation performed on the adaptive codebook gain.
A specific post processing manner is not limited in this embodiment of the present disclosure, and specific post processing may be set according to a requirement or according to an application environment and an application scenario.
Referring to
The memory 505 may include a read-only memory (ROM) and a random access memory (RAM), and provide an instruction and data for the processor 501. A part of the memory 505 may further include a nonvolatile RAM (NVRAM).
In some implementation manners, the memory 505 stores the following elements, an executable module or a data structure, or a subset thereof, or an extended set thereof an operating system 5051, including various system programs, and used to implement various basic services and process hardware-based tasks, and an application program module 5052, including various application programs, and configured to implement various application services.
The application program module 5052 includes but is not limited to a parameter acquiring unit 310, a post processing unit 320, a recovery unit 330, and the like.
In this embodiment of the present disclosure, by invoking a program or an instruction stored in the memory 505, the processor 501 may be configured to perform the steps as described in the previous method embodiments.
It may be understood that in this embodiment, by invoking the program or the instruction stored in the memory 505, the processor 501 may perform post processing on at least one speech/audio decoding parameter of a bandwidth extension envelope, an adaptive codebook gain, an algebraic codebook, a pitch period, or a spectral pair parameter of the current speech/audio frame. Further, how many parameters are selected and which parameters are selected for post processing may be determined according to an application scenario and an application environment, which is not limited in this embodiment of the present disclosure.
Different post processing may be performed on different speech/audio decoding parameters. For example, post processing performed on the spectral pair parameter of the current speech/audio frame may be adaptive weighting performed using the spectral pair parameter of the current speech/audio frame and a spectral pair parameter of the speech/audio frame previous to the current speech/audio frame, to obtain a post-processed spectral pair parameter of the current speech/audio frame, and post processing performed on the adaptive codebook gain of the current speech/audio frame may be adjustment such as attenuation performed on the adaptive codebook gain. The specific implementation details about the post processing can refer to related descriptions of the foregoing method embodiments
An embodiment of the present disclosure further provides a computer storage medium, where the computer storage medium may store a program. When being executed, the program includes some or all steps of any speech/audio bitstream decoding method described in the foregoing method embodiments.
It should be noted that, to make the description brief, the foregoing method embodiments are expressed as a series of actions. However, persons skilled in the art should appreciate that the present disclosure is not limited to the described action sequence, because according to the present disclosure, some steps may be performed in other sequences or performed simultaneously.
In the foregoing embodiments, the description of each embodiment has respective focuses. For a part that is not described in detail in an embodiment, refer to related descriptions in other embodiments.
In the several embodiments provided in this application, it should be understood that the disclosed apparatus may be implemented in another manner. For example, the described apparatus embodiment is merely exemplary. For example, the unit division is merely logical function division and may be other division in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not performed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections may be implemented through some interfaces. The indirect couplings or communication connections between the apparatuses or units may be implemented in electronic or other forms.
The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the objectives of the solutions of the embodiments.
In addition, functional units in the embodiments of the present disclosure may be integrated into one processing unit, or each of the units may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in a form of hardware, or may be implemented in a form of a software functional unit.
When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, the integrated unit may be stored in a computer-readable storage medium. Based on such an understanding, the technical solutions of the present disclosure essentially, or the part contributing to other approaches, or all or a part of the technical solutions may be implemented in the form of a software product. The software product is stored in a storage medium and includes several instructions for instructing a computer device (which may be a personal computer, a server, or a network device, and may further be a processor in a computer device) to perform all or a part of the steps of the foregoing methods described in the embodiments of the present disclosure. The foregoing storage medium may include any medium that can store program code, such as a universal serial bus (USB) flash drive, a magnetic disk, a RAM, a ROM, a removable hard disk, or an optical disc.
The foregoing embodiments are merely intended for describing the technical solutions of the present disclosure, but not for limiting the present disclosure. Although the present disclosure is described in detail with reference to the foregoing embodiments, persons of ordinary skill in the art should understand that they may still make modifications to the technical solutions described in the foregoing embodiments or make equivalent replacements to some technical features thereof, without departing from the scope of the technical solutions of the embodiments of the present disclosure.
Number | Date | Country | Kind |
---|---|---|---|
201410108478.6 | Mar 2014 | CN | national |
This application is a continuation of U.S. patent application Ser. No. 15/256,018 filed on Sep. 2, 2016, which is a continuation of International Patent Application No. PCT/CN2015/070594 filed on Jan. 13, 2015, which claims priority to Chinese Patent Application No. 201410108478.6 filed on Mar. 21, 2014. All of the afore-mentioned patent applications are hereby incorporated by reference in their entireties.
Number | Name | Date | Kind |
---|---|---|---|
4731846 | Secrest et al. | Mar 1988 | A |
5615298 | Chen | Mar 1997 | A |
5699478 | Nahumi | Dec 1997 | A |
5717824 | Chhatwal | Feb 1998 | A |
5907822 | Prieto | May 1999 | A |
6385576 | Amada et al. | May 2002 | B2 |
6597961 | Cooke | Jul 2003 | B1 |
6665637 | Bruhn | Dec 2003 | B2 |
6757654 | Westerlund et al. | Jun 2004 | B1 |
6952668 | Kapilow | Oct 2005 | B1 |
6973425 | Kapilow | Dec 2005 | B1 |
6985856 | Wang | Jan 2006 | B2 |
7031926 | Makinen et al. | Apr 2006 | B2 |
7047187 | Cheng et al. | May 2006 | B2 |
7069208 | Wang | Jun 2006 | B2 |
7529673 | Makinen et al. | May 2009 | B2 |
7590525 | Chen | Sep 2009 | B2 |
7693710 | Jelinek | Apr 2010 | B2 |
7933769 | Bessette | Apr 2011 | B2 |
7979271 | Bessette | Jul 2011 | B2 |
8255207 | Vaillancourt et al. | Aug 2012 | B2 |
8364472 | Ehara | Jan 2013 | B2 |
20020091523 | Makinen et al. | Jul 2002 | A1 |
20030200083 | Serizawa et al. | Oct 2003 | A1 |
20040002856 | Bhaskar | Jan 2004 | A1 |
20040117178 | Ozawa | Jun 2004 | A1 |
20040128128 | Wang | Jul 2004 | A1 |
20050154584 | Jelinek | Jul 2005 | A1 |
20050171770 | Yamaura | Aug 2005 | A1 |
20050207502 | Ozawa | Sep 2005 | A1 |
20060088093 | Lakaniemi et al. | Apr 2006 | A1 |
20060173687 | Spindola et al. | Aug 2006 | A1 |
20060271357 | Wang | Nov 2006 | A1 |
20070225971 | Bessette | Sep 2007 | A1 |
20070239462 | Makinen et al. | Oct 2007 | A1 |
20070271480 | Oh et al. | Nov 2007 | A1 |
20070282603 | Bessette | Dec 2007 | A1 |
20080071530 | Ehara | Mar 2008 | A1 |
20080195910 | Sung | Aug 2008 | A1 |
20090076808 | Xu | Mar 2009 | A1 |
20090234644 | Reznik | Sep 2009 | A1 |
20090240490 | Kim et al. | Sep 2009 | A1 |
20090240491 | Reznik | Sep 2009 | A1 |
20090248404 | Ehara et al. | Oct 2009 | A1 |
20100057447 | Ehara | Mar 2010 | A1 |
20100094642 | Zhan et al. | Apr 2010 | A1 |
20100115370 | Laaksonen et al. | May 2010 | A1 |
20100125455 | Wang et al. | May 2010 | A1 |
20100195490 | Nakazawa | Aug 2010 | A1 |
20100312553 | Fang | Dec 2010 | A1 |
20110099004 | Krishnan et al. | Apr 2011 | A1 |
20110125505 | Vaillancourt et al. | May 2011 | A1 |
20110173010 | Lecomte | Jul 2011 | A1 |
20110173011 | Geiger | Jul 2011 | A1 |
20120265523 | Greer | Oct 2012 | A1 |
20130028409 | Li et al. | Jan 2013 | A1 |
20130096930 | Neuendorf | Apr 2013 | A1 |
20130191121 | Rajendran et al. | Jul 2013 | A1 |
20130246068 | Lee | Sep 2013 | A1 |
20140019145 | Moriya et al. | Jan 2014 | A1 |
20150036679 | Deng et al. | Feb 2015 | A1 |
20160343382 | Liu et al. | Nov 2016 | A1 |
20160372122 | Zhang et al. | Dec 2016 | A1 |
Number | Date | Country |
---|---|---|
2179228 | Oct 2004 | CA |
2315699 | Nov 2004 | CA |
1787078 | Jun 2006 | CN |
101189662 | May 2008 | CN |
101256774 | Sep 2008 | CN |
101261836 | Sep 2008 | CN |
101325537 | Dec 2008 | CN |
101379551 | Mar 2009 | CN |
101751925 | Jun 2010 | CN |
101777963 | Jul 2010 | CN |
101866649 | Oct 2010 | CN |
101894558 | Nov 2010 | CN |
102105930 | Jun 2011 | CN |
102438152 | May 2012 | CN |
102726034 | Oct 2012 | CN |
102760440 | Oct 2012 | CN |
102968997 | Mar 2013 | CN |
103325373 | Sep 2013 | CN |
103366749 | Oct 2013 | CN |
103460287 | Dec 2013 | CN |
104751849 | Jul 2015 | CN |
104934035 | Sep 2017 | CN |
1204092 | May 2002 | EP |
1235203 | Aug 2002 | EP |
1564723 | Aug 2005 | EP |
2017829 | Jan 2009 | EP |
1775717 | Jun 2009 | EP |
2003533916 | Nov 2003 | JP |
2004151424 | May 2004 | JP |
2004522178 | Jul 2004 | JP |
2005534950 | Nov 2005 | JP |
2009538460 | Nov 2009 | JP |
20080075050 | Aug 2008 | KR |
101833409 | Feb 2018 | KR |
101839571 | Mar 2018 | KR |
2437172 | Dec 2011 | RU |
2459282 | Aug 2012 | RU |
0063885 | Oct 2000 | WO |
2001086637 | Nov 2001 | WO |
2004038927 | May 2004 | WO |
2004059894 | May 2005 | WO |
2008007698 | Jan 2008 | WO |
2008056775 | May 2008 | WO |
2009008220 | Jan 2009 | WO |
2012158159 | Nov 2012 | WO |
2012161675 | Nov 2012 | WO |
2013016986 | Feb 2013 | WO |
2013109956 | Jul 2013 | WO |
Entry |
---|
Foreign Communication From a Counterpart Application, Chinese Application No. 201710648936.9, Chinese Office Action dated Jan. 17, 2020, 11 pages. |
Foreign Communication From a Counterpart Application, Chinese Application No. 201710648936.9, Chinese Search dated Dec. 6, 2019, 2 pages. |
Foreign Communication From a Counterpart Application, European Application No. 15765124.1, European Office Action dated Jan. 24, 2019, 4 pages. |
Foreign Communication From a Counterpart Application, Japanese Application No. 2016-543574, Japanese Notice of Allowance dated Jan. 8, 2019, 3 pages. |
Foreign Communication From a Counterpart Application, Japanese Application No. 2017-500113, Japanese Notice of Allowance dated May 13, 2019, 3 pages. |
Machine Translation and Abstract of International Application No. WO2013016986, Feb. 7, 2013, 31 pages. |
Foreign Communication From a Counterpart Application, Chinese Application No. 201710648938.8, Chinese Office Action dated Dec. 13, 2019, 10 pages. |
Foreign Communication From a Counterpart Application, Chinese Application No. 201710648938.8, Chinese Search Report dated Dec. 2, 2019, 4 pages. |
Machine Translation and Abstract of Chinese Publication No. CN101751925, Jun. 23, 2010, 26 pages. |
Machine Translation and Abstract of Chinese Publication No. CN101866649, Oct. 20, 2010, 29 pages. |
Machine Translation and Abstract of Chinese Publication No. CN102968997, Mar. 13, 2013, 22 pages. |
Foreign Communication From a Counterpart Application, Indian Application No. 201627030158, Indian Office Action dated Sep. 1, 2020, 6 pages. |
Foreign Communication From a Counterpart Application, Chinese Application No. 201710648937.3, Chinese Office Action dated Feb. 3, 2020, 8 pages. |
Foreign Communication From a Counterpart Application, Chinese Application No. 201710648937.3, Chinese Search Report dated Jan. 13, 2020, 3 pages. |
ITU Recommendation G.718. Series G: Transmission Systems and Media, Digital Systems and Networks. Digital terminal equipments—Coding of voice and audio signals. Frame error robust narrow-band and wideband embedded variable bit-rate coding of speech and audio from 8-32 kbit/s. Telecommunication Standardization Sector of ITU, Jun. 2008, 257 pages. |
Milan Jelinek et al., G.718: A New Embedded Speech and Audio Coding Standard with High Resilience to Error-Prone Transmission Channels. ITU-T Standards, IEEE Communications Magazine, Oct. 2009, 7 pages. |
G.729-based embedded variable bit-rate coder: An 8-32 kbit/s scalable widebandcoder bitstream interoperable with G.729, ITU-T Recommendation G.729.1, May 2006, 100 pages. |
Recommendation ITU-T G.722, 7 kHz audio-coding within 64 kbit/s, Sep. 2012, 262 pages. |
“Wideband coding of speech at around 16 kbit/s using adaptive multi-rate wideband (amr-wb); G.722.2 appendix 1 (Jan. 2002); error concealment of erroneous or lost frames,” Jan. 13, 2002, XP17400860, 18 pages. |
Wideband coding of speech at around 16 kbit/s using adaptive multi-rate wideband (amr-wb); G722.2 (Jul. 2003); XP17464096, 72 pages. |
Enhanced Variable Rate Codec, Speech Service Options 3, 68, 70, 73, and 77 for Wideband Spread Spectrum Digital Systems; 3GPP2 C.S0014-E v1.0 (Dec. 2011), 358 pages. |
Number | Date | Country | |
---|---|---|---|
20190214025 A1 | Jul 2019 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 15256018 | Sep 2016 | US |
Child | 16358237 | US | |
Parent | PCT/CN2015/070594 | Jan 2015 | US |
Child | 15256018 | US |