The present invention is concerned with an audio codec supporting noise synthesis during inactive phases.
The possibility of reducing the transmission bandwidth by taking advantage of inactive periods of speech or other noise sources are known in the art. Such schemes generally use some form of detection to distinguish between inactive (or silence) and active (non-silence) phases. During inactive phases, a lower bitrate is achieved by stopping the transmission of the ordinary data stream precisely encoding the recorded signal, and only sending silence insertion description (SID) updates instead. SID updates may be transmitted in a regular interval or when changes in the background noise characteristics are detected. The SID frames may then be used at the decoding side to generate a background noise with characteristics similar to the background noise during the active phases so that the stopping of the transmission of the ordinary data stream encoding the recorded signal does not lead to an unpleasant transition from the active phase to the inactive phase at the recipient's side.
However, there is still a need for further reducing the transmission rate. An increasing number of bitrate consumers, such as an increasing number of mobile phones, and an increasing number of more or less bitrate intensive applications, such as wireless transmission broadcast, necessitate a steady reduction of the consumed bitrate.
On the other hand, the synthesized noise should closely emulate the real noise so that the synthesis is transparent for the users.
Accordingly, it is one objective of the present invention to provide an audio codec scheme supporting noise generation during inactive phases which enables reducing the transmission bitrate with maintaining the achievable noise generation quality.
According to an embodiment, an audio encoder may have: a background noise estimator configured to continuously update a parametric background noise estimate during an active phase based on an input audio signal; an encoder for encoding the input audio signal into a data stream during the active phase; and a detector configured to detect an entrance of an inactive phase following the active phase based on the input audio signal, wherein the audio encoder is configured to, upon detection of the entrance of the inactive phase, encode into the data stream the parametric background noise estimate as continuously updated during the active phase which the inactive phase detected follows. According to another embodiment, an audio decoder for decoding a data stream so as to reconstruct therefrom an audio signal, the data stream having at least an active phase followed by an inactive phase may have: a background noise estimator configured to continuously update a parametric background noise estimate from the data stream during the active phase; a decoder configured to reconstruct the audio signal from the data stream during the active phase; a parametric random generator; a background noise generator configured to synthesize the audio signal during the inactive phase by controlling the parametric random generator during the inactive phase depending on the parametric background noise estimate; wherein the decoder is configured to, in reconstructing the audio signal from the data stream, shape an excitation signal transform coded into the data stream, according to linear prediction coefficients also coded into the data stream; and wherein the background noise estimator is configured to update the parametric background noise estimate using the excitation signal.
According to another embodiment, an audio encoding method may have the steps of: continuously updating a parametric background noise estimate during an active phase based on an input audio signal; encoding the input audio signal into a data stream during the active phase; detecting an entrance of an inactive phase following the active phase based on the input audio signal; and upon detection of the entrance of the inactive phase, encoding into the data stream the parametric background noise estimate as continuously updated during the active phase which the inactive phase detected follows.
According to still another embodiment, an audio decoding method for decoding a data stream so as to reconstruct therefrom an audio signal, the data stream having at least an active phase followed by an inactive phase, my have the steps of: continuously updating a parametric background noise estimate from the data stream during the active phase; reconstructing the audio signal from the data stream during the active phase; synthesizing the audio signal during the inactive phase by controlling a parametric random generator during the inactive phase depending on the parametric background noise estimate; wherein the reconstruction of the audio signal from the data stream has shaping an excitation signal transform coded into the data stream, according to linear prediction coefficients also coded into the data stream, and wherein the continuous update of the parametric background noise estimate is performed using the excitation signal. Another embodiment may have a computer program having a program code for performing, when running on a computer, the above audio encoding method or the above audio decoding method.
The basic idea of the present invention is that valuable bitrate may be saved with maintaining the noise generation quality within inactive phases, if a parametric background noise estimate is continuously updated during an active phase so that the noise generation may immediately be started with upon the entrance of an inactive phase following the active phase. For example, the continuous update may be performed at the decoding side, and there is no need to preliminarily provide the decoding side with a coded representation of the background noise during a warm-up phase immediately following the detection of the inactive phase which provision would consume valuable bitrate, since the decoding side has continuously updated the parametric background noise estimate during the active phase and is, thus, prepared at any time to immediately enter the inactive phase with an appropriate noise generation. Likewise, such a warm-up phase may be avoided if the parametric background noise estimate is done at the encoding side. Instead of preliminarily continuing with providing the decoding side with a conventionally coded representation of the background noise upon detecting the entrance of the inactive phase in order to learn the background noise and inform the decoding side after the learning phase accordingly, the encoder is able to provide the decoder with the necessitated parametric background noise estimate immediately upon detecting the entrance of the inactive phase by falling back on the parametric background noise estimate continuously updated during the past active phase thereby avoiding the bitrate consuming preliminary further prosecution of supererogatorily encoding the background noise.
In accordance with specific embodiments of the present invention, a more realistic noise generation at moderate overhead in terms of, for example, bitrate and computational complexity is achieved. In particular, in accordance with these embodiments, the spectral domain is used in order to parameterize the background noise thereby yielding a background noise synthesis which is more realistic and thus leads to a more transparent active to inactive phase switching. Moreover, it has been found out that parameterizing the background noise in the spectral domain enables separating noise from the useful signal and accordingly, parameterizing the background noise in the spectral domain has an advantage when combined with the aforementioned continuous update of the parametric background noise estimate during the active phases as a better separation between noise and useful signal may be achieved in the spectral domain so that no additional transition from one domain to the other is necessary when combining both advantageous aspects of the present application.
Embodiments of the present application are described below with respect to the Figures among which:
The background noise estimator 12 is configured to continuously update a parametric background noise estimate during an active phase 24 based on an input audio signal entering the audio encoder 10 at input 18. Although
The encoding engine 14 is configured to encode the input audio signal arriving at input 18 into a data stream during the active phase 24. The active phase shall encompass all times where a useful information is contained within the audio signal such as speech or other useful sound of a noise source. On the other hand, sounds with an almost time-invariant characteristic such as a time-invariance spectrum as caused, for example, by rain or traffic in the background of a speaker, shall be classified as background noise and whenever merely this background noise is present, the respective time period shall be classified as an inactive phase 28. The detector 16 is responsible for detecting the entrance of an inactive phase 28 following the active phase 24 based on the input audio signal at input 18. In other words, the detector 16 distinguishes between two phases, namely active phase and inactive phase wherein the detector 16 decides as to which phase is currently present. The detector 16 informs encoding engine 14 about the currently present phase and as already mentioned, encoding engine 14 performs the encoding of the input audio signal into the data stream during the active phases 24. Detector 16 controls switch 22 accordingly so that the data stream output by encoding engine 14 is output at output 20. During inactive phases, the encoding engine 14 may stop encoding the input audio signal. At least, the data stream outputted at output 20 is no longer fed by any data stream possibly output by the encoding engine 14. In addition to that, the encoding engine 14 may only perform minimum processing to support the estimator 12 with some state variable updates. This action will greatly reduce the computational power. Switch 22 is, for example, set such that the output of estimator 12 is connected to output 20 instead of the encoding engine's output. This way, valuable transmission bitrate for transmitting the bitstream output at output 20 is reduced.
The background noise estimator 12 is configured to continuously update a parametric background noise estimate during the active phase 24 based on the input audio signal 18 as already mentioned above, and due to this, estimator 12 is able to insert into the data stream 30 output at output 20 the parametric background noise estimate as continuously updated during the active phase 24 immediately following the transition from the active phase 24 to the inactive phase 28, i.e. immediately upon the entrance into the inactive phase 28. Background noise estimator 12 may, for example, insert a silence insertion descriptor frame 32 into the data stream 30 immediately following the end of the active phase 24 and immediately following the time instant 34 at which the detector 16 detected the entrance of the inactive phase 28. In other words, there is no time gap between the detectors detection of the entrance of the inactive phase 28 and the insertion of the SID 32 necessary due to the background noise estimator's continuous update of the parametric background noise estimate during the active phase 24.
Thus, summarizing the above description the audio encoder 10 of
The background noise estimator 12 continuously updates the parametric background noise estimate during the active phase 24. Accordingly, the background noise estimator 12 may be configured to distinguish between a noise component and a useful signal component within the input audio signal in order to determine the parametric background noise estimate merely from the noise component. According to the embodiments further described below, the background noise estimator 12 may perform this updating in a spectral domain such as a spectral domain also used for transform coding within encoding engine 14. However, other alternatives are also available, such as the time-domain. If the spectral domain, same may be a lapped transform domain such as an MDCT domain, or a filterbank domain such as a complex valued filterbank domain such as an QMF domain.
Moreover, the background noise estimator 12 may perform the updating based on an excitation or residual signal obtained as an intermediate result within encoding engine 14 during, for example, predictive and/or transform coding rather than the audio signal as entering input 18 or as lossy coded into the data stream. By doing so, a large amount of the useful signal component within the input audio signal would already have been removed so that the detection of the noise component is easier for the background noise estimator 12.
During the active phase 24, detector 16 is also continuously running to detect an entrance of the inactive phase 28. The detector 16 may be embodied as a voice/sound activity detector (VAD/SAD) or some other means which decides whether a useful signal component is currently present within the input audio signal or not. A base criterion for detector 16 in order to decide whether an active phase 24 continues could be checking whether a low-pass filtered power of the input audio signal remains below a certain threshold, assuming that an inactive phase is entered as soon as the threshold is exceeded.
Independent from the exact way the detector 16 performs the detection of the entrance of the inactive phase 28 following the active phase 24, the detector 16 immediately informs the other entities 12, 14 and 22 of the entrance of the inactive phase 28. Due to the background noise estimator's continuous update of the parametric background noise estimate during the active phase 24, the data stream 30 output at output 20 may be immediately prevented from being further fed from encoding engine 14. Rather, the background noise estimator 12 would, immediately upon being informed of the entrance of the inactive phase 28, insert into the data stream 30 the information on the last update of the parametric background noise estimate in the form of the SID frame 32. That is, SID frame 32 could immediately follow the last frame of encoding engine which encodes the frame of the audio signal concerning the time interval within which the detector 16 detected the inactive phase entrance.
Normally, the background noise does not change very often. In most cases, the background noise tends to be something invariant in time. Accordingly, after the background noise estimator 12 inserted SID frame 32 immediately after the detector 16 detecting the beginning of the inactive phase 28, any data stream transmission may be interrupted so that in this interruption phase 34, the data stream 30 does not consume any bitrate or merely a minimum bitrate necessitated for some transmission purposes. In order to maintain a minimum bitrate, background noise estimator 12 may intermittently repeat the output of SID 32.
However, despite the tendency of background noise to not change in time, it nevertheless may happen that the background noise changes. For example, imagine a mobile phone user leaving the car so that the background noise changes from motor noise to traffic noise outside the car during the user phoning. In order to track such changes of the background noise, the background noise estimator 12 may be configured to continuously survey the background noise even during the inactive phase 28. Whenever the background noise estimator 12 determines that the parametric background noise estimate changes by an amount which exceeds some threshold, background estimator 12 may insert an updated version of parametric background noise estimate into the data stream 20 via another SID 38, whereinafter another interruption phase 40 may follow until, for example, another active phase 42 starts as detected by detector 16 and so forth. Naturally, SID frames revealing the currently updated parametric background noise estimate may alternatively or additionally interspersed within the inactive phases in an intermediate manner independent from changes in the parametric background noise estimate.
Obviously, the data stream 44 output by encoding engine 14 and indicated in
As will be explained in more detail below with regard to more specific embodiments, the encoding engine 14 may be configured to, in encoding the input audio signal, predictively code the input audio signal into linear prediction coefficients and an excitation signal with transform coding the excitation signal and coding the linear prediction coefficients into the data stream 30 and 44, respectively. One possible implementation is shown in
Based on the linear prediction coefficients determined by the linear prediction analysis module 60, the data stream output at output 58 is fed with respective information on the LPCs, and the frequency domain noise shaper is controlled so as to spectrally shape the audio signal's spectrogram in accordance with a transfer function corresponding to the transfer function of a linear prediction analysis filter determined by the linear prediction coefficients output by module 60. A quantization of the LPCs for transmitting them in the data stream may be performed in the LSP/LSF domain and using interpolation so as to reduce the transmission rate compared to the analysis rate in the analyzer 60. Further, the LPC to spectral weighting conversion performed in the FDNS may involve applying a ODFT onto the LPCs and appliying the resulting weighting values onto the transformer's spectra as divisor.
Quantizer 54 then quantizes the transform coefficients of the spectrally formed (flattened) spectrogram. For example, the transformer 50 uses a lapped transform such as an MDCT in order to transfer the audio signal from time domain to spectral domain, thereby obtaining consecutive transforms corresponding to overlapping windowed portions of the input audio signal which are then spectrally formed by the frequency domain noise shaper 52 by weighting these transforms in accordance with the LP analysis filter's transfer function.
The shaped spectrogram may be interpreted as an excitation signal and as it is illustrated by dashed arrow 62, the background noise estimator 12 may be configured to update the parametric background noise estimate using this excitation signal. Alternatively, as indicated by dashed arrow 64, the background noise estimator 12 may use the lapped transform representation as output by transformer 50 as a basis for the update directly, i.e. without the frequency domain noise shaping by noise shaper 52.
Further details regarding possible implementation of the elements shown in
Before, however, describing these more detailed embodiments, reference is made to
The audio decoder 80 of
The parametric random generator 94 may comprise one or more true or pseudo random number generators, the sequence of values output by which may conform to a statistical distribution which may be parametrically set via the background noise generator 96.
The background noise generator 96 is configured to synthesize the audio signal 98 during the inactive phase 88 by controlling the parametric random generator 94 during the inactive phase 88 depending on the parametric background noise estimate as obtained from the background noise estimator 90. Although both entities 96 and 94 are shown to be serially connected, the serial connection should not be interpreted as being limiting. The generators 96 and 94 could be interlinked. In fact, generator 94 could be interpreted to be part of generator 96.
Thus, the mode of operation of the audio decoder 80 of
In any case, the entrance of the inactive phase 88 occurs very suddenly, but this is not a problem since the background noise estimator 90 has continuously updated the parametric background noise estimate during the active phase 86 on the basis of the data stream portion 102. Due to this, the background noise estimator 90 is able to provide the background noise generator 96 with the newest version of the parametric background noise estimate as soon as the inactive phase 88 starts at 106. Accordingly, from time instant 106 on, decoding engine 92 stops outputting any audio signal reconstruction as the decoding engine 92 is not further fed with a data stream portion 102, but the parametric random generator 94 is controlled by the background noise generator 96 in accordance with a parametric background noise estimate such that an emulation of the background noise may be output at output 84 immediately following time instant 106 so as to gaplessly follow the reconstructed audio signal as output by decoding engine 92 up to time instant 106. Cross-fading may be used to transit from the last reconstructed frame of the active phase as output by engine 92 to the background noise as determined by the recently updated version of the parametric background noise estimate.
As the background noise estimator 90 is configured to continuously update the parametric background noise estimate from the data stream 104 during the active phase 86, same may be configured to distinguish between a noise component and a useful signal component within the version of the audio signal as reconstructed from the data stream 104 in the active phase 86 and to determine the parametric background noise estimate merely from the noise component rather than the useful signal component. The way the background noise estimator 90 performs this distinguishing/separation corresponds to the way outlined above with respect to the background noise estimator 12. For example, the excitation or residual signal internally reconstructed from the data stream 104 within decoding engine 92 may be used.
Similar to
With regard to
In accordance with
In the case of transmitting zero frames, i.e. during the interruption phase of the inactive phase, the detector 16 informs the background noise estimator 12, in particular the quantizer 152, to stop processing and to not send anything to the bitstream packager 154.
In accordance with
The mode of operation of the encoder of
In particular, the encoder of
In particular, the decoder of
The mode of operation and functionality of the individual modules of
In particular, the transformer 140 spectrally decomposes the input signal into a spectrogram such as by using a lapped transform. A noise estimator 146 is configured to determine noise parameters therefrom. Concurrently, the voice or sound activity detector 16 evaluates the features derived from the input signal so as to detect whether a transition from an active phase to an inactive phase or vice versa takes place. These features used by the detector 16 may be in the form of transient/onset detector, tonality measurement, and LPC residual measurement. The transient/onset detector may be used to detect attack (sudden increase of energy) or the beginning of active speech in a clean environment or denoised signal; the tonality measurement may be used to distinguish useful background noise such as siren, telephone ringing and music; LPC residual may be used to get an indication of speech presence in the signal. Based on these features, the detector 16 can roughly give an information whether the current frame can be classified for example, as speech, silence, music, or noise.
While the noise estimator 146 may be responsible for distinguishing the noise within the spectrogram from the useful signal component therein, such as proposed in [R. Martin, Noise Power Spectral Density Estimation Based on Optimal Smoothing and Minimum Statistics, 2001], parameter estimator 148 may be responsible for statistically analyzing the noise components and determining parameters for each spectral component, for example, based on the noise component.
The noise estimator 146 may, for example, be configured to search for local minima in the spectrogram and the parameter estimator 148 may be configured to determine the noise statistics at these portions assuming that the minima in the spectrogram are primarily an attribute of the background noise rather than foreground sound.
As an intermediate note it is emphasized that it may also be possible to perform the estimation by noise estimator without the FDNS 142 as the minima do also occur in the non-shaped spectrum. Most of the description of
Parameter quantizer 152, in turn, may be configured to parameterize the parameters estimated by parameter estimator 148. For example, the parameters may describe a mean amplitude and a first or higher order momentum of a distribution of the spectral values within the spectrogram of the input signal as far as the noise component is concerned. In order to save bitrate, the parameters may be forwarded to the data stream for insertion into the same within SID frames in a spectral resolution lower than the spectral resolution provided by transformer 140.
The stationarity measurer 150 may be configured to derive a measure of stationarity for the noise signal. The parameter estimator 148 in turn may use the measure of stationarity so as to decide whether or not a parameter update should be initiated by sending another SID frame such as frame 38 in
Module 152 quantizes the parameters calculated by parameter estimator 148 and LP analysis 144 and signals this to the decoding side. In particular, prior to quantizing, spectral components may be grouped into groups. Such grouping may be selected in accordance with psychoacoustical aspects such as conforming to the bark scale or the like. The detector 16 informs the quantizer 152 whether the quantization is needed to be performed or not. In case of no quantization is needed, zero frames should follow.
When transferring the description onto a concrete scenario of switching from an active phase to an inactive phase, then the modules of
During an active phase, encoding engine 14 keeps on coding the audio signal via packager into bitstream. The encoding may be performed frame-wise. Each frame of the data stream may represent one time portion/interval of the audio signal. The audio encoder 14 may be configured to encode all frames using LPC coding. The audio encoder 14 may be configured to encode some frames as described with respect to
In parallel, noise estimator 146 inspects the LPC flattened (LPC analysis filtered) spectra so as to identify the minima kmin within the TCX sprectrogram represented by the sequence of these spectra. Of course, these minima may vary in time t, i.e. kmin(t). Nevertheless, the minima may form traces in the spectrogram output by FDNS 142, and thus, for each consecutive spectrum i at time ti, the minima may be associatable with the minima at the preceding and succeeding spectrum, respectively.
The parameter estimator then derives background noise estimate parameters therefrom such as, for example, a central tendency (mean average, median or the like) m and/or dispersion (standard deviation, variance or the like) d for different spectral components or bands. The derivation may involve a statistical analysis of the consecutive spectral coefficients of the spectra of the spectrogram at the minima, thereby yielding m and d for each minimum at kmin. Interpolation along the spectral dimension between the aforementioned spectrum minima may be performed so as to obtain m and d for other predetermined spectral components or bands. The spectral resolution for the derivation and/or interpolation of the central tendency (mean average) and the derivation of the dispersion (standard deviation, variance or the like) may differ.
The just mentioned parameters are continuously updated per spectrum output by FDNS 142, for example.
As soon as detector 16 detects the entrance of an inactive phase, detector 16 may inform engine 14 accordingly so that no further active frames are forwarded to packager 154. However, the quantizer 152 outputs the just-mentioned statistical noise parameters in a first SID frame within the inactive phase, instead. The first SID frame may or may not comprise an update of the LPCs. If an LPC update is present, same may be conveyed within the data stream in the SID frame 32 in the format used in portion 44, i.e. during active phase, such as using quantization in the LSF/LSP domain, or differently, such as using spectral weightings corresponding to the LPC analysis or LPC synthesis filter's transfer function such as those which would have been applied by FDNS 142 within the framework of encoding engine 14 in proceeding with an active phase.
During the inactive phase, noise estimator 146, parameter estimator 148 and stationarity measurer 150 keep on co-operating so as to keep the decoding side updated on changes in the background noise. In particular, measurer 150 checks the spectral weighting defined by the LPCs, so as to identify changes and inform the estimator 148 when an SID frame should be sent to the decoder. For example, the measurer 150 could activate estimator accordingly whenever the afore-mentioned measure of stationarity indicates a degree of fluctuation in the LPCs which exceeds a certain amount. Additionally or alternatively, estimator could be triggered to send the updated parameters an a regular basis. Between these SID update frames 40, nothing would be send in the data streams, i.e. “zero frames”.
At the decoder side, during the active phase, the decoding engine 160 assumes responsibility for reconstructing the audio signal. As soon as the inactive phase starts, the adaptive parameter random generator 164 uses the dequantized random generator parameters sent during the inactive phase within the data stream from parameter quantizer 150 to generate random spectral components, thereby forming a random spectrogram which is spectrally formed within the spectral energy processor 166 with the synthesizer 168 then performing a retransformation from the spectral domain into the time domain. For spectral formation within FDNS 166, either the most recent LPC coefficients from the most recent active frames may be used or the spectral weighting to be applied by FDNS 166 may be derived therefrom by extrapolation, or the SID frame 32 itself may convey the information. By this measure, at the beginning of the inactive phase, the FDNS 166 continues to spectrally weight the inbound spectrum in accordance with a transfer function of an LPC synthesis filter, with the LPS defining the LPC synthesis filter being derived from the active data portion 44 or SID frame 32. However, with the beginning of the inactive phase, the spectrum to be shaped by FDNS 166 is the randomly generated spectrum rather than a transform coded on as in case of TCX frame coding mode. Moreover, the spectral shaping applied at 166 is merely discontinuously updated by use of the SID frames 38. An interpolation or fading could be performed to gradually switch from one spectral shaping definition to the next during the interruption phases 36.
As shown in
Briefly referring back to
Similar to the relationship between the embodiment of
While elements 146, 148 and 150 act as the background noise estimator 90 of
Summarizing
The random generator 164 may be controlled such that same models the type of noise as closely as possible. This could be accomplished if the target noise is known in advance. Some applications may allow this. In many realistic applications where a subject may encounter different types of noise, an adaptive method is necessitated as shown in
To make the parameter random generator adaptive, the random generator parameter estimator 146 adequately controls the random generator. Bias compensation may be included in order to compensate for the cases where the data is deemed to be statistically insufficient. This is done to generate a statistically matched model of the noise based on the past frames and it will update the estimated parameters. An example is given where the random generator 164 is supposed to generate a Gaussian noise. In this case, for example, only the mean and variance parameters may be needed and a bias can be calculated and applied to those parameters. A more advanced method can handle any type of noise or distribution and the parameters are not necessarily the moments of a distribution.
For the non-stationary noise, it needs to have a stationarity measure and a less adaptive parametric random generator can then be used. The stationarity measure determined by measurer 148 can be derived from the spectral shape of the input signal using various methods like, for example, the Itakura distance measure, the Kullback-Leibler distance measure, etc.
To handle the discontinuous nature of noise updates sent through SID frames such as illustrated by 38 in
As already noted above,
Among the scenarios shown in
All of the above embodiments could be combined with bandwidth extension techniques such as spectral band replication (SBR), although bandwidth extension in general may be used.
To illustrate this, see
It should be noted that the bandwidth extension information generated in accordance with the embodiments of
Thus,
That is, the bandwidth extension coding may be performed differently in the QMF or spectral domain depending on the silence or active phase being present. In the active phase, i.e. during active frames, regular SBR encoding is carried out by the encoder 202, resulting in a normal SBR data stream which accompanies data streams 44 and 102, respectively. In inactive phases or during frames classified as SID frames, only information about the spectral envelope, represented as energy scale factors, may be extracted by application of a time/frequency grid which exhibits a very low frequency resolution, and for example the lowest possible time resolution. The resulting scale factors might be efficiently coded by encoder 212 and written to the data stream. In zero frames or during interruption phases 36, no side information may be written to the data stream by the spectral band replication encoding module 206, and therefore no energy calculation may be carried out by calculator 210.
In conformity with
As shown in
As in accordance with the embodiment of
Further, the SBR decoder 224 of
Modules 230 to 236 operate as follows. Spectral decomposer 230 spectrally decomposes the time domain input signal so as to obtain a reconstructed low frequency portion. The HF generator 232 generates a high frequency replica portion based on the reconstructed low frequency portion and the envelope adjuster 234 spectrally forms or shapes the high frequency replica using a representation of a spectral envelope of the high frequency portion as conveyed via the SBR data stream portion and provided by modules not yet discussed but shown in
As already mentioned above with respect to
Thus, at the decoder side the following processing may be carried out. In active frames or during active phases, regular spectral band replication processing may be applied. During these active periods, the scale factors from the data stream, which are typically available for a higher number of scale factor bands as compared to comfort noise generating processing, are converted to the comfort noise generating frequency resolution by the scale factor combiner 242. The scale factor combiner combines the scale factors for the higher frequency resolution to result in a number of scale factors compliant to CNG by exploiting common frequency band borders of the different frequency band tables. The resulting scale factor values at the output of the scale factor combining unit 242 are stored for the reuse in zero frames and later reproduction by restorer 252 and are subsequently used for updating the filtering unit 246 for the CNG operating mode. In SID frames, a modified SBR data stream reader is applied which extracts the scale factor information from the data stream. The remaining configuration of the SBR processing is initialized with predefined values, the time/frequency grid is initialized to the same time/frequency resolution used in the encoder. The extracted scale factors are fed into filtering unit 246, where, for example, one IIR smoothing filter interpolates the progression of the energy for one low resolution scale factor band over time. In case of zero frames, no payload is read from the bitstream and the SBR configuration including the time/frequency grid is the same as is used in SID frames. In zero frames, the smoothing filters in filtering unit 246 are fed with a scale factor value output from the scale factor combining unit 242 which have been stored in the last frame containing valid scale factor information. In case the current frame is classified as an inactive frame or SID frame, the comfort noise is generated in TCX domain and transformed back to the time domain. Subsequently, the time domain signal containing the comfort noise is fed into the QMF analysis filterbank 230 of the SBR module 224. In QMF domain, bandwidth extension of the comfort noise is performed by means of copy-up transposition within HF generator 232 and finally the spectral envelope of the artificially created high frequency part is adjusted by application of energy scale factor information in the envelope adjuster 234. These energy scale factors are obtained by the output of the filtering unit 246 and are scaled by the gain adjustment unit 248 prior to application in the envelope adjuster 234. In this gain adjustment unit 248, a gain value for scaling the scale factors is calculated and applied in order to compensate for huge energy differences at the border between the low frequency portion and the high frequency content of the signal.
The embodiments described above are commonly used in the embodiments of
The audio encoder of
The spectral bandwidth extension data output by estimator 260 describe the spectral envelope of the high frequency portion of the spectrogram or spectrum output by the QMF analysis filterbank 200, which is then encoded, such as by entropy coding, by SBR encoder 264. Data stream multiplexer 266 inserts the spectral bandwidth extension data in active phases into the data stream output at an output 268 of the multiplexer 266.
Detector 270 detects whether currently an active or inactive phase is active. Based on this detection, an active frame, an SID frame or a zero frame, i.e. inactive frame, is to currently be output. In other words, module 270 decides whether an active phase or an inactive phase is active and if the inactive phase is active, whether or not an SID frame is to be output. The decisions are indicated in
SID frames (or, to be more precise, the information to be conveyed by same) are forwarded to SID encoder 274, which assumes responsibility for the functionalities of module 152 of
Multiplexer 266 multiplexes the respective encoded information into the data stream at output 268.
The audio decoder of
Thus, during active phases, the core decoder 92 reconstructs the low-frequency portion of the audio signal including both noise and useful signal components. The QMF analysis filterbank 282 spectrally decomposes the reconstructed signal and the spectral bandwidth extension module 284 uses spectral bandwidth extension information within the data stream and active frames, respectively, in order to add the high frequency portion. The noise estimator 286, if present, performs the noise estimation based on a spectrum portion as reconstructed by the core decoder, i.e. the low frequency portion. In inactive phases, the SID frames convey information parametrically describing the background noise estimate derived by the noise estimation 262 at the encoder side. The parameter updater 292 may primarily use the encoder information in order to update its parametric background noise estimate, using the information provided by the noise estimator 286 primarily as a fallback position in case of transmission loss concerning SID frames. The QMF synthesis filterbank 288 converts the spectrally decomposed signal as output by the spectral band replication module 284 in active phases and the comfort noise generated signal spectrum in the time domain. Thus,
In particular, in accordance with the embodiments of
Ideally, note that the noise estimation 262 applied at the encoder side should be able to operate during both inactive (i.e., noise-only) and active periods (typically containing noisy speech) so that the comfort noise parameters can be updated immediately at the end of each active period. In addition, noise estimation might be used at the decoder side as well. Since noise-only frames are discarded in a DTX-based coding/decoding system, the noise estimation at the decoder side is favorably able to operate on noisy speech contents. The advantage of performing the noise estimation at the decoder side, in addition to the encoder side, is that the spectral shape of the comfort noise can be updated even when the packet transmission from the encoder to the decoder fails for the first SID frame(s) following a period of activity.
The noise estimation should be able to accurately and rapidly follow variations of the background noise's spectral content and ideally it should be able to perform during both active and inactive frames, as stated above. One way to achieve these goals is to track the minima taken in each band by the power spectrum using a sliding window of finite length, as proposed in [R. Martin, Noise Power Spectral Density Estimation Based on Optimal Smoothing and Minimum Statistics, 2001]. The idea behind it is that the power of a noisy-speech spectrum frequently decays to the power of the background noise, e.g., between words or syllables. Tracking the minimum of the power spectrum provides therefore an estimate of the noise floor in each band, even during speech activity. However, these noise floors are underestimated in general. Furthermore, they do not allow to capture quick fluctuations of the spectral powers, especially sudden energy increases.
Nevertheless, the noise floor computed as described above in each band provides very useful side-information to apply a second stage of noise estimation. In fact, we can expect the power of a noisy spectrum to be close to the estimated noise floor during inactivity, whereas the spectral power will be far above the noise floor during activity. The noise floors computed separately in each band can hence be used as rough activity detectors for each band. Based on this knowledge, the background noise power can be easily estimated as a recursively smoothed version of the power spectrum as follows:
σN2(m,k)=β(m,k)*σN2(m−1,k)+(1−β(m,k))*σx2(m,k)
where σx2(m,k) denotes the power spectral density of the input signal at the frame m and band k, σN2(m,k) refers the noise power estimate, and β(m,k) is a forgetting factor (between 0 and 1) controlling the amount of smoothing for each band and each frame separately. Using the noise floor information to reflect the activity status, it should take a small value during inactive periods (i.e., when the power spectrum is close to the noise floor), whereas a high value should be chosen to apply more smoothing (ideally keeping σN2(m,k) constant) during active frames. To achieve this, a soft decision may be made by computing the forgetting factors as follows:
where σNF2 is the noise floor power and a is a control parameter. A higher value for a results in larger forgetting factors and hence causes overall more smoothing.
Thus, a Comfort Noise Generation (CNG) concept has been described where the artificial noise is produced at the decoder side in a transform domain. The above embodiments can be applied in combination with virtually any type of spectro-temporal analysis tool (i.e., a transform or filterbank) decomposing a time-domain signal into multiple spectral bands.
Thus, the above embodiments, inter alias, described a TCX-based CNG where a basic comfort noise generator employs random pulses to model the residual.
Although some aspects have been described in the context of an apparatus, it is clear that these aspects also represent a description of the corresponding method, where a block or device corresponds to a method step or a feature of a method step. Analogously, aspects described in the context of a method step also represent a description of a corresponding block or item or feature of a corresponding apparatus. Some or all of the method steps may be executed by (or using) a hardware apparatus, like for example, a microprocessor, a programmable computer or an electronic circuit. In some embodiments, some one or more of the most important method steps may be executed by such an apparatus.
Depending on certain implementation requirements, embodiments of the invention can be implemented in hardware or in software. The implementation can be performed using a digital storage medium, for example a floppy disk, a DVD, a Blu-Ray, a CD, a ROM, a PROM, an EPROM, an EEPROM or a FLASH memory, having electronically readable control signals stored thereon, which cooperate (or are capable of cooperating) with a programmable computer system such that the respective method is performed. Therefore, the digital storage medium may be computer readable.
Some embodiments according to the invention comprise a data carrier having electronically readable control signals, which are capable of cooperating with a programmable computer system, such that one of the methods described herein is performed.
Generally, embodiments of the present invention can be implemented as a computer program product with a program code, the program code being operative for performing one of the methods when the computer program product runs on a computer. The program code may for example be stored on a machine readable carrier.
Other embodiments comprise the computer program for performing one of the methods described herein, stored on a machine readable carrier.
In other words, an embodiment of the inventive method is, therefore, a computer program having a program code for performing one of the methods described herein, when the computer program runs on a computer.
A further embodiment of the inventive methods is, therefore, a data carrier (or a digital storage medium, or a computer-readable medium) comprising, recorded thereon, the computer program for performing one of the methods described herein. The data carrier, the digital storage medium or the recorded medium are typically tangible and/or non-transitionary.
A further embodiment of the inventive method is, therefore, a data stream or a sequence of signals representing the computer program for performing one of the methods described herein. The data stream or the sequence of signals may for example be configured to be transferred via a data communication connection, for example via the Internet.
A further embodiment comprises a processing means, for example a computer, or a programmable logic device, configured to or adapted to perform one of the methods described herein.
A further embodiment comprises a computer having installed thereon the computer program for performing one of the methods described herein.
A further embodiment according to the invention comprises an apparatus or a system configured to transfer (for example, electronically or optically) a computer program for performing one of the methods described herein to a receiver. The receiver may, for example, be a computer, a mobile device, a memory device or the like. The apparatus or system may, for example, comprise a file server for transferring the computer program to the receiver.
In some embodiments, a programmable logic device (for example a field programmable gate array) may be used to perform some or all of the functionalities of the methods described herein. In some embodiments, a field programmable gate array may cooperate with a microprocessor in order to perform one of the methods described herein. Generally, the methods may be performed by any hardware apparatus.
While this invention has been described in terms of several embodiments, there are alterations, permutations, and equivalents which will be apparent to others skilled in the art and which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and compositions of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations, and equivalents as fall within the true spirit and scope of the present invention.
This application is a continuation of copending International Application No. PCT/EP2012/052462, filed Feb. 14, 2012, which is incorporated herein by reference in its entirety, and additionally claims priority from U.S. Provisional Application No. 61/442,632, filed Feb. 14, 2011, which is also incorporated herein by reference in its entirety.
Number | Name | Date | Kind |
---|---|---|---|
5414796 | Jacobs et al. | May 1995 | A |
5537510 | Kim | Jul 1996 | A |
5606642 | Stautner et al. | Feb 1997 | A |
5754733 | Gardner et al. | May 1998 | A |
5848391 | Bosi et al. | Dec 1998 | A |
5953698 | Hayata | Sep 1999 | A |
5960389 | Jarvinen et al. | Sep 1999 | A |
5982817 | Wuppermann | Nov 1999 | A |
6070137 | Bloebaum et al. | May 2000 | A |
6134518 | Cohen et al. | Oct 2000 | A |
6173257 | Gao | Jan 2001 | B1 |
6236960 | Peng et al. | May 2001 | B1 |
6317117 | Goff | Nov 2001 | B1 |
6532443 | Nishiguchi et al. | Mar 2003 | B1 |
6757654 | Westerlund et al. | Jun 2004 | B1 |
6879955 | Rao | Apr 2005 | B2 |
7124079 | Johansson et al. | Oct 2006 | B1 |
7249014 | Kannan et al. | Jul 2007 | B2 |
7280959 | Bessette | Oct 2007 | B2 |
7343283 | Ashley et al. | Mar 2008 | B2 |
7363218 | Jabri et al. | Apr 2008 | B2 |
7519535 | Spindola | Apr 2009 | B2 |
7519538 | Villemoes et al. | Apr 2009 | B2 |
7536299 | Cheng et al. | May 2009 | B2 |
7610197 | Cruz-Zeno et al. | Oct 2009 | B2 |
7707034 | Sun et al. | Apr 2010 | B2 |
7747430 | Makinen | Jun 2010 | B2 |
7788105 | Miseki | Aug 2010 | B2 |
7873511 | Herre et al. | Jan 2011 | B2 |
7933769 | Bessette | Apr 2011 | B2 |
7979271 | Bessette | Jul 2011 | B2 |
7987089 | Krishnan et al. | Jul 2011 | B2 |
8121831 | Oh et al. | Feb 2012 | B2 |
8160274 | Bongiovi | Apr 2012 | B2 |
8255207 | Vaillancourt et al. | Aug 2012 | B2 |
8428936 | Mittal et al. | Apr 2013 | B2 |
8566106 | Salami et al. | Oct 2013 | B2 |
8630862 | Geiger et al. | Jan 2014 | B2 |
8630863 | Son et al. | Jan 2014 | B2 |
20020078771 | Kreichauf et al. | Jun 2002 | A1 |
20020111799 | Bernard | Aug 2002 | A1 |
20020184009 | Heikkinen | Dec 2002 | A1 |
20030009325 | Kirchherr et al. | Jan 2003 | A1 |
20030033136 | Lee | Feb 2003 | A1 |
20030046067 | Gradl | Mar 2003 | A1 |
20030078771 | Jung et al. | Apr 2003 | A1 |
20030225576 | Li et al. | Dec 2003 | A1 |
20040093368 | Lee et al. | May 2004 | A1 |
20040225505 | Andersen et al. | Nov 2004 | A1 |
20050065785 | Bessette | Mar 2005 | A1 |
20050091044 | Ramo et al. | Apr 2005 | A1 |
20050096901 | Uvliden et al. | May 2005 | A1 |
20050130321 | Nicholson et al. | Jun 2005 | A1 |
20050131696 | Wang et al. | Jun 2005 | A1 |
20050154584 | Jelinek et al. | Jul 2005 | A1 |
20050240399 | Makinen | Oct 2005 | A1 |
20050278171 | Suppappola et al. | Dec 2005 | A1 |
20060116872 | Byun et al. | Jun 2006 | A1 |
20060206334 | Kapoor et al. | Sep 2006 | A1 |
20060271356 | Vos | Nov 2006 | A1 |
20060293885 | Gournay et al. | Dec 2006 | A1 |
20070016404 | Kim et al. | Jan 2007 | A1 |
20070050189 | Cruz-Zeno et al. | Mar 2007 | A1 |
20070100607 | Villemoes | May 2007 | A1 |
20070147518 | Bessette | Jun 2007 | A1 |
20070171931 | Manjunath et al. | Jul 2007 | A1 |
20070225971 | Bessette | Sep 2007 | A1 |
20070253577 | Yen et al. | Nov 2007 | A1 |
20070282603 | Bessette | Dec 2007 | A1 |
20080010064 | Takeuchi et al. | Jan 2008 | A1 |
20080015852 | Kruger et al. | Jan 2008 | A1 |
20080027719 | Kirshnan et al. | Jan 2008 | A1 |
20080052068 | Aguilar et al. | Feb 2008 | A1 |
20080137881 | Bongiovi | Jun 2008 | A1 |
20080147518 | Haider et al. | Jun 2008 | A1 |
20080208599 | Rosec et al. | Aug 2008 | A1 |
20080275580 | Andersen | Nov 2008 | A1 |
20090024397 | Ryu et al. | Jan 2009 | A1 |
20090204397 | Den Drinker | Aug 2009 | A1 |
20090226016 | Fitz et al. | Sep 2009 | A1 |
20100017200 | Oshikiri et al. | Jan 2010 | A1 |
20100042407 | Crockett | Feb 2010 | A1 |
20100063812 | Gao | Mar 2010 | A1 |
20100070270 | Gao | Mar 2010 | A1 |
20100076754 | Kovesi et al. | Mar 2010 | A1 |
20100138218 | Geiger | Jun 2010 | A1 |
20100198586 | Edler et al. | Aug 2010 | A1 |
20100217607 | Neuendorf et al. | Aug 2010 | A1 |
20110153333 | Bessette | Jun 2011 | A1 |
20110161088 | Bayer et al. | Jun 2011 | A1 |
20110178795 | Bayer et al. | Jul 2011 | A1 |
20110218797 | Mittal et al. | Sep 2011 | A1 |
20110218799 | Mittal et al. | Sep 2011 | A1 |
20110311058 | Oh et al. | Dec 2011 | A1 |
20120022881 | Geiger et al. | Jan 2012 | A1 |
20120226505 | Lin et al. | Sep 2012 | A1 |
Number | Date | Country |
---|---|---|
2007312667 | Apr 2008 | AU |
1274456 | Nov 2000 | CN |
1274456 | Nov 2000 | CN |
1344067 | Apr 2002 | CN |
1381956 | Nov 2002 | CN |
1437747 | Aug 2003 | CN |
1539137 | Oct 2004 | CN |
1539138 | Oct 2004 | CN |
101351840 | Oct 2006 | CN |
101110214 | Jan 2008 | CN |
101110214 | Jan 2008 | CN |
101366077 | Feb 2009 | CN |
101371295 | Feb 2009 | CN |
101379551 | Mar 2009 | CN |
101388210 | Mar 2009 | CN |
101388210 | Mar 2009 | CN |
101425292 | May 2009 | CN |
101483043 | Jul 2009 | CN |
101483043 | Jul 2009 | CN |
101488344 | Jul 2009 | CN |
101743587 | Jun 2010 | CN |
101770775 | Jul 2010 | CN |
101770775 | Jul 2010 | CN |
001087 | Oct 2000 | EA |
0673566 | Sep 1995 | EP |
0758123 | Feb 1997 | EP |
0843301 | May 1998 | EP |
1120775 | Aug 2001 | EP |
1852851 | Nov 2007 | EP |
2107556 | Oct 2009 | EP |
H08-181619 | Jul 1996 | JP |
1039898 | Feb 1998 | JP |
10-214100 | Aug 1998 | JP |
10-105193 | Oct 1998 | JP |
H11502318 | Feb 1999 | JP |
11-98090 | Apr 1999 | JP |
2000330593 | Nov 2000 | JP |
2000357000 | Dec 2000 | JP |
2002118517 | Apr 2002 | JP |
2003501925 | Jan 2003 | JP |
2003506764 | Feb 2003 | JP |
2003195881 | Jul 2003 | JP |
2004514182 | May 2004 | JP |
2006504123 | Feb 2006 | JP |
2007065636 | Mar 2007 | JP |
2007523388 | Aug 2007 | JP |
2007525707 | Sep 2007 | JP |
2007525707 | Sep 2007 | JP |
2007538282 | Dec 2007 | JP |
2007538282 | Dec 2007 | JP |
2008015281 | Jan 2008 | JP |
2008261904 | Oct 2008 | JP |
2009508146 | Feb 2009 | JP |
2009522588 | Jun 2009 | JP |
2009527773 | Jun 2009 | JP |
2010538314 | Dec 2010 | JP |
2010539528 | Dec 2010 | JP |
2011501511 | Jan 2011 | JP |
2011527444 | Oct 2011 | JP |
10-2004-0043278 | May 2004 | KR |
1020070088276 | Aug 2007 | KR |
1020100059726 | Jun 2010 | KR |
2183034 | May 2002 | RU |
2374703 | Oct 2004 | RU |
2356046 | Oct 2005 | RU |
380246 | Jan 2000 | TW |
469423 | Dec 2001 | TW |
I253057 | Apr 2006 | TW |
200703234 | Jan 2007 | TW |
200729156 | Aug 2007 | TW |
200830277 | Jul 2008 | TW |
200841743 | Oct 2008 | TW |
1316225 | Oct 2009 | TW |
200943279 | Oct 2009 | TW |
200943792 | Oct 2009 | TW |
I320172 | Feb 2010 | TW |
201009810 | Mar 2010 | TW |
201009812 | Mar 2010 | TW |
1324762 | May 2010 | TW |
201027517 | Jul 2010 | TW |
201030735 | Aug 2010 | TW |
201032218 | Sep 2010 | TW |
201040943 | Nov 2010 | TW |
201103009 | Jan 2011 | TW |
9222891 | Dec 1992 | WO |
WO-9510890 | Apr 1995 | WO |
WO-9629696 | Sep 1996 | WO |
WO-0075919 | Dec 2000 | WO |
WO-0165544 | Sep 2001 | WO |
WO-02101724 | Dec 2002 | WO |
WO-2002101722 | Dec 2002 | WO |
WO-2004027368 | Apr 2004 | WO |
2005078706 | Aug 2005 | WO |
WO-2005078706 | Aug 2005 | WO |
WO-2005081231 | Sep 2005 | WO |
2005112003 | Nov 2005 | WO |
WO-2005112003 | Nov 2005 | WO |
WO-2006126844 | Nov 2006 | WO |
2006137425 | Dec 2006 | WO |
WO-2006130226 | Dec 2006 | WO |
WO-2007051548 | May 2007 | WO |
WO-2007073604 | Jul 2007 | WO |
WO-2007083931 | Jul 2007 | WO |
WO-2007096552 | Aug 2007 | WO |
WO-2008013788 | Jan 2008 | WO |
WO-2009029032 | Mar 2009 | WO |
WO-2009077321 | Jun 2009 | WO |
2009121499 | Oct 2009 | WO |
2010006717 | Jan 2010 | WO |
WO-2010003491 | Jan 2010 | WO |
WO-2010003532 | Jan 2010 | WO |
WO-2010003563 | Jan 2010 | WO |
WO-2010003663 | Jan 2010 | WO |
WO-2010040522 | Apr 2010 | WO |
WO-2010059374 | May 2010 | WO |
WO-2010093224 | Aug 2010 | WO |
WO-2011006369 | Jan 2011 | WO |
WO-2011048094 | Apr 2011 | WO |
WO-2011147950 | Dec 2011 | WO |
WO-2012110480 | Aug 2012 | WO |
Entry |
---|
U.S. Appl. No. 13/966,048 Non-Final Office Action dated Jun. 10, 2014, 9 pages. |
Bessette et al.; “Universal Speech/Audio Coding Using Hybrid ACELP/TCX Techniques,” IEEE International Conference on Acoustics, Speech, and Signal Processing, Philadelphia, Pennsylvania, Mar. 18-23, 2005; 3:301-304. |
Office Action and Search Report in co-pending Taiwan Patent Application No. 101104674 dated Apr. 3, 2014, 8 pages. |
Office Action and Search Report in co-pending Taiwan Patent Application No. 101104678 dated Apr. 3, 2014, 8 pages. |
3GPP; “3rd Generation Partnership Project; Technical Specification Group Service and System Aspects; Audio codec processing functions; Extended AMR Wideband codec; Transcoding functions (Release 6),” 3GPP TS 26.290, Sep. 2004; vol. 2.0.0. |
Ashley et al.; “Wideband Coding of Speech Using a Scalable Pulse Codebook,” Proc. IEEE Workshop on Speech Coding, Sep. 17, 2000; pp. 148-150. |
ETSI; “Digital cellular telecommunications system (Phase 2+); Universal Mobile Telecommunications System (UMTS); LTE; Speech codec speech processing functions; Adaptive Multi-Rate—Wideband (AMR-WB) speech codec; Transcoding functions (3GPP TS 26.190 version 9.0.0 Release 9,” ETSI TS 126 190 V9.0.0, Jan. 2010. |
Ferreira, Anibal J.S.; “Combined Spectral Envelope Normalization and Subtraction of Sinusoidal Components in the ODFT and MDCT Frequency Domains,” IEEE Workshop on Applications of Signal Processing to Audio Acoustics, 2010; pp. 51-54. |
Fischer et al.; “Enumeration Encoding and Decoding Algorithms for Pyramid Cubic Lattice and Trellis Codes,” IEEE Transaction on Information Theory, Nov. 1995; 41(6):2056-2061. |
Hermansky, Hynek; “Perceptual linear predictive (PLP) analysis of speech,” J. Acoust. Soc. Amer., Apr. 1990; 87(4):1738-1751. |
Hofbauer, Konrad; “Estimating Frequency and Amplitude of Sinusoid in Harmonic Signals—A Survey and the Use of Shifted Fourier Transforms”; Graz University of Technology, Graz University of Music and Dramatic Arts; Apr. 2004. |
IEEE Signal Processing Letters Table of Contents, 2008; 15:967-975. |
Joint Technical Committee ISO/IEC JTC 1; “Information technology—MPEG audio technologies—Part 3: Unified speech and audio coding,” ISO/IEC DIS 23003-3, Jan. 31, 2011. |
Lanciani et al.; “Subband-Domain Filtering of MPEG Audio Signals,” Proc. IEEE ICASSP, Phoenix, Arizona, Mar. 1999; pp. 917-920. |
Lauber et al.; “Error Concealment for Compressed Digital Audio,” Audio Engineering Society 111th Convention Paper 5460, Sep. 21-24, 2001, New York City, New York. |
Lee et al.; “A voice activity detection algorithm for communication systems with dynamically varying background acoustic noise,” Proc. Vehicular Technology Conference, May 1998; vol. 2; pp. 1214-1218. |
Motlicek et al.; “Audio Coding Based on Long Temporal Contexts,” URL:http://www.idiap.ch/publications/motlicek-idiap-rr-06-30.bib.abs.html; IDIAP-RR, Apr. 2006. |
Neuendorf et al.; “Completion of Core Experiment on Unification of USAC Windowing and Frame Transitions,” ISO/IEC JTC1/SC29/WG11, MPEG2010/M17167, Jan. 2010, Kyoto, Japan. |
Neuendorf, Max (editor); “WD7 of USAC,” ISO/IEC JTC1/SC29/WG11, MPEG2010/N11299, Apr. 2010, Dresden, Germany. |
Patwardhan et al.; “Effect of voice quality on frequency-warped modeling of vowel spectra,” Speech Communication, 2006; 48(8):1009-1023. |
Ryan et al.; “Reflected Simplex Codebooks for Limited Feedback MIMO Beamforming,” Proc. IEEE ICC, 2009. |
Terriberry et al.; “A Multiply-Free Enumeration of Combinations With Replacement and Sign,” IEEE Signal Processing Letters, 2008; vol. 15. |
Virette et al.; “Enhanced Pulse Indexing CE for ACELP in USAC,” International Organization for Standardization, ISO/IEC JTC1/SC29/WG11, Coding of Moving Pictures and Audio, Jan. 2011; MPEG2010/M19305, Daegu, Korea. |
Wang et al.; “Frequency domain adaptive postfiltering for enhancement of noisy speech,” Speech Communication, Mar. 1993; 12(1):41-56. |
Waterschoot et al.; “Comparison of Linear Prediction Models for Audio Signals,” EURASIP Journal on Audio, Speech, and Music Processing, Dec. 2008; Article ID 706935, 24 pages. |
3GPP, “Audio codec processing functions; Extended Adaptive Multi-Rate—Wideband (AMR-WB+) codec; Transcoding functions,” 2009, 3GPP TS 26.290. |
Bessette et al.; “A wideband speech and audio codec at 16/24/32 kbit/s using hybrid ACELP/TCX techniques,” Speech Coding Proceedings, 1999 IEEE Workshop in Porvoo, Finland, Jun. 20-23, 1999, and Piscataway, NJ, Jun. 20, 1999. |
Bessette et al.; “The Adaptive Multirate Wideband Speech Codec (AMR-WB),” IEEE Transactions on Speech and Audio Processing, Nov. 1, 2002; 10(8). |
Makinen et al.; “AMR-WB+: a New Audio Coding Standard for 3rd Generation Mobile Audio Services,” 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing, Mar. 2005; 2:1109-1112. |
Martin, Rainer; “Noise Power Spectral Density Estimation Based on Optimal Smoothing and Minimum Statistics,” IEEE Transactions on Speech and Audio Processing, Jul. 2001; 9(5):504-512. |
Neuendorf et al.; “A Novel Scheme for Low Bitrate Unified Speech and Audio Coding—MPEG RMO,” AES Convention 126, May 2009, New York City, New York. |
Neuendorf et al.; “Unified speech and audio coding scheme for high quality at low bitrates,” Acoustics, Speech and Signal Processing, 2009. IEEE International Conference on ICASSP, Piscataway, NJ, Apr. 19, 2009; pp. 1-4. |
Sjoberg et al.; “RTP Payload Format for the Extended Adaptive Multi-Rate Wideband (AMR-WB+) Audio Codec; rfc4352.txt,” Jan. 1, 2006. |
USAC codec (Unified Speech and Audio Codec), ISO/IEC CD 23003-3 dated Sep. 24, 2010. |
Zernicki et al.; “Report on CE on Improved Tonal Component Coding in eSBR,” 95. MPEG Meeting Jan. 24, 2011-Jan. 28, 2011; DAEGU (Motion Picture Expert Group or ISO/IEC JTC1/SC29/WG11); No. m19238; Jan. 20, 2011. |
International Telecommunication Union; “Annex B: A silence compression scheme for G.729 optimized for terminals conforming to Recommendation V.70,” ITU-T Recommendation G.729—Annex B; Series G: Transmission Systems and Media, Nov. 1996. |
Notification of Reason for Rejection in co-pending Japan Patent Application No. 2013-553881 dated Aug. 20, 2014, 3 pages. |
Notification of Reason for Rejection in co-pending Japan Patent Application No. 2013-553903 dated Jul. 2, 2014, 5 pages. |
Notification of Reasons for Rejection in co-pending Japan Patent Application No. 2013-553902 dated Oct. 7, 2014, 7 pages. |
Office Action and Search Report in co-pending Chinese Patent Application No. 2012800159977 dated Sep. 19, 2014, 7 pages. |
Office Action and Search Report in co-pending Chinese Patent Application No. 2012800164424 dated Sep. 28, 2014, 6 pages. |
Office Action and Search Report in co-pending Chinese Patent Application No. 2012800182653 dated Sep. 1, 2014, 7 pages. |
Office Action and Search Report in co-pending Taiwan Patent Application No. 101104682 dated May 7, 2014, 10 pages. |
Martin, Rainer; “Spectral Subtraction Based on Minimum Statistics,” Proc. EUSIPCO 94, pp. 1182-1185, 1994. |
Notification of Reasons for Refusal in co-pending Japan Patent Application No. 2013-553882 dated Aug. 13, 2014, 4 pages. |
Notification of Reasons for Refusal in co-pending Japan Patent Application No. 2013-553892 dated Aug. 28, 2014, 7 pages. |
Notification of Reasons for Refusal in co-pending Japan Patent Application No. 2013-553904 dated Sep. 24, 2014, 5 pages. |
U.S. Appl. No. 13/966,048 Final Office Action dated Nov. 4, 2014, 10 pages. |
Decision to Grant in co-pending Russian Patent Application No. 2013141935 dated Nov. 24, 2014, 7 pages. |
Notice of Allowance in co-pending U.S. Appl. No. 13/966,666 dated Dec. 22, 2014, 35 pages. |
Office Action and Search Report in co-pending Chinese Patent Application No. 201280014994.1 dated Oct. 10, 2014, 14 pages. |
Office Action and Search Report in co-pending Chinese Patent Application No. 201280015995.8 dated Nov. 2, 2014, 7 pages. |
Office Action and Search Report in co-pending Chinese Patent Application No. 201280018224.4 dated Nov. 2, 2014, 8 pages. |
Office Action and Search Report in co-pending Chinese Patent Application No. 2012800182511 dated Jan. 8, 2015, 8 pages. |
Office Action and Search Report in co-pending Chinese Patent Application No. 2012800182827 dated Oct. 20, 2014, 23 pages. |
Office Action and Search Report in co-pending Chinese Patent Application No. 2012800184818 dated Dec. 8, 2014, 9 pages. |
Terriberry, Timothy B.; “Pulse Vector Coding,” retrieved from the Internet Feb. 11, 2015; http://people.xiph.org/˜tterribe/notes/cwrs.html. |
Office Action in co-pending Korean Patent Application No. 10-2013-7024213 dated Mar. 12, 2015, 6 pages. |
Office action in co-pending U.S. Appl. No. 13/672,935, dated Apr. 16, 2015. |
Office action dated Apr. 13, 2015 in co-pending KR Patent Application No. 10-2013-7024070. |
Office action dated Apr. 13, 2015 in co-pending KR Patent Application No. 10-2013-7024347. |
Office action dated Jun. 5, 2015 in co-pending U.S. Appl. No. 13/966,635. |
“A Silence Compression Scheme for G.729 Optimized for Terminals Conforming to Recommendation V,70,” ITU-T Recommendation G.729—Annex B, International Teiecommunication Union, Nov. 1996. |
Makinen, J. et al., “AMR-WB+: a New Audio Coding Standard for 3rd Generation Mobile Audio Services,” 2005 IEEE International Conference on Acoustics, Speech, and Signal Processing; Philadelphia, PA; USA, Mar. 18, 2005. |
Sjoberg, J. et al., “RTP Payload Format for the Extended Adaptive Multi-Rate Wideband (AMR-WB+) Audio Codec,” Memo, The Internet Society, Network Working Group, Cataegory: Standards Track, Jan. 2006. |
Zernicki, T. et al., “Report on Ce on improved Tonal Component Coding in eSBR.” International Organisation for Standardisation Organisation Internationale De Normalisation ISO/IEC JTC1/SC29/WG11, Coding of Moving Pictures and Audio, Daegu, S. Korea, Jan. 2011. |
Office action dated Jun. 9, 2015 in co-pending JP Patent Appl. No. 2014-158475. |
Decision to Grant dated Mar. 31, 2015 in co-pending RU Patent Appl. No. 2013-142138. |
3GPP2, “3rd Generation Partnership Project 2, Enhanced Variable Rate Codec, Speech Service Options 3, 68, 70 and 73 for Wideband Spread Spectrum Digital Systems,” 3GPP2 C.S0014-D, Version 1, May 2009. |
Number | Date | Country | |
---|---|---|---|
20130332175 A1 | Dec 2013 | US |
Number | Date | Country | |
---|---|---|---|
61442632 | Feb 2011 | US |
Number | Date | Country | |
---|---|---|---|
Parent | PCT/EP2012/052462 | Feb 2012 | US |
Child | 13966087 | US |