Patent Application
20080071541
Other objects, features and advantages of the present invention will become more apparent from the following detailed description when read in conjunction with the accompanying drawings.
A description will now be given of an embodiment of the invention with reference to the accompanying drawings.
The non-encoded audio signal (or the original sound) will be attenuated in the amplitude of respective frequencies moderately, whereas the encoded audio signal in which some spectral components are skipped by the encoding will be attenuated in the amplitude of spectral components rapidly. According to the principle of this invention, a frequency band that should be interpolated is determined using the magnitude of a spectral movement (which is a movement in the amplitude of spectral components) in addition to the magnitude of spectral components, so that the band where the spectral components are skipped by the encoding can be determined correctly prior to performing the interpolation for the band.
In the audio signal interpolation device of
In the time-frequency transforming unit 12, the time-domain audio signal is transformed into a frequency-domain audio signal for every frame. Any of the known transforming methods, such as FFT (Fast Fourier Transform) and MDCT (Modified Discrete Cosine Transform), may be used for the time-frequency transforming by the time-frequency transforming unit 12. The frequency-domain audio signal generated (which is a frequency spectrum) is supplied to each of a spectral movement calculation unit 13, an interpolation band determining unit 15, and a spectrum interpolation unit 16, respectively.
The spectral movement calculation unit 13 determines a spectral movement by using the frequency spectrum received from the time-frequency transforming unit 12 and the frequency spectrum of the previous frame read from a spectrum storing unit 14, and supplies the spectral movement to the interpolation band determining unit 15.
The spectral movement determined by the spectral movement calculation unit 13 may be any of the amount of movement of spectral components from the previous frame to the current frame, the difference between the amount of movement of spectral components of the previous frame (or the amount of movement of spectral components from the further preceding frame to the previous frame) and the amount of movement of spectral components of the current frame (or the amount of movement of spectral components from the previous frame to the current frame), and the difference between the amount of movement from the spectral component of concern to the adjacent spectral component in the previous frame (or the difference in amplitude between the spectral component of concern and the adjacent spectral component in the previous frame) and the amount of movement from the spectral component of concern to the adjacent spectral component in the current frame (or the difference in amplitude of the spectral component of concern and the adjacent spectral component in the current frame).
After the spectral movement of the current frame is calculated, the spectral movement calculation unit 13 stores the frequency spectrum of the current frame into the spectrum storing unit 14 in order to calculate a spectral movement of the following frame. The determination of a spectral movement may be performed for every frequency band in which a plurality of adjacent spectral components are included.
The interpolation band determining unit 15 determines a frequency band to be interpolated based on the spectral movement received from the spectral movement calculation unit 13 as well as the frequency spectrum received from the time-frequency transforming unit 12. The interpolation band determining unit 15 may use any of the following methods for determining a frequency band to be interpolated, which will be given below.
Upon start of the interpolation band determining method of
The interpolation band determining unit 15 determines whether a decrease of the amplitude of the spectral components from the previous frame to the current frame (which is a spectral movement) is above a predetermined threshold Y [dB] at step S2.
When the amplitude of spectral components is below the threshold X [dBov] and the decrease of the amplitude of the spectral components from the previous frame to the current frame is above the threshold Y [dB], the frequency band concerned is determined as being a frequency band to be interpolated at step S3.
When the amplitude of spectral components is above the threshold X [dBov], or when the decrease of the amplitude of the spectral components from the previous frame to the current frame is below the threshold Y [dB], the frequency band concerned is determined as being a frequency band which does not require interpolation at step S4. For example, the thresholds X and Y in this embodiment are set to as X=−60 and Y=20.
Upon start of the interpolation band determining method of
The interpolation band determining unit 15 determines whether a difference ((Y1-Y2)[dB]) between the amount of movement of spectral components (Y1 [dB]) from the further preceding frame to the previous frame and the amount of movement of spectral components (Y2 [dB]) from the previous frame to the current frame is above a predetermined threshold a at step S12.
When the amplitude of spectral components is below the threshold X [dBov] and the difference (Y1-Y2) [dB] is above the threshold a, the frequency band concerned is determined as being a frequency band to be interpolated at step S13.
When the amplitude of spectral components is above the threshold X [dBov], or when the difference (Y1-Y2) [dB] is below the threshold a, the frequency bands concerned is determined as being a frequency band which does not require interpolation at step S14.
For example, the threshold a in this embodiment is set to 5. In addition, the difference concerning the amount of movement of spectral components from the still further preceding frame to the further preceding frame may be used instead.
Upon start of the interpolation band determining method of
The interpolation band determining unit 15 determines whether a difference ((Z1-Z2) [dB]) between a difference in amplitude between the spectral component of concern and the adjacent spectral component in the previous frame (Z1 [dB]) and a difference in amplitude between the spectral component of concern and the adjacent spectral component in the current frame (Z2 [dB]) is above a predetermined threshold β at step S22.
When the amplitude of spectral components is below the threshold X [dBov] and the difference (Z1-Z2) [dB] is above the threshold β, the frequency band concerned is determined as being a frequency band to be interpolated at step S23.
When the amplitude of spectral components is above the threshold X [dBov], or when the difference (Z1-Z2) [dB] is below the threshold β, the frequency band concerned is determined as being a frequency band which does not require interpolation at step S24. For example, the threshold β in this embodiment is set to be 5.
In the above-described embodiments of
In addition, each of the thresholds X, Y, α, and β may be changed dynamically such that a value of the threshold is generated by multiplying the average power of an input audio signal over all the bands of the frequency spectrum of the current frame by a predetermined coefficient. Alternatively, one of different threshold values may be selectively used depending on the audio coding method concerned (such as AAC or MP3). Alternatively, the audio signal interpolation device may be configured so that the user is permitted to change each value of the thresholds X, Y, α, and β arbitrarily.
Referring back to
The method of interpolation used by the spectrum interpolation unit 16 may be the same as the conventional method. Namely, in the method of interpolation by the spectrum interpolation unit 16, the frequency spectrum of the current frame which is determined as the frequency band to be interpolated is interposed using the spectral components of a corresponding band in the preceding or following frame for the band to be interpolated in the current frame. Alternatively, another interpolation method may be used in which the spectral components of a low-frequency-side band in the current frame are copied and they are interpolated.
The frequency-time transforming unit 17 performs the frequency-time transforming for the frequency spectrum after interpolation for every frame, to restore the time-domain audio signal so that the time-domain audio signal is outputted to an output terminal 18.
In this embodiment, the frequency band to be interpolated is determined using the magnitude of a spectral movement (which is a movement in the amplitude of spectral components from the previous frame) in addition to the magnitude of spectral components, and the interpolation for the determined band is performed. Thus, it is possible to prevent interpolating of a frequency band which must not be interpolated, and the degradation of the sound quality due to the interpolation for the incorrect frequency band does not arise. The interpolation for the frequency band where spectral components are skipped by encoding can be performed appropriately, to restore the audio signal in the form near the spectrum before encoding, and the sound quality can be improved.
In
In the audio signal interpolation device of
In the time-frequency transforming unit 12, the time-domain audio signal is transformed into a frequency-domain audio signal for every frame. Any of the known transforming methods, such as the FFT or the MDCT, may be used for the time-frequency transforming by the time-frequency transforming unit 12. The generated frequency-domain audio signal (which is a frequency spectrum) is supplied to each of the spectral movement calculation unit 13, the interpolation band determining unit 15, and the spectrum interpolation unit 16, respectively.
The spectral movement calculation unit 13 determines a spectral movement by using the frequency spectrum of the current frame received from the time-frequency transforming unit 12 and the frequency spectrum of the previous frame read from a spectrum storing unit 20, and supplies the spectral movement to the interpolation band determining unit 15.
The spectral movement determined by the spectral movement calculation unit 13 may be any of the amount of movement of spectral components from the previous frame to the current frame, the difference between the amount of movement of spectral components of the previous frame (or the amount of movement of spectral components from the further preceding frame to the previous frame) and the amount of movement of spectral components of the current frame (or the amount of movement of spectral components from the previous frame to the current frame), and the difference between the amount of movement from the spectral component of concern to the adjacent spectral component in the previous frame (or the difference in amplitude between the spectral component of concern and the adjacent spectral component in the previous frame) and the amount of movement from the spectral component of concern to the adjacent spectral component in the current frame (or the difference in amplitude of the spectral component of concern and the adjacent spectral component in the current frame).
The spectral movement calculation unit 13 in this embodiment does not store the frequency spectrum of the current frame into the spectrum storing unit 20 after the spectral movement of the current frame is calculated. The determination of a spectral movement may be performed for every frequency band in which a plurality of adjacent spectral components are included.
The interpolation band determining unit 15 determines a frequency band to be interpolated based on the spectral movement received from the spectral movement calculation unit 13 as well as the frequency spectrum received from the time-frequency transforming unit 12. The interpolation band determining unit 15 may use any of the interpolation band determining methods shown in
The spectrum interpolation unit 16 interpolates the spectrum components of the frequency band determined by the interpolation band determining unit 15. The method of interpolation used by the spectrum interpolation unit 16 may be the same as the conventional method. Namely, in the method of interpolation by the spectrum interpolation unit 16, the frequency spectrum of the current frame which is determined as the frequency band to be interpolated is interposed using the spectral components of a corresponding band in the preceding or following frame for the band to be interpolated in the current frame. Alternatively, another interpolation method may be used in which the spectral components of a low-frequency-side band in the current frame are copied and they are interpolated.
The spectrum interpolation unit 16 stores the frequency spectrum of the current frame after interpolation into the spectrum storing unit 20. The frequency-time transforming unit 17 performs the frequency-time transforming of the frequency spectrum after interpolation for every frame, and restores the time-domain audio signal so that the time-domain audio signal is outputted from the output terminal 18.
In this embodiment, the frequency spectrum of the current frame after interpolation is stored into the spectrum storing unit 20, and the determination of a spectral movement is performed using the frequency spectrum of the previous frame after interpolation read from the spectrum storing unit 20. Thus, the interpolation for the band where spectral components are skipped by encoding can be performed appropriately when the spectral components of the same band in a plurality of continuous frames are skipped by encoding. The accuracy of the interpolation can be made better, the frequency spectrum before encoding can be restored, and the sound quality can be improved.
In
In the audio coding technique of AAC or MP3, the time-domain audio signal (the original sound) is transformed into the frequency-domain audio signal, and some spectral components in the frequency-domain audio signal are skipped, and then encoding is performed to generate the encoded audio data.
In the audio signal interpolation device of
The spectral movement calculation unit 13 determines a spectral movement by using the frequency spectrum of the current frame received from the spectrum decoding unit 22 and the frequency spectrum of the previous frame read from the spectrum storing unit 14, and supplies the spectral movement to the interpolation band determining unit 15.
The spectral movement determined by the spectral movement calculation unit 13 may be any of the amount of movement of spectral components from the previous frame to the current frame, the difference between the amount of movement of spectral components of the previous frame (or the amount of movement of spectral components from the further preceding frame to the previous frame) and the amount of movement of spectral components of the current frame (or the amount of movement of spectral components from the previous frame to the current frame), and the difference between the amount of movement from the spectral component of concern to the adjacent spectral component in the previous frame (or the difference in amplitude between the spectral component of concern and the adjacent spectral component in the previous frame) and the amount of movement from the spectral component of concern to the adjacent spectral component in the current frame (or the difference in amplitude of the spectral component of concern and the adjacent spectral component in the current frame).
The spectral movement calculation unit 13 in this embodiment stores the frequency spectrum of the current frame into the spectrum storing unit 14 after the spectral movement of the current frame is calculated, in order to calculate a spectral movement of the following frame. The determination of a spectral movement may be performed for every frequency band in which a plurality of adjacent spectral components are included.
The interpolation band determining unit 15 determines a frequency band to be interpolated based on the spectral movement received from the spectral movement calculation unit 13 as well as the frequency spectrum received from the spectrum decoding unit 22. The interpolation band determining unit 15 may use any of the interpolation band determining methods of shown in
The spectrum interpolation unit 16 interpolates the spectrum components of the frequency band determined by the interpolation band determining unit 15. The method of interpolation used by the spectrum interpolation unit 16 may be the same as the conventional method. Namely, in the method of interpolation by the spectrum interpolation unit 16, the frequency spectrum of the current frame which is determined as the frequency band to be interpolated is interposed using the spectral components of a corresponding band in the preceding or following frame for the band to be interpolated in the current frame. Alternatively, another interpolation method may be used in which the spectral components of a low-frequency-side band in the current frame are copied and they are interpolated.
The frequency-time transforming unit 17 performs the frequency-time transforming of the frequency spectrum after interpolating for every frame, and restores the time-domain audio signal so that the time-domain audio signal is outputted from the output terminal 18.
In this embodiment, the interpolation is performed for the frequency-domain audio signal containing the encoded audio data which is generated in the frequency domain, prior to restoring of the time-domain audio signal. According to this embodiment, the device or process for performing the time-frequency transform as in the embodiment of
In
In the audio signal interpolation device of
The spectral movement calculation unit 13 determines a spectral movement by using the frequency spectrum of the current frame received from the spectrum decoding unit 22 and the frequency spectrum of the previous frame read from the spectrum storing unit 20, and supplies the spectral movement to the interpolation band determining unit 15.
The spectral movement determined by the spectral movement calculation unit 13 may be any of the amount of movement of spectral components from the previous frame to the current frame, the difference between the amount of movement of spectral components of the previous frame (or the amount of movement of spectral components from the further preceding frame to the previous frame) and the amount of movement of spectral components of the current frame (or the amount of movement of spectral components from the previous frame to the current frame), and the difference between the amount of movement from the spectral component of concern to the adjacent spectral component in the previous frame (or the difference in amplitude between the spectral component of concern and the adjacent spectral component in the previous frame) and the amount of movement from the spectral component of concern to the adjacent spectral component in the current frame (or the difference in amplitude of the spectral component of concern and the adjacent spectral component in the current frame).
The spectral movement calculation unit 13 in this embodiment does not store the frequency spectrum of the current frame into the spectrum storing unit 20 after the spectral movement of the current frame is calculated. The determination of a spectral movement may be performed for every frequency band in which a plurality of adjacent spectral components are included.
The interpolation band determining unit 15 determines a frequency band to be interpolated by using the spectral movement received from the spectral movement calculation unit 13 as well as the frequency spectrum received from the spectrum decoding unit 22. The interpolation band determining unit 15 may use any of the interpolation band determining methods shown in
The spectrum interpolation unit 16 interpolates the spectral components of the frequency band determined by the interpolation band determining unit 15. The method of interpolation used by the spectrum interpolation unit 16 may be the same as the conventional method. Namely, in the method of interpolation by the spectrum interpolation unit 16, the frequency spectrum of the current frame which is determined as the frequency band to be interpolated is interposed using the spectral components of a corresponding band in the preceding or following frame for the band to be interpolated in the current frame. Alternatively, another interpolation method may be used in which the spectral components of a low-frequency-side band in the current frame are copied and they are interpolated.
The spectrum interpolation unit 16 stores the frequency spectrum of the current frame after interpolation into the spectrum storing unit 20. The frequency-time transforming unit 17 performs the frequency-time transforming of the frequency spectrum after interpolation for every frame, and restores the time-domain audio signal so that the time-domain audio signal is outputted from the output terminal 18.
In this embodiment, the frequency spectrum of the current frame after interpolation is stored into the spectrum storing unit 20, and the determination of a spectral movement is performed by using the frequency spectrum of the previous frame after interpolation read from the spectrum storing unit 20. Thus, the interpolation for the band where spectral components are skipped by encoding can be performed appropriately when the spectral components of the same band in a plurality of continuous frames are skipped by encoding. The accuracy of the interpolation can be made better, the frequency spectrum before encoding can be restored, and the sound quality can be improved.
The spectrum storing units 14 and 20 in the above embodiments are equivalent to a spectrum storing unit in the claims. The spectral movement calculation unit 13 in the above embodiments is equivalent to a spectral movement calculation unit in the claims. The interpolation band determining unit 15 in the above embodiments is equivalent to an interpolation band determination unit in the claims. The spectrum interpolation unit 16 in the above embodiments is equivalent to a spectrum interpolation unit in the claims. The time-frequency transforming unit 12 in the above embodiments is equivalent to a transforming unit in the claims. And the spectrum decoding unit 22 in the above embodiment is equivalent to a decoding unit in the claims.
The present invention is not limited to the specifically disclosed embodiments, and variations and modifications may be made without departing from the scope of the present invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2006-254425 | Sep 2006 | JP | national |
