Patent Application
20160104499
The present invention relates to a signal processing device and a signal processing method for interpolating high frequency components of an audio signal by generating an interpolation signal and synthesizing the interpolation signal with the audio signal.
As formats for compression of audio signals, nonreversible compression formats such as MP3 (MPEG Audio Layer-3), WMA (Windows Media Audio, registered trademark), and AAC (Advanced Audio Coding) are known. In the nonreversible compression formats, high compression rates are achieved by drastically cutting high frequency components that are near or exceed the upper limit of the audible range. At the time when this type of technique was developed, it was thought that auditory sound quality degradation does not occur even when high frequency components are drastically cut. However, in recent years, a thought that drastically cutting high frequency components slightly changes sound quality and degrades auditory sound quality is becoming the mainstream. Therefore, high frequency interpolation devices that improve sound quality by performing high frequency interpolation on the nonreversibly compressed audio signals have been proposed. Specific configurations of this type of high frequency interpolation devices are disclosed for example in Japanese Patent Provisional Publication No. 2007-25480A (hereinafter, Patent Document 1) and in Re-publication of Japanese Patent Application No. 2007-534478 (hereinafter, Patent Document 2).
A high frequency interpolation device disclosed in Patent Document 1 calculates a real part and an imaginary part of a signal obtained by analyzing an audio signal (raw signal), forms an envelope component of the raw signal using the calculated real part and imaginary part, and extracts a high-harmonic component of the formed envelope component. The high frequency interpolation device disclosed in Patent Document 1 performs the high frequency interpolation on the raw signal by synthesizing the extracted high-harmonic component with the raw signal.
A high frequency interpolation device disclosed in Patent Document 2 inverses a spectrum of an audio signal, up-samples the signal of which the spectrum is inverted, and extracts an extension band component of which a lower frequency end is almost the same as a high frequency range of the baseband signal from the up-sampled signal. The high frequency interpolation device disclosed in Patent Document 2 performs the high frequency interpolation of the baseband signal by synthesizing the extracted extension band component with the baseband signal.
A frequency band of a nonreversibly compressed audio signal changes in accordance with a compression encoding format, a sampling rate, and a bit rate after compression encoding. Therefore, if the high frequency interpolation is performed by synthesizing an interpolation signal of a fixed frequency band with an audio signal as disclosed in Patent Document 1, a frequency spectrum of the audio signal after the high frequency interpolation becomes discontinuous, depending on the frequency band of the audio signal before the high frequency interpolation. Thus, performing the high frequency interpolation on audio signals using the high frequency interpolation device disclosed in Patent Document 1 may have an adverse effect of degrading auditory sound quality.
Furthermore, as a general characteristic, attenuation of a level of an audio signal is greater at higher frequencies, but there is a case where a level of an audio signal instantaneously amplifies at the high frequency side. However, in Patent Document 2, only the former general characteristic is taken into account as characteristics of audio signals to be inputted to the device. Therefore, immediately after an audio signal of which a level amplifies at the high frequency side is inputted, a frequency spectrum of the audio signal becomes discontinuous, and a high frequency region is excessively emphasized. Thus, as with the high frequency interpolation device disclosed in Patent Document 1, performing the high frequency interpolation on audio signals using the high frequency interpolation device disclosed in Patent Document 2 may have an adverse effect of degrading auditory sound quality.
The present invention is made in view of the above circumstances, and the object of the present invention is to provide a signal processing device and a signal processing method that are capable of achieving sound quality improvement by the high frequency interpolation regardless of frequency characteristics of nonreversibly compressed audio signals.
One aspect of the present invention provides a signal processing device comprising a band detecting means for detecting a frequency band which satisfies a predetermined condition from an audio signal; a reference signal generating means for generating a reference signal in accordance with a detection band by the band detecting means; a reference signal correcting means for correcting the generated reference signal on a basis of a frequency characteristic of the generated reference signal; a frequency band extending means for extending the corrected reference signal up to a frequency band higher than the detection band; an interpolation signal generating means for generating an interpolation signal by weighting each frequency component within the extended frequency band in accordance with a frequency characteristic of the audio signal; and a signal synthesizing means for synthesizing the generated interpolation signal with the audio signal.
According to the above configuration, since the reference signal is corrected with a value in accordance with a frequency characteristic of an audio signal and the interpolation signal is generated on the basis of the corrected reference signal and synthesized with the audio signal, sound quality improvement by the high frequency interpolation is achieved regardless of a frequency characteristic of an audio signal.
For example, the reference signal correcting means corrects the reference signal generated by the reference signal generating means to a flat frequency characteristic.
Also, the reference signal correcting means may be configured to perform a first regression analysis on the reference signal generated by the reference signal generating means; calculate a reference signal weighting value for each frequency of the reference signal on a basis of frequency characteristic information obtained by the first regression analysis; and correct the reference signal by multiplying the calculated reference signal weighting value for each frequency and the reference signal together.
For example, the reference signal generating means extracts a range that is within n% of the overall detection band at a high frequency side and sets the extracted components as the reference signal.
The band detecting means may be configured to calculate levels of the audio signal in a first frequency range and a second frequency range being higher than the first frequency range; set a threshold on a basis of the calculated levels in the first and second frequency ranges; and detect the frequency band from the audio signal on the basis of the set threshold.
Also, for example, the band detecting means detects, from the audio signal, a frequency band of which an upper frequency limit is a highest frequency point among at least one frequency point where the level falls below the threshold.
The interpolation signal generating means may be configured to perform a second regression analysis on at least a portion of the audio signal; calculate an interpolation signal weighting value for each frequency component within the extended frequency band on a basis of frequency characteristic information obtained by the second regression analysis; and generate the interpolation signal by multiplying the calculated interpolation signal weighting value for each frequency component and each frequency component within the extended frequency band together.
For example, the frequency characteristic information obtained by the second regression analysis includes a rate of change of the frequency components within the extended frequency band. In this case, the interpolation signal generating means increases the interpolation signal weighting value as the rate of change gets greater in a minus direction.
Also, for example, the interpolation signal generating means increases the interpolation signal weighting value as an upper frequency limit of a range for the second regression analysis gets higher.
Also, when at least one of following conditions (1) to (3) is satisfied, the signal processing device may be configured not to perform generation of the interpolation signal by the interpolation signal generating means:
(1) the detected amplitude spectrum Sa is equal to or less than a predetermined frequency range;
(2) the signal level at the second frequency range is equal to or more than a predetermined value; or
(3) a signal level difference between the first frequency range and the second frequency range is equal to or less than a predetermined value.
Another aspect of the present invention provides a signal processing method comprising a band detecting step of detecting a frequency band which satisfies a predetermined condition from an audio signal; a reference signal generating step of generating a reference signal in accordance with a detection band detected by the band detecting means;
a reference signal correcting step of correcting the generated reference signal on a basis of a frequency characteristic of the generated reference signal; a frequency band extending step of extending the corrected reference signal up to a frequency band higher than the detection band; an interpolation signal generating step of generating an interpolation signal by weighting each frequency component within the extended frequency band in accordance with a frequency characteristic of the audio signal; and a signal synthesizing step of synthesizing the generated interpolation signal with the audio signal.
According to the above configuration, since the reference signal is corrected with a value in accordance with a frequency characteristic of an audio signal and the interpolation signal is generated on the basis of the corrected reference signal and synthesized with the audio signal, sound quality improvement by the high frequency interpolation is achieved regardless of a frequency characteristic of an audio signal.
For example, in the reference signal correcting step, the reference signal generated by the reference signal generating means may be corrected to a flat frequency characteristic.
In the reference signal correcting step, a first regression analysis may be performed on the reference signal generated by the reference signal generating means; a reference signal weighting value may be calculated for each frequency of the reference signal on a basis of frequency characteristic information obtained by the first regression analysis; and the reference signal may be corrected by multiplying the calculated reference signal weighting value for each frequency of the reference signal and the reference signal together.
In the reference signal generating step, a range that is within n% of the overall detection band at a high frequency side may be extracted, and the extracted components may be set as the reference signal.
In the band detecting step, levels of the audio signal in a first frequency range and a second frequency range being higher in frequency than the first frequency range may be calculated; a threshold may be set on a basis of the calculated levels in the first and second frequency ranges; and the frequency band may be detected from the audio signal on a basis of the set threshold.
In the band detecting step, a frequency band of which an upper frequency limit is a highest frequency point among at least one frequency point where the level falls below the threshold may be detected from the audio signal.
In the interpolation signal generating step, a second regression analysis may be performed on at least a portion of the audio signal; an interpolation signal weighting value may be calculated for each frequency component within the extended frequency band on a basis of frequency characteristic information obtained by the second regression analysis; and the interpolation signal may generated by multiplying the calculated interpolation signal weighting value for each frequency component and each frequency component within the extended frequency band together.
The frequency characteristic information obtained by the second regression analysis includes a rate of change of the frequency components within the extended frequency band, and in the interpolation signal generating step, the interpolation signal weighting value may be increased as the rate of change gets greater in a minus direction.
In the interpolation signal generating step, the interpolation signal weighting value may be increased as an upper frequency limit of a range for the second regression analysis gets higher.
When at least one of following conditions (1) to (3) is satisfied, the signal processing method may be configured not to generate interpolation signal in the interpolation signal generating step:
(1) the detected amplitude spectrum Sa is equal to or less than a predetermined frequency range;
(2) the signal level at the second frequency range is equal to or more than a predetermined value; or
(3) a signal level difference between the first frequency range and the second frequency range is equal to or less than a predetermined value.
Hereinafter, a sound processing device according to an embodiment of the present invention will be described with reference to the accompanying drawings.
[Overall Configuration of Sound Processing device 1]
To the FFT unit 10, an audio signal which is generated by a sound source by decoding an encoded signal in a nonreversible compressing format is inputted from the sound source. The nonreversible compressing format is MP3, WMA, AAC or the like. The FFT unit 10 performs an overlapping process and weighting by a window function on the inputted audio signal, and then converts the weighted signal from the time domain to the frequency domain using STFT (Short-Term Fourier Transform) to obtain a real part frequency spectrum and an imaginary part frequency spectrum. The FFT unit 10 converts the frequency spectrums obtained by the frequency conversion to an amplitude spectrum and a phase spectrum. The FFT unit 10 outputs the amplitude spectrum to the high frequency interpolation processing unit 20 and the phase spectrum to the IFFT unit 30. The high frequency interpolation processing unit 20 interpolates a high frequency region of the amplitude spectrum inputted from the FFT unit 10 and outputs the interpolated amplitude spectrum to the IFFT unit 30. A band that is interpolated by the high frequency interpolation processing unit 20 is, for example, a high frequency band near or exceeding the upper limit of the audible range, drastically cut by the nonreversible compression. The IFFT unit 30 calculates real part frequency spectra and imaginary part frequency spectra on the basis of the amplitude spectrum of which the high frequency region is interpolated by the high frequency interpolation processing circuit 20 and the phase spectrum which is outputted from the FFT unit 10 and held as it is, and performs weighting using a window function. The IFFT unit 30 converts the weighted signal from the frequency domain to the time domain using STFT and overlap addition, and generates and outputs the audio signal of which the high frequency region is interpolated.
[Configuration of High Frequency Interpolation Processing Unit 20]
The band detecting unit 210 converts the amplitude spectrum S (linear scale) of the audio signal inputted from the FFT unit 10 to the decibel scale. The band detecting unit 210 calculates signal levels of the amplitude spectrum S, converted to the decibel scale, within a predetermined low/middle frequency range and a predetermined high frequency range, and sets a threshold on the basis of the calculated signal levels within the low/middle frequency range and the high frequency range. For example, as shown in
The band detecting unit 210 detects an audio signal (amplitude spectrum Sa), having a frequency band of which the upper frequency limit is a frequency point where the signal level falls below the threshold, from the amplitude spectrum S (linear scale) inputted from the FFT unit 10. If there are a plurality of frequency points where the signal level falls below the threshold as shown in
(1) The detected amplitude spectrum Sa is equal to or less than a predetermined frequency range.
(2) The signal level at the high frequency range is equal to or more than a predetermined value.
(3) A signal level difference between the low/middle frequency range and the high frequency range is equal to or less than a predetermined value.
The high frequency interpolation is not performed on amplitude spectra which are judged that the generation of the interpolation signal is not necessary.
To the reference signal extracting unit 220, the amplitude spectrum Sa detected by the band detecting unit 210 is inputted. The reference signal extracting unit 220 extracts a reference signal Sb from the amplitude spectrum Sa in accordance with the frequency band of the amplitude spectrum Sa (see
The reference signal extracting unit 220 shifts the frequency of the reference signal Sb extracted from the amplitude spectrum Sa to the low frequency side (DC side) (see
The reference signal correcting unit 230 converts the reference signal Sb (linear scale) inputted from the reference signal extracting unit 220 to the decibel scale, and detects a frequency slope of the decibel scale converted reference signal Sb using linear regression analysis. The reference signal correcting unit 230 calculates an inverse characteristic of the frequency slope (a weighting value for each frequency of the reference signal Sb) detected using the linear regression analysis. Specifically, when the weighting value for each frequency of the reference signal Sb is defined as P1(x), an FFT sample position in the frequency domain on the horizontal axis (x axis) is defined as x, a value of the frequency slope of the reference signal Sb detected using the linear regression analysis is defined as α1, and ½ of the number of FFT samples corresponding to a frequency band of the reference signal Sb is defined as β1, the reference signal correcting unit 230 calculates the inverse characteristic of the frequency slope (the weighting value P1(x) for each frequency of the reference signal Sb) using the following expression (1).
P
1(x)=−α1x+β1 [EXPRESSION 1]
As shown in
To the interpolation signal generating unit 240, the reference signal Sb′ corrected by the reference signal correcting unit 230 is inputted. The interpolation signal generating unit 240 generates an interpolation signal Sc that includes a high frequency region by extending the reference signal Sb′ up to a frequency band that is higher than that of the amplitude spectrum Sa (see
To the interpolation signal correcting unit 250, the interpolation signal Sc generated by the interpolation signal generating unit 240 is inputted. The interpolation signal correcting unit 250 converts the amplitude spectrum S (linear scale) inputted from the FFT unit 10 to the decibel scale, and detects a frequency slope of the amplitude spectrum S converted to the decibel scale using linear regression analysis. It is noted that, in place of detecting the frequency slope of the amplitude spectrum S, a frequency slope of the amplitude spectrum Sa inputted from the band detecting unit 210 may be detected. A range of the regression analysis may be arbitrarily set, but typically, the range of the regression analysis is a range corresponding to a predetermined frequency band that does not include low frequency components to smoothly join the high frequency side of the audio signal and the interpolation signal. The interpolation signal correcting unit 250 calculates a weighting value for each frequency on the basis of the detected frequency slope and the frequency band corresponding to the range of the regression analysis. Specifically, when the weighting value for the interpolation signal Sc at each frequency is defined as P2(x), the FFT sample position in the frequency domain on the horizontal axis (x axis) is defined as x, an upper frequency limit of the range of the regression analysis is defined as b, a sample length for the FFT is defined as s, a slope in a frequency band corresponding to the range of the regression analysis is defined as α2, and a predetermined correction coefficient is defined as k, the interpolation signal correcting unit 250 calculates the weighting value P2(x) for the interpolation signal Sc at each frequency using the following expression (2).
P
2(x)=−α′x+β2 [EXPRESSION 2]
where
α′=α2[1−(b/s)]/k
β2=−α′b
when x<b, P2(x)=−∞
As shown in
To the adding unit 260, the interpolation signal Sc′ is inputted from the interpolation signal correcting unit 250 as well as the amplitude spectrum S from the FFT unit 10. The amplitude spectrum S is an amplitude spectrum of an audio signal of which high frequency components are drastically cut, and the interpolation signal Sc′ is an amplitude spectrum in a frequency region higher than a frequency band of the audio signal. The adding unit 260 generates an amplitude spectrum S′ of the audio signal of which the high frequency region is interpolated by synthesizing the amplitude spectrum S and the interpolation signal Sc′ (see
In the present embodiment, the reference signal Sb is extracted in accordance with the frequency band of the amplitude spectrum Sa, and the interpolation signal Sc′ is generated from the reference signal Sb′, obtained by correcting the extracted reference signal Sb, and synthesized with the amplitude spectrum S (audio signal). Thus, a high frequency region of an audio signal is interpolated with a spectrum having a natural characteristic of continuously attenuating with respect to the audio signal, regardless of a frequency characteristic of the audio signal inputted to the FFT unit 10 (for example, even when a frequency band of an audio signal has changed in accordance with the compression encoding format or the like, or even when an audio signal of which the level amplifies at the high frequency side is inputted). Therefore, improvement in auditory sound quality is achieved by the high frequency interpolation.
The followings are exemplary operating parameters of the sound processing device 1 of the present embodiment.
(FIT unit 10/IFFT unit 30)
sample length: 8,192 samples
window function: Hanning
overlap length: 50%
(Band Detecting Unit 210)
minimum control frequency: 7 kHz
low/middle frequency range: 2 kHz˜6 kHz
high frequency range: 20 kHz˜22 kHz
high frequency range level judgement: −20 dB
signal level difference: 20 dB
threshold: 0.5
(Reference Signal Extracting Unit 220) reference band width: 2.756 kHz
(Interpolation Signal Correcting Unit 250)
lower frequency limit: 500 Hz
correction coefficient k: 0.01
“Minimum control frequency (=7 kHz)” means that the high frequency interpolation is not performed if the amplitude spectrum Sa detected by the band detecting unit 210 is less than 7 kHz. “High frequency range level judgement (=−20 dB)” means that the high frequency interpolation is not performed if the signal level at the high frequency range is equal to or more than −20 dB. “signal level difference (=20 dB)” means that the high frequency interpolation is not performed if a signal level difference between the high low/middle frequency range and the high frequency range is equal to or less than 20 dB. “Threshold (=0.5)” means that a threshold for detecting the amplitude spectrum Sa is an intermediate value between a signal level (average value) of the low/middle frequency range and a signal level (average value) of the high frequency range. “Reference band width (=2.756 kHz)” is a band width of the reference signal Sb, corresponding to the “minimum control frequency (=7 kHz).” “Lower frequency limit (=500 Hz)” indicates a lower limit of the range of the regression analysis by the interpolation signal correcting unit 250 (that is, frequencies below 500 Hz are not included in the range of the regression analysis).
Referring to
In the example shown in
In the example shown in in
The above is the description of the illustrative embodiment of the present invention. Embodiments of the present invention are not limited to the above explained embodiment, and various modifications are possible within the scope of the technical concept of the present invention. For example, appropriate combinations of the exemplary embodiment specified in the specification and/or exemplary embodiments that are obvious from the specification are also included in the embodiments of the present invention. For example, in the present embodiment, the reference signal correcting unit 230 uses linear regression analysis to correct the reference signal Sb of which the level uniformly amplifies or attenuates within a frequency band. However, the characteristic of the reference signal Sb is not limited to the linear one, and in some cases, it may be nonlinear. In case of the correction of the reference signal Sb of which the signal level repeatedly amplifies and attenuates within a frequency band, the reference signal correcting unit 230 calculates the inverse characteristic using regression analysis of increased degree, and corrects the reference signal Sb using the calculated inverse characteristic.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2013-116004 | May 2013 | JP | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2014/063789 | 5/26/2014 | WO | 00 |
