Method and apparatus for detecting pitch by using spectral auto-correlation

Abstract

A method and an apparatus for detecting a pitch in input voice signals by using a spectral auto-correlation. The pitch detection method includes: performing a Fourier transform on the input voice signals after performing a pre-processing on the input voice signals, performing an interpolation on the transformed voice signals, calculating a spectral difference from a difference between spectrums of the interpolated voice signals, calculating a spectral auto-correlation by using the calculated spectral difference, determining a voicing region based on the calculated spectral auto-correlation, and extracting a pitch by using the spectral auto-correlation corresponding to the voicing region.

Description

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority from Korean Patent Application No. 10-2006-0008161, filed on Jan. 26, 2006, in the Korean Intellectual Property Office, the disclosure of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method and an apparatus for detecting a pitch in input voice signals by using a spectral auto-correlation.

2. Description of Related Art

In the field of voice signal processing such as speech recognition, voice synthesis, and analysis, it is important to exactly extract a basic frequency, i.e. a pitch cycle. The exact extraction of the basic frequency may enhance recognition accuracy through reduced speaker-dependent speech recognition, and also easily alter or maintain naturalness and personality in voice synthesis. Additionally, voice analysis synchronized with a pitch may allow for obtaining a correct vocal track parameter from which effects of glottis are removed.

For the above reasons, a variety of ways of implementing a pitch detection in a voice signal have been proposed. Such conventional proposals may be divided into a time domain detection method, a frequency domain detection method, and a time-frequency hybrid domain detection method.

The time domain detection method, such as parallel processing, average magnitude difference function (AMDF), and auto-correlation method (ACM), is a technique to extract a pitch by decision logic after emphasizing periodicity of a waveform. Being performed mostly in a time domain, this method may require only a simple operation such as an addition, a subtraction, and a comparison logic without requiring a domain conversion. However, when a phoneme ranges over a transition region, the pitch detection may be difficult due to excessive variations of a level in a frame and fluctuations in a pitch cycle, and also may be much influenced by formant. Especially, in the case of a noise-mixed voice, a complicated decision logic for the pitch detection may increase unfavorable errors in extraction.

The frequency domain detection method is a technique to extract a basic frequency of voicing by measuring a harmonics interval in a speech spectrum. A harmonics analysis technique, a lifter technique, a comb-filtering technique, etc., have been proposed as such methods. Generally, a spectrum is obtained according to a frame unit. So, even if a transition or variation of a phoneme or a background noise appears, this method may be not much affected since it may average out. However, calculations may become complicated because a conversion to a frequency domain is required for processing. Also, if pointers of a Fast Fourier Transform (FFT) increase in number to raise the precision of the basic frequency, a calculation time required is increased while being insensitive to variation characteristics.

The time-frequency hybrid domain detection method combines the merits of the aforementioned methods, that is, a short calculation time and high precision of the pitch in the time domain detection method and the ability to exactly extract pitch despite a background noise or a phoneme variation in the frequency domain detection method. This hybrid method, for example, includes a cepstrum technique and a spectrum comparison technique, may invite errors while performed between time and frequency domains, thus unfavorably influencing pitch extraction. Also, a double use of the time and frequency domains may create a complicated calculation process.

BRIEF SUMMARY

An aspect of the present invention provides a method for detecting a pitch in input voice signals by using a spectral difference and its spectral auto-correlation like time domain signals. Another aspect of the present invention provides a method for detecting a pitch in input voice signals by using normalized local center of gravity and its spectral auto-correlation like time domain signals. Still another aspect of the present invention provides an apparatus that executes the above methods.

One aspect of the present invention provides a pitch detection apparatus, which includes: a pre-processing unit performing a predetermined pre-processing on input voice signals, a Fourier transform unit performing a Fourier transform on the pre-processed voice signals, an interpolation unit performing an interpolation on the transformed voice signals, a spectral difference calculation unit calculating a spectral difference from a difference between spectrums of the interpolated voice signals, a spectral auto-correlation calculation unit calculating a spectral auto-correlation by using the calculated spectral difference, a voicing region decision unit determining a voicing region based on the calculated spectral auto-correlation, and a pitch extraction unit extracting a pitch by using the spectral auto-correlation corresponding to the voicing region.

Another aspect of the invention provides a pitch detection apparatus, which includes: a pre-processing unit performing a predetermined pre-processing on input voice signals, a Fourier transform unit performing a Fourier transform on the pre-processed voice signals, an interpolation unit performing an interpolation on the transformed voice signals, a normalized local center of gravity (NLCG) calculation unit calculating an NLCG on a spectrum of the interpolated voice signals, a spectral auto-correlation calculation unit calculating a spectral auto-correlation by using the calculated NLCG, a voicing region decision unit determining a voicing region based on the calculated spectral auto-correlation, and a pitch extraction unit extracting a pitch by using the spectral auto-correlation corresponding to the voicing region.

Another aspect of the invention provides a pitch detection method, which includes: performing a Fourier transform on input voice signals after performing a predetermined pre-processing on the input voice signals, performing an interpolation on the transformed voice signals, calculating a spectral difference from a difference between spectrums of the interpolated voice signals, calculating a spectral auto-correlation by using the calculated spectral difference, determining a voicing region based on the calculated spectral auto-correlation, and extracting a pitch by using the spectral auto-correlation corresponding to the voicing region.

Still another aspect of the invention provides a pitch detection method, which includes: performing a Fourier transform on input voice signals after performing a pre-processing on the input voice signals, performing an interpolation on the transformed voice signals, calculating a normalized local center of gravity (NLCG) on a spectrum of the interpolated voice signals, calculating spectral auto-correlation by using the calculated NLCG, determining a voicing region based on the calculated spectral auto-correlation, and extracting a pitch by using the spectral auto-correlation corresponding to the voicing region.

According to an aspect of the present invention, there is provided a method of detecting a pitch in input voice signals, the method including: Fourier transforming the input voice signals after the input voice signals are pre-processed; interpolating the transformed voice signals; calculating a spectral difference from a difference between spectrums of the interpolated voice signals; calculating a spectral auto-correlation using the calculated spectral difference; determining a voicing region based on the calculated spectral auto-correlation; and extracting a pitch using a spectral auto-correlation corresponding to the voicing region.

According to other aspects of the present invention, there are provided computer-readable storage media encoded with processing instructions for causing a processor to execute the aforementioned methods.

Additional and/or other aspects and advantages of the present invention will be set forth in part in the description which follows and, in part, will be obvious from the description, or may be learned by practice of the invention.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and/or other aspects and advantages of the present invention will become apparent and more readily appreciated from the following detailed description, taken in conjunction with the accompanying drawings of which:

FIG. 1 is a block diagram illustrating a pitch detection apparatus according to an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a pitch detection method utilizing the apparatus of FIG. 1.

FIG. 3, parts (a)-(c), is a view illustrating resultant waveforms obtained from experiments utilizing the method of FIG. 2.

FIG. 4 is a block diagram illustrating a pitch detection apparatus according to another embodiment of the present invention.

FIG. 5 is a flowchart illustrating a pitch detection method utilizing the apparatus of FIG. 4.

FIG. 6, parts (a)-(c), is a view illustrating resultant waveforms obtained from experiments utilizing the method of FIG. 5.

FIGS. 7A-7D are views for comparing waveform between spectral difference and normalized local center of gravity.

DETAILED DESCRIPTION OF EMBODIMENTS

Reference will now be made in detail to exemplary embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the like elements throughout. The exemplary embodiments are described below in order to explain the present invention by referring to the figures.

FIG. 1 is a block diagram illustrating a pitch detection apparatus 100 according to an embodiment of the present invention.

As shown in FIG. 1, the pitch detection apparatus 100 includes a pre-processing unit 101, a Fourier transform unit 102, an interpolation unit 103, a spectral difference calculation unit 104, a spectral auto-correlation calculation unit 105, a voicing region decision unit 106, and a pitch extraction unit 107.

The pitch detection apparatus 100 detects a pitch in input voice signals by using a spectral difference and its spectral auto-correlation. A waveform of the spectral difference appears in a shape similar to the waveform in a time domain. A graph of a spectral auto-correlation calculated by using a spectral difference represents peaks corresponding to pitch frequencies.

FIG. 2 is a flowchart illustrating a pitch detection method utilizing, by way of a non-limiting example, the apparatus shown in FIG. 1.

Referring to FIGS. 1 and 2, in a first operation S201, the pre-processing unit 101 performs a predetermined pre-processing on input voice signals. In a next operation S202, the Fourier transform unit 102 performs a Fourier transform on the pre-processed voice signals as shown in Equation 1.

$\begin{array}{cc}\begin{array}{c}A\left(f\right)=A\left(e^{j2\pi k/N}\right)\\ =\sum _{n=0}^{N-1}s\left(n\right)e^{j2\pi k/N}\end{array}& \left[\mathrm{Equation}1\right]\end{array}$

In a next operation S203, the interpolation unit 103 performs an interpolation on the transformed voice signals as shown in the following Equation 2.A(f_k)A(f_i)  [Equation 2]

• • Here, k=1, 2, . . . , L_k, i=1, 2, . . . , L_i, and R=L_i/L_k

In this operation S203, the interpolation unit 103 performs a low-pass interpolation with regard to amplitudes corresponding to low-pass frequencies, e.g. 0˜1.5 kHz, and may also re-sample sequence to correspond to R (L_i/L_k) times of an initial sample rate as shown in equation 2. Such interpolation may reduce a drop in a resolution due to narrower sample intervals, and also improve a frequency resolution.

In a next operation S204, the spectral difference calculation unit 104 calculates a spectral difference from a difference between frequencies in a spectrum of transformed and interpolated voice signals. This is shown in Equation 3.dA(f_i)=A(f_i)−A(f_i−1)  [Equation 3]

In this operation S204, the spectral difference calculation unit 104 may calculate a spectral difference by a positive difference of a spectrum. The waveform of the calculated spectral difference is in a shape similar to the waveform in a time region.

In a next operation S205, the spectral auto-correlation calculation unit 105 calculates spectral auto-correlation by using the calculated spectral difference. Here, the spectral auto-correlation calculation unit 105 uses the calculated spectral difference and then calculates a spectral auto-correlation by performing a normalization as shown in Equation 4.

$\begin{array}{cc}\mathrm{sa}\left(f_{\tau }\right)=\sum _i\mathrm{dA}\left(f_i\right)·\mathrm{dA}\left(f_{i-\tau }\right)/\sum _i\mathrm{dA}\left(f_i\right)·\mathrm{dA}\left(f_i\right)& \left[\mathrm{Equation}4\right]\end{array}$

In a next operation S206, the voicing region decision unit 106 determines a voicing region by means of a frequency component of the calculated spectral auto-correlation. Here, the voicing region decision unit 106 compares a maximum of the calculated spectral auto-correlation with a predetermined value Tsa. Then, as shown in Equation 5, a region in which the maximum spectral auto-correlation is greater than the predetermined value is determined as the voicing region.voiced if max{sa(f_τ)}>T_sa unvoiced if max {sa(f_τ)}<T_sa  [Equation 5]

In a next operation S207, the pitch extraction unit 107 extracts a pitch by using the spectral auto-correlation corresponding to the voicing region as shown in Equation 6.

$\begin{array}{cc}P=\underset{\tau }{\max }\left\{\mathrm{sa}\left(f_{\tau }\right)\right\}\mathrm{if}\mathrm{voiced}& \left[\mathrm{Equation}6\right]\end{array}$

In this operation S207, the pitch extraction unit 107 may extract the pitch by performing a parabolic interpolation or a sync function interpolation to the spectral auto-correlation corresponding to the voicing region. Namely, the pitch extraction unit 107 may obtain the pitch from the position of a local peak corresponding to the maximum spectral auto-correlation among interpolated spectral auto-correlations.

FIG. 3 is a view illustrating resultant waveforms obtained from experiments utilizing the method of FIG. 2.

In FIG. 3, part (a) represents input signals. Specifically, 1 is a man's voice signal, 2 is a mixed signal of the man's voice and a white noise, and 3 is a mixed signal of the man's voice and an airplane noise. Also, 4 is a woman's voice signal, 5 is a mixed signal of the woman's voice and a white noise, and 6 is a mixed signal of the woman's voice and an airplane noise.

Furthermore, parts (b) and (c) in FIG. 3 illustrate waveforms after the respective input signals are processed by the above-described method shown in FIG. 2. Specifically, part (b) shows a step of determining the voicing region by using both the calculated spectral auto-correlation and a predetermined value T_sa. Finally, part (c) shows a result of extracting the pitch by using the spectral auto-correlation corresponding to the voicing region.

FIG. 4 is a block diagram illustrating a pitch detection apparatus according to another embodiment of the present invention.

As shown in FIG. 4, the pitch detection apparatus 400 of the present embodiment includes a pre-processing unit 401, a Fourier transform unit 402, an interpolation unit 403, a normalized local center of gravity calculation unit 404, a spectral auto-correlation calculation unit 405, a voicing region decision unit 406, and a pitch extraction unit 407.

The pitch detection apparatus 400 detects a pitch in input voice signals by using a normalized local center of gravity and its spectral auto-correlation. The waveform of the normalized local center of gravity appears in a shape similar to the waveform in a time domain. Moreover, a periodic structure of harmonics may be effectively preserved in comparison with the previous embodiment. A graph of spectral auto-correlation calculated by using the normalized local center of gravity represents peaks corresponding to pitch frequencies.

FIG. 5 is a flowchart illustrating a pitch detection method utilizing, by way of a non-limiting example, the apparatus shown in FIG. 4.

Referring to FIGS. 4 and 5, in a first operation S501, the pre-processing unit 401 performs a predetermined pre-processing on input voice signals. In a next operation S502, the Fourier transform unit 402 performs a Fourier transform on the pre-processed voice signals as set forth in the above Equation 1.

In a next operation S503, the interpolation unit 403 performs interpolation on the transformed voice signals as set forth in the above Equation 2. Here, the interpolation unit 403 performs a low-pass interpolation with regard to amplitudes corresponding to low-pass frequencies, e.g. 0-1.5 kHz, and may also re-sample a sequence to correspond to R (L_i/L_k) times of an initial sample rate as shown in the above Equation 2. Such interpolation may reduce a drop in resolution due to narrower sample intervals, and also improve a frequency resolution.

In a next operation S504, the normalized local center of gravity calculation unit 404 calculates a normalized local center of gravity (NLCG) on spectrum of transformed and interpolated voice signals. This is shown in the following Equation 7.

$\begin{array}{cc}\mathrm{cA}\left(f_i\right)=\frac{1}{U}\frac{\sum _{j=1}^{j=U}\mathrm{iA}\left(f_{i-U/2+j}\right)}{\sum _{j=1}^{j=U}A\left(f_{i-U/2+j}\right)}-0.5& \left[\mathrm{Equation}7\right]\end{array}$

Here, a symbol U represents a local region. The waveform of the calculated NLCG is in a shape similar to the waveform in time region. Moreover, a periodic structure of harmonics may be effectively preserved in the present embodiment, as compared with the previous embodiment.

In a next operation S505, the spectral auto-correlation calculation unit 405 calculates spectral auto-correlation by using the calculated NLCG. This is shown in the following Equation 8.

$\begin{array}{cc}\mathrm{sa}\left(f_{\tau }\right)=\sum _i\mathrm{cA}\left(f_i\right)·\mathrm{cA}\left(f_{i-\tau }\right)& \left[\mathrm{Equation}8\right]\end{array}$

Here, contrary to the previous embodiment, the spectral auto-correlation calculation unit 405 does not separately perform normalization. The reason is that normalization has been already performed in the above-discussed NLCG calculation step.

In a next operation S506, the voicing region decision unit 406 determines a voicing region based on the calculated spectral auto-correlation. Here, the voicing region decision unit 406 compares a maximum spectral auto-correlation with a predetermined value as shown in the above Equation 5. Then a region in which the maximum spectral auto-correlation is greater than the predetermined value is determined as the voicing region.

In a next operation S507, the pitch extraction unit 407 extracts a pitch by using the spectral auto-correlation corresponding to the voicing region as shown in the above Equation 6. Here, the pitch extraction unit 407 may extract the pitch by performing a parabolic interpolation or a sync function interpolation to the spectral auto-correlation corresponding to the voicing region. That is, the pitch extraction unit 407 may obtain the pitch from a position of a local peak corresponding to the maximum spectral auto-correlation among interpolated spectral auto-correlations.

FIG. 6 is a view illustrating resultant waveforms obtained by experiment utilizing the method of FIG. 5.

In FIG. 6, part (a) represents input signals. Specifically, 1 is a man's voice signal, 2 is a mixed signal of the man's voice and a white noise, and 3 is a mixed signal of the man's voice and an airplane noise. Also, 4 is a woman's voice signal, 5 is a mixed signal of the woman's voice and a white noise, and 6 is a mixed signal of the woman's voice and an airplane noise.

Furthermore, parts (b) and (c) in FIG. 6 illustrate waveforms after the respective input signals are processed by the above-described method shown in FIG. 5. Specifically, part (b) shows a step of determining the voicing region by using both the calculated spectral auto-correlation and a predetermined value T_sa. Finally, part (c) shows a result of extracting the pitch by using the spectral auto-correlation corresponding to the voicing region.

FIGS. 7A-D are views for comparing waveforms between spectral difference and normalized local center of gravity.

FIG. 7A shows a waveform of spectrum (up to 1.5 kHz) obtained from a single frame of man's voice with noise. FIG. 7B further shows an interpolated waveform, a waveform calculated by a spectral difference, and a waveform calculated by an NLCG.

As marked with circle on the waveforms in FIGS. 7C and 7D, the waveform of the NLCG emphasizes a harmonic component more than that of the spectral difference. Therefore, a periodic structure of harmonics can be effectively preserved.

The pitch detection method according to the above-described embodiments of the present invention includes a computer-readable medium including a program instruction for executing various operations realized by a computer. The computer-readable medium may include a program instruction, a data file, and a data structure, separately or cooperatively. The program instructions and the media may be those specially designed and constructed for the purposes of the present invention, or they may be of the kind well-known and available to those skilled in the art of computer software arts. Examples of the computer-readable media include magnetic media (e.g., hard disks, floppy disks, and magnetic tapes), optical media (e.g., CD-ROMs or DVD), magneto-optical media (e.g., optical disks), and hardware devices (e.g., ROMs, RAMs, or flash memories, etc.) that are specially configured to store and perform program instructions. Examples of the program instructions include both machine code, such as produced by a compiler, and files containing high-level language codes that may be executed by the computer using an interpreter.

According to the above-described embodiments of the present invention, provided are a method for detecting a pitch in input voice signals by using a spectral difference and its spectral auto-correlation like time domain signals, a method for detecting a pitch in input voice signals by using normalized local center of gravity and its auto-spectral correlation like time domain signals, and an apparatus executing such methods.

Additionally, according to the above-described embodiments of the present invention, provided are a new pitch detection method and apparatus that allow a minimized deviation between periods, have less influence on a noise environment, and thereby improve the exactness of a pitch detection.

Although a few exemplary embodiments of the present invention have been shown and described, the present invention is not limited to the described exemplary embodiments. Instead, it would be appreciated by those skilled in the art that changes may be made to these exemplary embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A method of detecting a pitch in input voice signals implemented by a processor, the method comprising: performing, using the processor, a Fourier transform on the input voice signals after performing a pre-processing on the input voice signals;performing an interpolation on the transformed voice signals;calculating a normalized local center of gravity (NLCG) on a portion of a spectrum of the interpolated voice signals in a local region, instead of the entire spectrum;calculating a spectral auto-correlation using the calculated NLCG;determining a voicing region based on the calculated spectral auto-correlation; andextracting a pitch using a spectral auto-correlation corresponding to the voicing region,wherein the calculating of the NLCG includes calculating the NLCG on a portion of the spectrum in the local region, instead of the entire spectrum, so that a center of gravity on a spectrum in the local region among spectrum of the interpolated voice signals is included within a predetermined range, andwherein the calculating of the spectral auto-correlation comprises automatically performing a normalization when the NLCG is included within a predetermined range,wherein the NLCG is calculated by the equation

2. The method of claim 1, wherein the performing an interpolation includes: performing a low-pass interpolation with regard to amplitudes corresponding to low-pass frequencies of the transformed voice signals; andre-sampling a sequence to correspond to R times of an initial sample rate.

3. The method of claim 1, wherein the determining a voicing region includes: comparing a maximum of the calculated spectral auto-correlation with a predetermined value; anddetermining, as the voicing region, a region in which the maximum calculated spectral auto-correlation is greater than the critical value.

4. The method of claim 1, wherein the extracting a pitch includes extracting the pitch by performing a parabolic interpolation or a sync function interpolation on the spectral auto-correlation corresponding to the voicing region.

5. The method of claim 4, wherein the pitch is extracted from a position of a local peak corresponding to a maximum spectral auto-correlation among interpolated spectral auto-correlations.

6. An apparatus for detecting a pitch in input voice signals, the apparatus comprising: a processor comprising a pre-processing unit performing a predetermined pre-processing on the input voice signals;a Fourier transform unit performing a Fourier transform on the pre-processed voice signals;an interpolation unit performing an interpolation on the transformed voice signals;a normalized local center of gravity (NLCG) calculation unit calculating an NLCG on a portion of a spectrum of the interpolated voice signals in a local region, instead of the entire spectrum;a spectral auto-correlation calculation unit calculating a spectral auto-correlation using the calculated NLCG;a voicing region decision unit determining a voicing region based on the calculated spectral auto-correlation; anda pitch extraction unit extracting a pitch using a spectral auto-correlation corresponding to the voicing region,wherein the NLCG calculation unit calculates the NLCG on a portion of the spectrum in the local region, instead of the entire spectrum, so that a center of gravity on a spectrum in the local region among spectrum of the interpolated voice signals is included within a predetermined range, andwherein the spectral auto-correlation calculation unit automatically performs a normalization when the NLCG is included within a predetermined range,wherein the NLCG is calculated by the equation

7. A method of detecting a pitch in input voice signals implemented by a processor, the method comprising: performing, using the processor, a Fourier transform on the input voice signals after performing a pre-processing on the input voice signals;performing an interpolation on the transformed voice signals;calculating a normalized local center of gravity (NLCG) on a portion of a spectrum of the interpolated voice signals in a local region, instead of the entire spectrum;calculating a spectral auto-correlation using the calculated NLCG;determining a voicing region based on the calculated spectral auto-correlation; andextracting a pitch using a spectral auto-correlation corresponding to the voicing region,wherein the NLCG is calculated by the equation

8. An apparatus for detecting a pitch in input voice signals, the apparatus comprising: a processor comprising a pre-processing unit performing a predetermined pre-processing on the input voice signals;a Fourier transform unit performing a Fourier transform on the pre-processed voice signals;an interpolation unit performing an interpolation on the transformed voice signals;a normalized local center of gravity (NLCG) calculation unit calculating an NLCG on a portion of a spectrum of the interpolated voice signals in a local region, instead of the entire spectrum;a spectral auto-correlation calculation unit calculating a spectral auto-correlation using the calculated NLCG;a voicing region decision unit determining a voicing region based on the calculated spectral auto-correlation; anda pitch extraction unit extracting a pitch using a spectral auto-correlation corresponding to the voicing region,wherein the NLCG calculation unit calculates the NLCG by the equation