SOUND PROCESSING METHOD AND DEVICE USING DJ TRANSFORM

Abstract

A sound processing method according to an embodiment of the present disclosure comprises the steps of: sampling, by a computer, natural frequencies of a plurality of springs, the plurality of springs having natural frequencies different from each other and oscillate according to an input sound; determining filtered pure-tone amplitudes of the plurality of springs by the computer-which includes calculating transient-state-pure-tone amplitudes of the plurality of modeled springs, calculating expected steady-state amplitudes of the plurality of modeled springs, calculating predicted pure-tone amplitudes based on the expected steady-state amplitudes, and calculating filtered pure-tone amplitudes by multiplying the transient-state-pure-tone amplitudes with the predicted pure-tone amplitudes-; extracting, by the computer, a natural frequency of at least one spring of the plurality of springs which corresponds to a local maximum value among the filtered pure-tone amplitudes; and using the natural frequency for sound recognition or sound synthesis.

Description

TECHNICAL FIELD

The present disclosure generally relates to sound processing method and device which can increase the temporal resolution as well as the frequency resolution simultaneously by extracting a frequency of an input sound using DJ transform. The frequency extracted according to the present invention can be used in various fields such as sound recognition and sound synthesis.

BACKGROUND

The Short-time Fourier Transform (STFT) is used in various fields dealing with sound, such as speech recognition, speaker recognition, etc. to extract frequencies from a given sound. However, when frequencies are extracted by the STFT, there is a limitation on increasing the temporal resolution as well as the frequency resolution due to the Fourier uncertainty principle. The Fourier uncertainty principle states that if a sound of a short duration is transformed into a frequency component, then the resolution of the frequency component is relatively low, and if a sound with a longer duration is used to obtain a more precise frequency, then the temporal resolution for the instant when the frequency component is extracted decreases.

For example, when using the STFT, assume that a window size is 25 milliseconds, and a rectangular filter is used. The frequency component extracted under these conditions has a resolution of 40 Hz. In that case, even if 420 Hz frequency exists in an input sound, only 400 Hz frequency and 440 Hz frequency appear as the extraction result, and the 420 Hz frequency does not appear. For that reason, the distinction between a pure tone composed of 420 Hz frequency only and a complex tone composed of 400 Hz and 440 Hz frequencies is not clear. Now, assume that 4 kHz frequency exists on the extracted result. The extraction result does not give any information on the time point when the 4 kHz frequency occurred within the 25 milliseconds window. For example, it is not possible to distinguish whether the 4 kHz frequency occurred in the range of 0˜10 milliseconds or in the range of 10˜20 milliseconds.

In order to get a frequency resolution of 20 Hz, the window size should be extended to 50 milliseconds. However, since the temporal resolution is inversely proportionate to the frequency resolution, the temporal resolution decreases due to the 50 milliseconds window. Also, if the window size is reduced to 12.5 milliseconds to increase the temporal resolution, the frequency resolution is lowered to 80 Hz. Due to this trade-off, the temporal resolution and the frequency resolution cannot be improved simultaneously when using the STFT.

SUMMARY

According to research findings, it is known that human hearing ability is not restricted by the Fourier uncertainty principle. The present disclosure intends to propose the sound processing method and device using the DJ transform method, a new frequency extraction method from understanding of the human hearing ability that improves the temporal resolution as well as the frequency resolution simultaneously based on the operating principle of hair cells constituting the cochlea.

A sound processing method according to an embodiment of the present disclosure comprises the steps of: sampling, by a computer, natural frequencies of a plurality of springs, the plurality of springs having natural frequencies different from each other and oscillate according to an input sound; determining filtered pure-tone amplitudes of the plurality of springs by the computer: calculating transient-state-pure-tone amplitudes of the plurality of modeled springs; calculating expected steady-state amplitudes of the plurality of modeled springs; calculating predicted pure-tone amplitudes based on the expected steady-state amplitudes; calculating filtered pure-tone amplitudes by multiplying the transient-state-pure-tone amplitudes with the predicted pure-tone amplitudes; extracting, by the computer, a natural frequency of at least one spring of the plurality of springs which corresponds to a local maximum value among the filtered pure-tone amplitudes; and using the natural frequency for sound recognition or sound synthesis.

A sound processing device according to an embodiment of the present disclosure comprises: a memory; and a processor configured to: produce displacements and velocities of a plurality of springs by modeling the plurality of springs which have natural frequencies different from each other and oscillate according to an input sound, the displacements and the velocities being recorded in the memory, calculate transient-state-pure-tone amplitudes of the plurality of modeled springs, calculate expected steady-state amplitudes of the plurality of modeled springs, calculating predicted pure-tone amplitudes on the basis of the expected steady-state amplitudes, calculate filtered pure-tone amplitudes by multiplying the transient-state-pure-tone amplitudes with the predicted pure-tone amplitudes, extract the natural frequency of at least one spring of the plurality of springs which corresponds to a local maximum value among the filtered pure-tone amplitudes, and use the natural frequency for sound recognition or sound synthesis.

A sound processing method according to an embodiment of the present disclosure comprises the steps of: sampling, by a computer, natural frequencies of a plurality of springs, the plurality of springs having natural frequencies different from each other and oscillate according to an input sound; estimating an expected steady-state amplitude of the spring of which the amplitude is the highest among the plurality of modeled springs; calculating an energy of at least one spring of the plurality of springs of which the amplitude is the highest based on the expected steady-state amplitudes; calculating an amplitude of the input pure tone based on the energy, and using the amplitude of the input pure tone for sound recognition or sound synthesis.

A sound processing device according to an embodiment of the present disclosure comprises: a memory; and a processor configured to: produce displacements and velocities of a plurality of springs by modeling the plurality of springs which have natural frequencies different from each other and oscillate according to an input sound, the displacements and the velocities being recorded in the memory, estimate an expected steady-state amplitude of a spring of which the amplitude is the highest among the plurality of modeled springs, calculate an energy of a spring of which the amplitude is the highest based on the expected steady-state amplitudes, calculate an input pure tone amplitude based on said energy, and use the input pure tone amplitude for sound recognition or sound synthesis.

Said expected steady-state amplitude can be calculated based on the amplitudes at two different time points within a duration of the input sound.

Said expected steady-state amplitude A_{i, s} can be calculated by means of the equation below:

$A_{i,s}=\frac{A_i\left(t_2\right)-A_i\left(t_1\right)e^{-\zeta \omega \left(t_2-t_1\right)}}{1-e^{-\zeta \omega \left(t_2-t_1\right)}}$

where A_i,s is the expected steady-state amplitude of i-th spring Si among the plurality of springs, wherein I is a positive integer, where t₁ and t₂ are two different time points within a duration of input sound, t₂>t₁, Ai(t₁) is an amplitude of said spring Si at t₁, Ai(t₂) is an amplitude of said spring Si at t₂, ζ is a damping ratio of said spring Si, and ω satisfies the equation ω=ω_i√{square root over (1−2ζ²)}, where ω_i is the natural frequency of said spring Si.

A difference between the two different time points can be a period of the natural frequency of the corresponding spring.

If one of the two time points is t₁, a sampling rate of the input sound is SR, and a period of the natural frequency of the corresponding spring is T, then the other t₂ of the two time points can be calculated by the equation below.

t ₂ =[t ₁ +SR×T+0.5]

The expected steady-state amplitude can be calculated by substituting amplitudes at least two points in the duration of the input sound into the following equation and using a linear regression analysis.

A(t)=A_s+i(A_c−A_s)e^{−ζω(t−tc)}

where A(t) is an amplitude of any spring among said plurality of springs at t, A_s is the expected steady-state amplitude of said spring, A_c is an amplitude of said spring at t_c, t_c is a time point before the at least two points in the duration of the input sound, ζ is a damping ratio of said spring, and ω satisfies the equation ω=w_i√{square root over (1−2ζ²)}, where ω_i is the natural frequency of the spring.

Said modeling step can comprise the steps of: measuring displacements and velocities at time points for each of the plurality of springs; calculating energy at each time point for each of the plurality of springs based on the displacements and the velocities; and calculating an amplitude at each time point for each of the plurality of springs based on said energy.

The number of the plurality of springs may be determined based on a range and a resolution of the frequency to be extracted.

The sound recognition can include at least one of: speech recognition; speaker verification; speaker identification; source separation; sound direction detection; sound-based nomenclature diagnostics; sound-based machine fault diagnostics; or Sonar for navigation undersea terrain or ranging objects.

Said method may be recorded on a non-transitory computer-readable recording medium according to an embodiment of the present disclosure.

A method for checking error among pure tone frequencies comprising: inputting, by a computer, a frequency of a plurality of springs to which an input sound is applied, the frequency maintains a first value to a certain point of time and turns into a second value at the certain point, wherein a result of frequency transform to the certain point indicates the first value, and checking that immediately after the turning point, a transient error from the first value to the second value is within 10%.

According to an embodiment of the present disclosure, a method for extracting a sound frequency and a sound amplitude, which shows improved temporal and frequency resolution simultaneously, is provided. Accordingly, sounds having similar frequencies can be further subdivided and classified, and accuracy of speech recognition can be improved by precisely extracting the order of phonemes information from the speech. In addition, a stable speech recognition can be performed in a noisy environment, and the size of data required for speech recognition learning can be reduced.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other aspects, features, and advantages of certain embodiments of the present disclosure will be more apparent from the following description taken in conjunction with the accompanying drawings, in which:

FIG. 1 is an example of a graph showing the displacement of a spring when an external force is zero;

FIG. 2 is an example of a graph showing the changes in amplitude of a spring when an external force is applied and disappears;

FIG. 3 is a flowchart showing sound processing method according to an embodiment of the present disclosure;

FIG. 4A is a graph showing the transient-state-pure-tone amplitude, and FIG. 4B showing the amplitude of the input pure tone;

FIGS. 5A to 5I show graphs for the transient-state-pure-tone amplitude, the predicted pure-tone amplitude, and the filtered pure-tone amplitude according to an embodiment of the present disclosure when 1 kHz sound with constant amplitude is input;

FIG. 6 shows a graph of the filtered pure-tone amplitude when a complex tone is input;

FIG. 7 shows a graph of the filtered pure-tone amplitude when a complex tone which is different from FIG. 6 is input;

FIG. 8 is a flowchart showing a sound processing method according to an embodiment of the present disclosure;

FIGS. 9A to 9F are drawings which show the result of the STFT, the frequencies of the input sounds, and the result of the DJ transform according to the present disclosure when pure tones are input;

FIGS. 10A to 10D are drawings which show the results of the DJ transform according to the present disclosure when the frequencies of the input pure tones are changed;

FIGS. 11A to 11D are drawings which show the results of the STFT when the frequencies of the input pure tones are changed;

FIGS. 12A to 12C are drawings which show the frequency components of the input signals, the results of DJ transform, and the results of the STFT when a flickering signal and a lasting signal are input;

FIGS. 13A to 13C are drawings which show the frequency components of an input sound, the results of the DJ transform and the STFT when 1 kHz and 2 kHz sounds are alternately input;

FIGS. 14A to 14C are drawings which show the results of the DJ transform and the STFT when a pure tone and a complex tone are input;

FIG. 15 shows a sound processing device according to an embodiment of the present disclosure.

DETAILED DESCRIPTION

The following description with reference to the accompanying drawings is provided to assist in a comprehensive understanding of various embodiments of the present disclosure.

Hair cells convert mechanical signals generated in the basement membrane into electrical signals and transfer the signals to the primary auditory)cortex. Hair cells consist of about 3,500 inner hair cells and 12,000 external hair cells, and each hair cell reacts sensitively to the sound of its own natural frequency. This characteristic of hair cells is similar to a phenomenon occurred in a spring of which amplitude increases because of resonance when the spring receives an external force with a frequency that matches the natural frequency of the spring. Using this similarity, the present disclosure models the behavior of hair cells using a plurality of springs.

The human audible frequency is known to be in the range of 20-20,000 Hz and the human voice frequency is known to be in the range of 80-8,000 Hz. The frequency range covered in the field such as speech recognition is within 8 kHz. Considering the same, when used for a voice processing, the natural frequencies of the springs from 50 Hz to 8 kHz are classified by 1 Hz intervals, and 7,951 different types of springs can be used based on those natural frequencies. This means that the frequency resolution is 1 Hz unit. However, this is only an example, and widening the frequency range or increasing the resolution by using more springs is possible.

The behavior of a hair cell modeled by a spring can be represented as a differential equation of motion for driven harmonic oscillations. A sound corresponds to an external force made up of a combination of various sine waves which are applied to a spring. Each spring has its own natural frequency and draws its own motion trajectory by a series of sound samples. The motion trajectory of each spring can be obtained by calculating the solution of the differential equation of motion for driven harmonic oscillations using numerical analysis techniques such as the Runge-Kuta method.

Assume that ω_i is the natural frequency of a spring S_i (1≤i≤N). The spring S_i is used to model the response of a hair cell that are most sensitive to the sound of ω_i frequency among the hair cells constituting the human hearing system.

When the sound F₀ cos(ωt) is input, the reaction x_i(t) of the spring S_i to the sound can be represented by the equation of motion of the following equation (1):

$\begin{array}{cc}\frac{d^2{x}_i}{{\mathrm{dt}}^2}+2\zeta {\omega }_i\frac{{\mathrm{dx}}_i}{\mathrm{dt}}+{\omega }_i^2{x}_i=\frac{F_0\cos \left(\omega t\right)}{m}& \left(1\right)\end{array}$

where x_i is the length of the spring which deviates from the balance point (displacement), and m is the mass of the object suspended in the spring. ζ is a damping ratio and when a friction coefficient is b_i,

$\zeta =\frac{b_i}{2\sqrt{{\mathrm{mk}}_i}}.$

k_i is a spring constant. ω_i is the natural frequency of the spring when both ζ and F_i are zero, and ω_i=√{square root over (k_i/m)}.

Equation (1) is a differential equation with a general solution. When ζ<1, the solution is the same as the equation (2) below.

$\begin{array}{cc}{x}_i\left(t\right)=A_i{e}^{-\zeta {\omega }_{i}t}\cos \left(\sqrt{1-{\zeta }^2}{\omega }_{i}t+{\beta }_i\right)+\frac{F_0}{{\mathrm{mZ}}_i}\cos \left(\omega t+{\phi }_i\right)& \left(2\right)\end{array}$

where A_i and β_i are determined by the initial conditions of the spring, and Z_i and φ_i are as below:

$\begin{array}{cc}{Z}_i=\sqrt{{\left(2{\omega }_i\omega \zeta \right)}^2+{\left({\omega }_i^2-{\omega }^2\right)}^2}& \left(3\right)\end{array}$ $\begin{array}{cc}{\phi }_i=\mathrm{arc}\tan \left(\frac{2\omega {\omega }_i\zeta }{{\omega }^2-{\omega }_i^2}\right)+n\pi & \left(4\right)\end{array}$

The integer n is specified so that φ_i is between −180° and 0°. If F₀=0, the spring is subjected to periodically damping oscillation as shown in FIG. 1. If F₀>0 and the spring reaches a steady state after a certain period of time, the first term in the equation (2) disappears and the second term remains only so that the trajectory X_i,s(t) of the spring in a steady state follows the equation (5).

$\begin{array}{cc}{x}_{i,s}\left(t\right)=\frac{F_0}{{\mathrm{mZ}}_i}\cos \left(\omega t+{\phi }_i\right)& \left(5\right)\end{array}$

Consider a situation in which a sound having a frequency identical with the natural frequency ω_i of a spring S_i in a stop state is applied to the spring as an external force. The behavior of the spring in the process of reaching a steady state is described by the equation (6) below.

x _i(t)=(1−e^−ζωit)x_i,s(t)  (6)

Therefore, the amplitude A_i(t) of the spring gradually increases along the trajectory of

$A_i\left(t\right)=\frac{F_0}{{\mathrm{mZ}}_i}\left(1-e^{-\zeta {\omega }_{i}t}\right)$

and finally becomes F₀/mZ_i.

As the external force disappears at the point to, the amplitude of the spring gradually decreases to zero. This corresponds F₀=0 in the equation (2), and the amplitude change in this process follows the equation below.

A _i(t)=A_i(t₀)e^{−ζω(t−t0)}  (7)

FIG. 2 is an example of a graph showing the changes in amplitude of a spring when an external force is applied and disappears.

According to the embodiments of present disclosure, two methods for extracting the frequency and amplitude of the input sound are proposed based on the behavior of the spring modeled as hair cells.

Method I for Extracting the Frequency and Amplitude of the Input Sound

1. In a Steady State

(1) Extraction of Frequency

Based on the characteristic that a resonating spring oscillates with a greater amplitude than other springs, a frequency of an input sound can be extracted.

Given a pure sound F_o cos(ωt), an amplitude of a spring S_i in a steady state becomes F₀/mZ_i by the equation (5). If the mass m of the object suspended in each spring is equal to each other, the spring with the greatest amplitude is the spring having the minimum Z_i. The relationship between the natural frequency ω_i of the spring and the frequency ω of pure tone can be obtained by differentiating Equation (3) with respect to Oi, and the result is as follows:

ω=ω_i√{square root over (1−2ζ²)}  (8)

where <1/√{square root over (2)}. If ζ is a small value near zero, then ω≈ωi. For example, ζ could be 0.001.

In order to find out the spring having the greatest amplitude, a numerical analysis method such as Runge-Kuta, which solves differential equations, is used. Given a pure sound F_o cos(ωt), the displacement x_i(t) and the velocity v_i(t) of each spring S_i which corresponds to the solution of equation (1) are calculated using the numerical analysis method. Since an energy of each spring is the sum of a kinetic energy and a potential energy, the energy of spring S_i can be obtained by equation (9).

$\begin{array}{cc}{E}_i=\frac{1}{2}{k}_i{x}_i^2+\frac{1}{2}{\mathrm{mv}}_i^2& \left(9\right)\end{array}$

The energy of the spring that has reached a steady state maintains a constant value. Thus, the displacement x at the time when the velocity v_i is 0 becomes the amplitude of the spring S_i. Therefore, the amplitude A_i of spring S_i in a steady state can be calculated by the equation below:

$\begin{array}{cc}{A}_i=\sqrt{\frac{2E_i}{k_i}}& \left(10\right)\end{array}$

The spring having the largest amplitude among the extracted amplitudes of the springs is the resonating spring. Therefore, it is possible to obtain the frequency of an input pure tone by using both the natural frequency ω_i of the spring having the largest amplitude and the equation (8).

(2) Extraction of Amplitude

In a steady state, the trajectory of the spring is given by the equation (5). Therefore, the relationship between an energy of a spring in a steady state, E_i,s, and an amplitude F_o of a given pure tone can be represented by the equation (11).

$\begin{array}{cc}{E}_{i,s}=\frac{1}{2}{k_i\left(\frac{F_o}{{\mathrm{mZ}}_i}\right)}^2& \left(11\right)\end{array}$

In addition, the energy in a steady state, E_i,s, can be obtained by putting the displacement x_i and the velocity v_i in the steady state, which are obtained by solving the equation (1) with the numerical analysis method, into the equation (9). Therefore, the amplitude F_o of a given pure tone becomes as below:

$\begin{array}{cc}{F}_o={\mathrm{mZ}}_i\sqrt{\frac{2E_{i,s}}{k_i}}& \left(12\right)\end{array}$

The natural frequency ω_i of the spring that resonates with an external force is almost the same with the frequency of the external force. Therefore, if putting ω≈ω_i into the equation (3), then Zi=2ω_i²ζ. If putting both of this result and ω_i=√{square root over (k_i/m)} into the equation (12), the amplitude F₀ of the input pure tone can be calculated by the equation (13).

F _o=2ζω_i√{square root over (2mE_i,s)}  (13)

2. In a Transient State

(1) Extraction of Frequency

Assume that a pure tone F_o cos(ωt) is given over a time interval [t_a, t_b]. All springs start to move in an initial state where both displacements and velocities are zero. Using the numerical analysis technique, the energies of the springs are calculated at each time point, and the calculated results are put into the equation (10) to obtain the amplitudes of the springs at each time point. After that, the natural frequency of the spring having the largest amplitude is substituted into the equation (8) to calculate the frequency of the given pure tone.

(2) Extraction of Amplitude

Assume that an energy of a resonating spring S_i found by the numerical analysis is E_i(t). The amplitude A_i(t) of a spring S_i at time t can be calculated from E_i(t) using the equation (10).

According to the general solution of the equation (1), the amplitude A_i(t) of the spring S_i resonating with a given sound wave follows the trajectory of the equation (6), so that the spring S_i follows the trajectory of A_i(t)=(1—e^{−ζω(t−ta)})A_i,s in a time interval [t_a, t_b] starting from the initial state until it reaches the steady state. Here, A_i,s is the amplitude of the spring when it reaches the steady state. We call it an expected steady-state amplitude.

The energies E_i(t1) and E_i(t2) at two time points t₁, t₂ within the time interval [t_a, t_b] can be obtained with the numerical analysis method. Therefore, the amplitudes A_i(t₁) and A_i(t₂) can be obtained by substituting these results into the equation (10). The expected steady-state amplitude, A_i,s, can be obtained by putting the result into A_i(t)=(1−e^{−ζω(t−ta)})A_i,s, and the result is as the equation below:

$\begin{array}{cc}{A}_{i,s}=\frac{A_i\left(t_2\right)-A_i\left(t_1\right)e^{-\zeta \omega \left(t_2-t_1\right)}}{1-e^{-\zeta \omega \left(t_2-t_1\right)}}& \left(14\right)\end{array}$

Next, regarding the case where the frequency is the same but the volume of the sound changes, assume that the amplitude of the sound given at the point t_c has changed from F₁ to F₂. Let A_c be the amplitude of a spring at the time point t_c and let A_s be the amplitude of a spring at the time the spring will have approached a steady state after the external force changes to F₂. The behavior of the amplitude over time can be described by the following equation.

A(t)=A_s+(A_c−A_s)e^{−ζω(t−tc)}  (15)

Given the amplitudes A(t₁) and A(t₂) at two time points t₁ and t₂ within the time interval that the amplitude changes from A_c to A_s, it can be seen that the obtained A_s is the same as Equation (14).

For example, consider the case where the external force F₂=0 at the time point t_c. When the external force disappears, the energy of the spring decreases exponentially according to the equation (7). Namely, the measured amplitude of the spring after ΔT seconds from the time when the external force disappears will be A(t_c+ΔT)=A(t_c)e^−ζωΔt. Putting this measurement result into the equation (14) makes A_s=0, and it means the external force has disappeared.

Therefore, the expected steady-state amplitude, A_s, can be obtained by measuring the energy of the spring more than once. Using equation (10) which represents the correlation between amplitude and energy, the energy in the steady state, E_s, can be calculated and consequently the amplitude F_o of a given pure tone can be calculated using the equation (13).

Since the force applied to the spring is in the form of a periodic function, the energy does not increase uniformly within a period of a transient state. Considering this characteristic, when selecting the two time points t₁ and t₂ described above, the time interval is made to be the same with the period.

In this regard, it may not be able to select two time points of which a time difference between them is one period due to the relationship between the sampling rate of the sound data and the natural frequency of the spring. In this case, an error may occur, and two methods can be used to correct this error.

The first method is to select an adjacent sample which shows a less difference with a period. When the position S₁ of a sample and the period T of an audio data are given, the position S₂ of the second sample is calculated as [S₁+sampling rate×T+0.5]. The expected steady-state amplitude, A_s, is calculated by putting the time information of the two points and the amplitudes at the two points into the equation (14).

The second method uses a linear regression analysis. After extracting the amplitude at several points and putting the extracted data into the equation (15), the expected steady-state amplitude, A_s, is calculated by the linear regression analysis.

Based on the above theoretical background, a method for extracting a frequency of an input sound can be proposed as below.

Referring FIG. 3, a method, of which each step is performed by a computer, for extracting a frequency of an input sound according to an embodiment of the present disclosure may comprise the steps of:

• • (a) modeling a plurality of springs which have natural frequencies different from each other and oscillate in accordance with the input sound;
  • (b) estimating an expected steady-state amplitude, A_i,s, of the spring of which amplitude A_i(t) is the highest among the plurality of modeled springs;
  • (c) calculating energy E_i,s of said spring of which amplitude is the highest based on the expected steady-state amplitude, A_i,s; and
  • (d) calculating the amplitude F₀ of the input sound based on said energy E_i,s.

The step (a) may comprise the steps of: measuring displacements x_i(t) and velocities v_i(t) at time points for each of the plurality of springs (see the equation 1); calculating energy E_i(t) at each time point for each of the plurality of springs based on the displacements and the velocities (see the equation 9); and calculating an amplitude A_i(t) of each of the plurality of springs based on the energies E_i(t) (see the equation 10).

The step (b) can be calculated with the equation (14).

In the step (b), said expected steady-state amplitude, A_i,s(t), can be calculated based on the amplitudes at two different time points within a duration of the input sound.

A difference between the two different time points can be a period of the natural frequency of the corresponding spring.

When one of the two time points is t₁, a sampling rate of the input sound is SR, and the period of the natural frequency of the corresponding spring is T, the other t₂ of the two time points can be calculated by means of the equation below.

t ₂ =[t ₁ +SR×T+0.5]

The number of the plurality of springs N may be determined based on a range and a resolution of the frequency to be extracted.

FIGS. 4A to 4C are graphs representing the experimental results according to embodiments of the present disclosure.

FIG. 4A shows the result obtained by putting the energy E₂₀₀₀(t) of a spring, of which natural frequency is 2 kHz over time when a pure tone having a frequency of 2 kHz with a constant amplitude is input between 0.2 and 0.8 seconds, into the equation (13). This result is called a transient-state-pure-tone amplitude. The transient-state-pure-tone amplitude is an amplitude of the input pure tone which is calculated under the assumption that there is no change in the energy of the spring. As time goes by, the energy of the spring reaches a steady state. Therefore, as shown in FIG. 4A, the transient-state-pure-tone amplitude gradually reaches a steady state, and the amplitude at this time corresponds to the amplitude F_m(t) of the input pure tone. Here, m indicates a natural frequency of a spring.

FIG. 4B shows the amplitude F_m(t) of the input pure tone that is obtained by putting the measured amplitude of the spring into the equation (14) to obtain the expected steady-state amplitude of the spring, A_m,s(t), and applying the results to the steps (c) and (d) of the frequency extraction method above. As shown in FIG. 4B, the amplitude of the input pure tone is extracted from the starting point of the pure tone.

Method II for Extracting the Frequency and Amplitude of the Input Sound

According to the method I for extracting the frequency and amplitude of the input sound described above, if the input sound is a pure tone, the frequency and amplitude of the input sound can be effectively extracted.

Now, assume that there are n types of pure tones constituting a complex tone F(t)=Σ_jF_j cos(ω_jt+φ_j). If n=1, the pure tone of a given sound can be found by selecting the spring having the largest amplitude among the springs. However, if n>1, it is difficult to find out pure tones constituting the complex tone by selecting top n springs in the order of amplitude.

The first reason is that the amplitude of a spring of which the frequency is adjacent to the spring having the largest amplitude could be greater than the amplitude of the spring which resonates with other pure tones constituting the complex tone. The second reason is that, as shown in the trajectory after 0.8 seconds in FIG. 2, even though the external force disappears, it takes time until the amplitude of the spring reaches 0, so the amplitude of the sound that does not exist anymore could be greater than the amplitude of other pure tones.

Accordingly, in this embodiment, instead of finding the local maximum value among the spring amplitudes at each time point, a method of finding the local maximum value from the results of multiplying an expected steady-state amplitude and a transient-state-pure-tone amplitude is proposed.

1. Expected Steady-State Amplitude and Filtered Pure-Tone Amplitude

First, in order to extract the pure tones constituting a complex tone, the amplitude A_i(t) of each spring S_i is calculated by applying the step (a) of the method I to each spring for extracting the frequency of an input sound. FIG. 5A shows the amplitudes of springs of which natural frequencies are around 1 kHz as a result measured at 215 milliseconds when a sound having a frequency of 1 kHz with a constant amplitude starts at 200 milliseconds. FIG. 5A shows that the amplitude of the spring that does not resonate is lower than that of the spring that resonates.

Next, an expected steady-state amplitude, A_i,s(t), is calculated by applying the step (b) of the method I for extracting the frequency of an input sound to the amplitude A_i(t) of each spring S_i. However, the equation (14) which calculates the expected steady-state amplitude is an equation derived from the equation (7) which describes the behavior of a resonating spring. Therefore, high amplitudes could be resulted even at the frequencies away from the resonant frequency as in FIG. 5B.

Accordingly, the following steps are performed. The third step is to calculate a transient-state-pure-tone amplitude, F_i,t(t), by putting the amplitude A_i(t) of the spring S_i into the equation (13). In addition, a predicted pure-tone amplitude, F_i,s(t), is calculated by applying steps (c) and (d) of the method I for extracting the frequency of the input sound to the expected steady-state amplitude, A_i,s(t).

As the final step, a filtered pure-tone amplitude, F_i,p(t), is calculated by multiplying the transient-state-pure-tone amplitude, F_i,t(t), with the predicted pure-tone amplitude, F_i,s(t), as in F_i,p(t)=F_i,t(t)×F_i,s(t). Additionally, the result of multiplication of the amplitudes may be divided by the maximum amplitude of the sound in order not to exceed 1 but to be normalized. For example, if the sound is expressed as a 16-bit integer, the result is divided by 32,767.

A filtered pure-tone amplitude has the characteristic that 1) the amplitude becomes 0 when the sound disappears, and 2) the amplitudes of frequencies away from a resonant frequency in the frequency domain are low.

FIG. 5C shows the filtered pure-tone amplitude, which is the result of multiplication of the amplitudes in FIGS. 5A and 5B with respect to the same frequency. FIGS. 5D to 5F show the transient-state-pure-tone amplitude, the predicted pure-tone amplitude, and the filtered pure-tone amplitude obtained by the spring with a natural frequency of 1 kHz, respectively. Especially, it is shown that, after the input sound disappears at 0.8 seconds, the amplitude in FIG. 5D remains not to be zero, but the amplitudes in FIGS. 5E and 5F become zero. FIG. 5G to 51 show the results for the spring with the natural frequency of 1,020 Hz. Apparently, the filtered pure-tone amplitude, F_1020,p(t), is very small compared to the filtered pure-tone amplitude, F_1000,p(t), of the resonating spring of FIG. 5F.

2. Finding a Pure Tone from Local Maximum Values

FIG. 6 is a graph showing frequency vs. filtered pure-tone amplitude of a complex tone composed of five pure tones of 100 Hz, 250 Hz, 500 Hz, 1 kHz, and 4 kHz. As shown in FIG. 6, if frequency intervals of the sounds constituting the complex tone are broad, each pure tone frequency generates a local maximum value among local maximum values in a frequency. Using these characteristics, several local maxima are obtained from a frequency vs. amplitude graph obtained by using the filtered pure-tone amplitude. Then the local maxima of those several local maxima are obtained again. Finally, frequencies corresponding to the local maxima are regarded as frequencies of the pure tones constituting the complex tone.

However, if the frequency interval is narrow, no local maximum might exist between two adjacent local maxima. FIG. 7 is a part of the graph for frequency vs. filtered pure-tone amplitude of a complex tone composed of five pure tones of 112 Hz, 181 Hz, 1,034 Hz, 5,017 Hz, and 5,034 Hz. It shows that no local maximum exists between the two local maxima that are generated by the two adjacent frequencies, 5,017 Hz and 5,034 Hz. The characteristic of this case is that the frequency interval is narrow and the two filtered pure-tone amplitudes are similar. Therefore, if the frequency difference between of two adjacent local maxima in filtered pure-tone amplitudes is within a certain width (e.g., the bandwidth of a high-amplitude frequency) and the ratio of those filtered pure-tone amplitudes is equal to or greater than a certain level (e.g. 0.5), both frequencies are treated as the frequencies of pure tones constituting the complex tone.

Based on the theoretical background described above, the following method for extracting the frequency of the input sound is proposed.

Referring FIG. 8, a method, of which each step is performed by a computer, for extracting a frequency of an input sound according to an embodiment of the present disclosure comprises the steps of:

• • (1) modeling a plurality of springs, each spring S_i (1≤i≤N) of which has natural frequencies ω_i, being different from each other, and oscillates according to the input sound;
  • (2) calculating transient-state-pure-tone amplitudes of the plurality of modeled springs at each time t, {F_i,t(t)|1≤i≤N}, based on displacements and velocities of the modeled springs;
  • (3) calculating expected steady-state amplitudes of the plurality of modeled springs at each time t, {A_i,s(t)|1≤i≤N};
  • (4) calculating predicted pure-tone amplitudes, {F_i,s(t)|1≤i≤N}, based on the expected steady-state amplitudes at each time t, {A_i,s(t)|1≤i≤N};
  • (5) calculating filtered pure-tone amplitudes at each time t, {F_i,p(t)|1≤i≤N}, by multiplying the transient-state-pure-tone amplitude, F_i,t(t), with the predicted pure-tone amplitude, F_i,s(t), for each spring S_i;
  • (6) extracting natural frequencies of the springs, each filtered pure-tone amplitude of which is a local maximum in a frequency range; and
  • (7) using the natural frequency for sound recognition or sound synthesis.

The step (1) may comprise the steps of: measuring displacements x_i(t) and velocities v_i(t) at different time points for each of the plurality of springs (see the equation 1); calculating an energy E_i(t) at each time point for each of the plurality of springs based on the displacements x_i(t) and the velocities v_i(t) (see the equation 9); and calculating an amplitude A_i(t) at each time point for each of the plurality of springs based on the energy E_i(t) (see the equation 10).

The equation 13 can be used in the step (2), the equation 14 can be used in the step (3), and the equation 13 can be used in the step (4).

The number of the plurality of springs, N, may be determined based on a range and a resolution of the frequencies to be extracted.

In the step (3), the expected steady-state amplitudes, A_i,s(t), can be calculated based on the amplitudes at two time points within a duration of the input sound.

In the step (3), the expected steady-state amplitudes, A_i,s(t), can be calculated by means of the equation below:

$A_{i,s}=\frac{A_i\left(t_2\right)-A_i\left(t_1\right)e^{-\zeta \omega \left(t_2-t_1\right)}}{1-e^{-\zeta \omega \left(t_2-t_1\right)}}$

where t₁ and t₂ are the two different time points within the duration of input sound, t₂>t₁, Ai(t₁) is an amplitude of any spring among the plurality of springs at t₁, Ai(t₂) is an amplitude of said spring at t₂, ζ is a damping ratio of said spring, and ω satisfies the equation ω=ω_i√{square root over (1−2ζ²)}, where ω_i is the natural frequency of said spring.

A difference between the two different time points can be a period of the natural frequency of the corresponding spring.

When one of the two time points is t₁, a sampling rate of the input sound is SR, and a period of the natural frequency of the corresponding spring is T, the other t₂ of the two time points is calculated by the equation below.

t ₂ =[t ₁ +SR×T+0.5]

In step (7), the natural frequency may be used for sound recognition or sound synthesis.

The sound processing method and sound processing apparatus according to the present embodiment can be applied not only to human voice but also to all types of sounds such as objects such as musical instruments and animals. In the present disclosure, sound recognition includes: speech recognition in a sense of converting human speech into text; speaker verification/speaker identification for determining whose voice an input sound corresponds to; source separation such as discrimination of a specific person's voice in a state in which the voices of a plurality of speakers are mixed, separation of voice from noise when noise is mixed, and separation of vocals from songs excluding instruments; sound direction detection; sound-based nomenclature diagnostics such as coughing or breathing; sound-based machine fault diagnostics based on mechanical sounds; and Sonar for navigating undersea terrain, ranging objects and more.

Sound recognition or sound synthesis are example to which the natural frequency obtained by the present invention can be applied, and the scope of the present invention is not limited thereto. The present invention can be applied to any field in which periodic properties or Fourier transforms are used such as price prediction for cryptocurrencies and stocks and image processing such as denoising.

Hereinafter, the experimental results according to the present embodiment will be described. To show the performance of the DJ transform according to the present disclosure, the results of the DJ transform and that of the STFT were compared. In the DJ transform, 7,951 springs of which natural frequencies are from 50 Hz to 8,000 Hz were used, respectively. The frequency interval of springs was 1 Hz. A 25 milliseconds window was used for the STFT.

The DJ transform was performed in an NVIDIA M40 GPU environment with 3,072 cores and 12 GB of memory and was implemented using the C language API of Cuda Toolkit 8.0. It took about 0.6 seconds to do the DJ transform for a 1 second audio data.

FIGS. 9A to 9F are diagrams showing the results of the STFT and the DJ transform in terms of the frequency resolution. In FIGS. 9A to 9F, the first rows show the results of the STFT, the second rows show the frequencies of the input sounds, and the third rows show the results of the DJ transform according to an embodiment of the present disclosure.

As shown in FIGS. 9A to 9F, the frequency resolution of the STFT result was 40 Hz. In addition, when the frequencies of pure tones were 400 Hz, 408 Hz, and 416 Hz, the peak was output at 400 Hz, and when the frequencies of pure tones were 424 Hz, 432 Hz, and 440 Hz, the peak was output at 440 Hz. However, the DJ transform results were matched with all the frequencies of pure tones. That means the frequency resolution of the DJ transform result was 1 Hz.

Three experiments were conducted to compare the results of the DJ transform with the STFT in terms of temporal resolution.

The first experiment was to check the frequency extracted at the time point where an input frequency changes. FIG. 10A shows a result of extracted frequencies by the DJ transform when a 1 kHz pure tone had been input for 500 milliseconds and a 2 kHz pure tone was input just after 500 milliseconds, FIG. 10B shows a result of extracted frequencies by the DJ transform when a 2 kHz pure tone had been input for 500 milliseconds and a 1 kHz pure tone was input just after 500 milliseconds, FIG. 10C shows a result of extracted frequencies by the DJ transform when a 4 kHz pure tone had been input for 500 milliseconds and a 2 kHz pure tone was input just after 500 milliseconds, and FIG. 10D shows a result of extracted frequencies by the DJ transform when a 2 kHz pure tone had been input for 500 milliseconds and a 4 kHz pure tone was input just after 500 milliseconds. Obviously, FIGS. 10A to 10D show that the boundaries between the two frequencies were at 500-milliseconds. Specifically, until 500 milliseconds, the frequencies of 1 kHz, 2 kHz, 4 kHz and 2 kHz of the input pure tones were clearly displayed, and immediately after 500 milliseconds, the frequencies of 2 kHz, 1 kHz, 2 kHz and 4 kHz of the changed pure tone were displayed with about 10% error only. However, in the STFT results shown in FIGS. 11A to 11D, two frequencies are simultaneously extracted on the 500-millisecond boundary.

The second experiment is to extract frequencies from the sounds that appear and disappear rapidly. The first rows of FIGS. 12A to 12C show the frequency extraction results when a 1 kHz pure tone is generated for 5 milliseconds, and silent for the next 5 milliseconds from 200 milliseconds to 800 milliseconds (when a flicker signal is repeatedly input). The second rows show the results when a 1 kHz pure tone continuously is input from 200 milliseconds to 800 milliseconds (when a continuous signal is input). FIG. 12A is for the frequency components of the input sound over time, FIG. 12B is for the DJ transform results, and FIG. 12C is for the STFT result.

In FIG. 12B showing the results of DJ transform, the repeated flicker signal results in a broken line while the continuous signal results in a solid line thereby two signals are distinguished apparently. On the other hand, the results of the STFT shown in FIG. 12C show a solid line at 1 kHz, therefore, the distinction between the flicker signal and the continuous signal is not clear.

The upper drawing in FIG. 12B shows relatively weak broken lines at 1.1 kHz and 0.9 kHz. These lines are interpreted as the result of 100 Hz signal due to the repeated input of every 10 milliseconds cycle. On the other hand, in the STFT result, solid lines appear at 0.88 kHz, 0.92 kHz, 0.96 kHz, 1.04 kHz, 1.08 kHz and 1.12 kHz when looking at the upper drawing in FIG. 12C. It is conjectured that the reason the STFT result occurs is because 0.9 kHz and 1.1 kHz frequency components are generated by the 100 Hz signal and those components are represented by 40 Hz intervals due to the 40 Hz frequency resolution of the STFT.

The third experiment is an extension of the second experiment, which shows the results in frequency extraction when a 1 kHz and a 2 kHz pure tones are alternately generated for 5 milliseconds from 200 milliseconds to 800 milliseconds (FIGS. 13A to 13C). FIG. 13B shows that the DJ transform produces the 1 kHz pure tone and the 2 kHz pure tone that are clearly separated in 5 milliseconds units. On the other hand, when the STFT is used, boundaries between the pure tones are not distinguishable as shown in FIG. 13C.

The first rows of FIGS. 14A to 14C show the input waveform, the result of the DJ transform, and the result of the STFT when a 420 Hz pure tone is input, and the second rows show the input waveform, the DJ transform result, and the STFT result when a complex tone composed of 400 Hz and 440 Hz is input. FIG. 14A shows input waveforms, and FIGS. 14B and 14C show the DJ transform results and the STFT results, respectively.

As can be seen in FIGS. 14B and 14C, the DJ transform extracts 420 Hz frequencies from a pure tone, and 400 Hz and 440 Hz frequencies from a complex tone. On the other hand, there is little difference between the results extracted from both pure tones and the complex tone with the STFT.

Since the complex tone is composed of 400 Hz and 440 Hz, the amplitude fluctuates in a 40 Hz cycle as shown in the bottom of FIG. 14A. On the other hand, as in the bottom of FIG. 14B, the DJ transform well reflects the characteristic of the amplitude fluctuation.

FIG. 15 shows a sound processing device 100 according to an embodiment of the present disclosure.

The sound processing device 100 may be any one of various types of digital computers. For example, the sound processing device may be a laptop computer, a desktop computer, a workstation, a server, a blade server, a mainframe, or any other suitable computers. Alternatively, the sound processing device may be any one of various types of mobile devices. For example, the sound processing device may be a personal digital assistant (PDA), a cellular phone, a smartphone, a wearable device, or any other similar computing devices. Components, connections and relations therebetween, and functions thereof, disclosed in the present disclosure, are merely illustrative and do not limit the scope of the present disclosure.

As shown in FIG. 15, the sound processing device 100 includes a computing unit 101, and performs an appropriate operation and process according to a computer program stored in a read-only memory (ROM) 102 or a computer program loaded into a random access memory (RAM) 103 from a storage unit 108. The RAM 103 may store programs and data required to operate the sound processing device 100. The computing unit 101, the ROM 102, and the RAM 103 are connected to each other via a bus 104. An I/O interface 105 is also connected to the bus 904.

A plurality of components of the sound processing device 100 are connected to the I/O interface 105. The plurality of components include an input unit 106, such as a keyboard, a mouse, or a microphone, an output unit 107, such as a monitor, or a speaker, a storage unit 108, such as a magnetic disk or an optical disc, and a communication unit 109, such as a network card, a modem, or a wireless communication transceiver. For example, a sound from which a fundamental frequency is to be extracted may be input through the microphone. The communication unit 109 allows the sound processing device 100 to exchange information/data with other devices through a computer network, such as the Internet, and/or telegraph networks.

The computing unit 101 may be a general purpose/dedicated processing component having processing and calculation functions. Some examples of the computing unit 101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), a dedicated artificial intelligence calculation chip, a computing unit configured to execute a machine learning model algorithm, a digital signal processor (DSP), and any other suitable processors, controllers, and microcontrollers. The computing unit 901 performs the sound processing method described above. For example, in an embodiment, the sound processing method may be implemented by a computer software program and may be stored in a machine-readable medium, such as the storage unit 108. In an embodiment, some or the entirety of a computer program may be loaded into and/or installed in the sound processing device 100 by the ROM 102 and/or the communication unit 109. When the computer program is loaded into the RAM 103 and executed by the computing unit 101, one step or a plurality of steps of the sound processing method described above may be performed. In another embodiment, the computing unit 901 is configured to perform the sound processing method according to the embodiment of the present disclosure in any other suitable manners (e.g. firmware).

In the present disclosure, the machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses, and devices, or suitable combinations thereof. More specific examples of the machine-readable storage medium may include electrical connection based on one line or a plurality of lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), a erasable programmable read-only memory (EPROM or flash memory), optical fiber, CD-ROM, an optical storage device, a magnetic storage device, or any suitable combinations thereof.

A sound may be input to the sound processing device 100 through the microphone. The sound input through the microphone may be stored in an electronic form and may then be used. Alternatively, the input sound may be directly provided as an electronic file through the storage unit 108 or may be stored in an electronic form through the communication unit 109 and may then be used.

Although the present disclosure has been described in detail through preferred embodiments, the present disclosure is not limited thereto, and various changes and applications can be made without departing from the technical spirit of the present disclosure, which is obvious to a person skilled in the art. Therefore, the scope of protection for the present disclosure should be interpreted by the following claims, and all technical ideas within the scope equivalent thereto should be interpreted as being included in the scope of the present disclosure.

Claims

1. A sound processing method comprising the steps of: sampling, by a computer, natural frequencies of a plurality of springs, the plurality of springs having natural frequencies different from each other and oscillate according to an input sound;determining filtered pure-tone amplitudes of the plurality of springs by the computer: calculating transient-state-pure-tone amplitudes of the plurality of modeled springs; calculating expected steady-state amplitudes of the plurality of modeled springs; calculating predicted pure-tone amplitudes based on the expected steady-state amplitudes; calculating filtered pure-tone amplitudes by multiplying the transient-state-pure-tone amplitudes with the predicted pure-tone amplitudes;extracting, by the computer, a natural frequency of at least one spring of the plurality of springs which corresponds to a local maximum value among the filtered pure-tone amplitudes; andusing the natural frequency for sound recognition or sound synthesis.

2. The method according to claim 1, wherein said expected steady-state amplitude is calculated based on the amplitudes at least two time points within a duration of the input sound.

3. The method according to claim 1, wherein said expected steady-state amplitude is calculated by the equation below:

4. The method according to claim 2, wherein a difference between the two different time points is a period of the natural frequency of the corresponding spring.

5. The method according to claim 2, wherein if one of the two time points is t1, a sampling rate of the input sound is SR, and a period of the natural frequency of the corresponding spring is T, then the other t2 of the two time points is calculated by the equation below. t2=[t1+SR×T+0.5]

6. The method according to claim 2, wherein the expected steady-state amplitude is calculated by substituting amplitudes at least two points in the duration of the input sound into the following equation and using a linear regression analysis: A(t)=As+i(Ac−As)e−ζω(t−tc) where A(t) is an amplitude of any spring among said plurality of springs at t,As is the expected steady-state amplitude of said spring,Ac is an amplitude of said spring at te,tc is a time point before the at least two points in the duration of the input sound,ζ is a damping ratio of said spring, andω satisfies the equation ω=√{square root over (ω1−2ζ2)}, where ωi is the natural frequency of said spring.

7. The method according to claim 1, wherein said modeling step comprises the steps of: measuring displacements and velocities at time points for each of the plurality of springs;calculating an energy at each time point for each of the plurality of springs based on the displacements and the velocities; andcalculating an amplitude at each time point for each of the plurality of springs based on the energy.

8. The method according to claim 1, wherein the number of the plurality of springs is determined based on a range and a resolution of the frequency to be extracted.

9. The method according to claim 1, wherein the sound recognition includes at least one of: speech recognition; speaker verification; speaker identification; source separation; sound direction detection; sound-based nomenclature diagnostics; sound-based machine fault diagnostics; or Sonar for navigation undersea terrain or ranging objects.

10. A non-transitory computer-readable recording medium on which the method according to claim 1 is recorded.

11. A sound processing device comprising: a memory; anda processor configured to: produce displacements and velocities of a plurality of springs by modeling the plurality of springs which have natural frequencies different from each other and oscillate according to an input sound, the displacements and the velocities being recorded in the memory,calculate transient-state-pure-tone amplitudes of the plurality of modeled springs,calculate expected steady-state amplitudes of the plurality of modeled springs, calculating predicted pure-tone amplitudes on the basis of the expected steady-state amplitudes,calculate filtered pure-tone amplitudes by multiplying the transient-state-pure-tone amplitudes with the predicted pure-tone amplitudes,extract the natural frequency of at least one spring of the plurality of springs which corresponds to a local maximum value among the filtered pure-tone amplitudes, anduse the natural frequency for sound recognition or sound synthesis.

12. The device according to claim 11, wherein the sound recognition includes at least one of: speech recognition; speaker verification; speaker identification; source separation; sound direction detection; sound-based nomenclature diagnostics; sound-based machine fault diagnostics; or Sonar for navigation undersea terrain or ranging objects.

13. A sound processing method comprising the steps of: sampling, by a computer, natural frequencies of a plurality of springs, the plurality of springs having natural frequencies different from each other and oscillate according to an input sound;estimating an expected steady-state amplitude of the spring of which the amplitude is the highest among the plurality of modeled springs;calculating an energy of at least one spring of the plurality of springs of which the amplitude is the highest based on the expected steady-state amplitudes;calculating an amplitude of the input pure tone based on the energy; andusing the amplitude of the input pure tone for sound recognition or sound synthesis.

14. The method according to claim 13, wherein said expected steady-state amplitude is calculated by the equation below:

15. The method according to claim 13, wherein said sampling step comprises the steps of: measuring a displacement and a velocity at each time point for each of the plurality of springs;calculating an energy at each time point for each of the plurality of springs based on the displacement and the velocity; andcalculating an amplitude at each time point for each of the plurality of springs based on the energy.

16. The method according to claim 13, wherein the sound recognition includes at least one of: speech recognition; speaker verification; speaker identification; source separation; sound direction detection; sound-based nomenclature diagnostics; sound-based machine fault diagnostics; or Sonar for navigation undersea terrain or ranging objects.

17. A non-transitory computer-readable recording medium on which the method according to claim 13 is recorded.

18. A sound processing device comprising: a memory; anda processor configured to: produce displacements and velocities of a plurality of springs by modeling the plurality of springs which have natural frequencies different from each other and oscillate according to an input sound, the displacements and the velocities being recorded in the memory,estimate an expected steady-state amplitude of a spring of which the amplitude is the highest among the plurality of modeled springs,calculate an energy of a spring of which the amplitude is the highest based on the expected steady-state amplitudes,calculate an input pure tone amplitude based on said energy, anduse the input pure tone amplitude for sound recognition or sound synthesis.

19. A method for checking error among pure tone frequencies comprising: inputting, by a computer, a frequency of a plurality of springs to which an input sound is applied, the frequency maintains a first value to a certain point of time and turns into a second value at the certain point,wherein a result of frequency transform to the certain point indicates the first value, andchecking that immediately after the turning point, a transient error from the first value to the second value is within 10%.

20. The method according to claim 19, wherein the method comprising the steps of: sampling, by the computer, natural frequencies of the plurality of springs which have natural frequencies different from each other and oscillate according to the input sound;calculating transient-state-pure-tone amplitudes of the plurality of springs;calculating expected steady-state amplitudes of the plurality of springs;calculating predicted pure-tone amplitudes based on the expected steady-state amplitudes;calculating filtered pure-tone amplitudes by multiplying the pure-tone amplitudes with the predicted pure-tone amplitudes; and extracting the natural frequency of at least one spring of the plurality of springs which corresponds to a local maximum value among the filtered pure-tone amplitudes.