Embodiments of the present invention will be described below by referring to the drawings.
The input section 1 receives a musical acoustic signal from which the tempo is to be detected. An analog signal received from a microphone or other device may be converted to a digital signal by an A-D converter (not shown), or digitized musical data, such as that in a music CD, may be directly taken (ripped) as a file and opened. When a digital signal received in this way is a stereo signal, it is converted to a monaural signal to simplify subsequent processing.
The digital signal is input to the scale-note-power detection section 2. The scale-note-power detection section 2 is formed of sections shown in
Among them, a waveform pre-processing section 20 down-samples the acoustic signal sent from the input section 1, at a sampling frequency suited to subsequent processing.
The down-sampling rate is determined by the range of a musical instrument used for beat detection. Specifically, to use the performance sounds of rhythm instruments having a high range, such as cymbals and hi-hats, for beat detection, it is necessary to set the sampling frequency after down-sampling to a high frequency. To mainly use the bass note, the sounds of musical instruments such as bass drums and snare drums, and the sounds of musical instruments having a middle range for beat detection, it is not necessary to set the sampling frequency after down-sampling to such a high frequency.
When it is assumed that the highest note to be detected is A6 (C4 serves as the center “do”), for example, since the fundamental frequency of A6 is about 1,760 Hz (when A4 is set to 440 Hz), the sampling frequency after down-sampling needs to be 3,520 Hz or higher, and the Nyquist frequency is thus 1,760 Hz or higher. Therefore, when the original sampling frequency is 44.1 kHz (which is used for music CDs), the down-sampling rate needs to be about one twelfth. In this case, the sampling frequency after down-sampling is 3,675 Hz.
Usually in down-sampling processing, a signal is passed through a low-pass filter which removes components having the Nyquist frequency (1,837.5 Hz in the current case), that is, half of the sampling frequency after down-sampling, or higher, and then data in the signal is skipped (11 out of 12 waveform samples are discarded in the current case).
Down-sampling processing is performed in this way in order to reduce the FFT calculation time by reducing the number of FFT points required to obtain the same frequency resolution in FFT calculation to be performed after the down-sampling processing.
Such down-sampling is necessary when a sound source has already been sampled at a fixed sampling frequency, as in music CDs. However, when an analog signal input from a microphone or other device to the input section 1 is converted to a digital signal by the A-D converter, the waveform pre-processing section 20 can be omitted by setting the sampling frequency of the A-D converter to the sampling frequency after down-sampling.
When down-sampling is finished in this way in the waveform pre-processing section 20, an FFT calculation section 21 applies FFT to the output signal of the waveform preprocessing section 20 at predetermined time intervals (frames).
FFT parameters (number of FFT points and FFT window shift) should be set to values suitable for beat detection. Specifically, if the number of FFT points is increased to increase the frequency resolution, the FFT window size is enlarged to use a longer time period for one FFT cycle, reducing the time resolution. This FFT characteristic needs to be taken into account. (In other words, for beat detection, it is better to increase the time resolution with the frequency resolution suppressed.) There is a method in which, instead of using a waveform having the same length as the window length, waveform data is specified only for a part of the window and the remaining part is filled with zeros to increase the number of FFT points without suppressing the time resolution. However, the number of waveform samples needs to be set up to a certain point in order to also detect a low-note power correctly.
The above points have been taken into account. In the apparatus, the number of FFT points is set to 512, the window shift is set to 32 samples (window overlap is 15/16), and filling with zeros is not performed. When the FFT calculation is performed with these settings, the time resolution is about 8.7 ms, and the frequency resolution is about 7.2 Hz. A time resolution of 8.7 ms is sufficient because the length of a thirty-second note is 25 ms in a musical piece having a tempo of 300 quarter notes per minute.
The FFT calculation is performed in this way in each frame interval; the squares of the real part and the imaginary part of the FFT result are added and the sum is square-rooted to calculate the power spectrum; and the power spectrum is sent to a power detection section 22.
The power detection section 22 calculates the power of each note in the scale from the power spectrum calculated in the FFT calculation section 21. The FFT calculates just the powers of frequencies that are integer multiples of the value obtained when the sampling frequency is divided by the number FFT points. Therefore, the following process is performed to detect the power of each note in the scale from the power spectrum. The power of the spectrum having the maximum power among power spectra corresponding to the frequencies falling in the range of 50 cents (100 cents correspond to one semitone) above and below the fundamental frequency of each note (from C1 to A6) in the scale is set to the power of the note.
When the powers of all the notes in the scale have been detected, they are stored in a buffer. The waveform reading position is advanced by a predetermined time interval (one frame, which corresponds to 32 samples in the above case), and the processes in the FFT calculation section 21 and the power detection section 22 are performed again. This set of steps is repeated until the waveform reading position reaches the end of the waveform.
With the above-described processing, the power of each note in the scale for each predetermined time interval is stored in the buffer 23 for the acoustic signal input to the input section 1.
The structure of the beat detection section 3, shown in
The beat detection section 3 detects an average beat interval (that is, tempo) and the positions of beats, based on a change in the power of each note in the scale for each frame interval, the power being output from the scale-note-power detection section 2. The beat detection section 3 first calculates, in step S100, the total of incremental values of the powers of the notes in the scale (the total of the incremental values in power from the preceding frame for all the notes in the scale; if the power is reduced from the preceding frame, zero is added).
When the power of the i-th note in the scale at frame time “t” is called Li(t), an incremental value Laddi (t) of the power of the i-th note is as shown in the following expression 1. The total L(t) of incremental values of the powers of all the notes in the scale at frame time “t” can be calculated by the following expression 2, where T indicates the total number of notes in the scale.
The total value L(t) indicates the degree of change in all the notes in each frame interval. This value suddenly becomes large when notes start sounding and increases when the number of notes that start sounding at the same time increases. Since notes start sounding at the position of a beat in many musical pieces, it is highly possible that the position where this value becomes large is the position of a beat.
As an example,
As shown in the lower row in the figure, the total of the incremental values in power of the notes in the scale has peaks periodically. The positions of these periodic peaks are those of beats.
To obtain the positions of beats, the beat detection section 3 first obtains the time difference between these periodic peaks, that is, the average beat interval. The average beat interval can be obtained from the autocorrelation of the total of the incremental values in power of the notes in the scale (in step S102 in
The autocorrelation φ(τ) of the total L(t) of the incremental values in power of the notes in the scale at frame time “t” is given by the following expression 3:
where N indicates the total number of frames and τ indicates a time delay.
The range of “τ” where the autocorrelation is obtained needs to be changed according to an expected tempo range of the musical piece. For example, when calculation is performed in a range of 30 to 300 quarter notes per minute in metronome marking, the range where autocorrelation is calculated is from 0.2 to 2.0 seconds. The conversion from time (seconds) to frames is given by the following expression 4.
The beat interval may be set to “τ” where the autocorrelation φ(τ) is maximum in the range. However, since “τ” where the autocorrelation is maximum in the range is not necessarily the beat interval for all musical pieces, it is desired that candidates for the beat interval be obtained from “τ” values where the autocorrelation is local maximum in the range (in step S104 in
When the beat interval is determined in this way (the determined beat interval is called “τmax”), the starting beat position is determined first.
A method for determining the starting beat position will be described with reference to
The cross-correlation of L(t) and M(t) is calculated with the function M(t) shifted in a range of 0 to “Tmax”−1.
The cross-correlation r(s) can be calculated from the characteristics of the function M(t) by the following expression 6.
In this case, “In” needs to be determined appropriately according to the length of a top no-sound part (“n”=10 in the case shown in
The cross-correlation r(s) is obtained in the range where “s” is changed from 0 to “τmax”−1. The starting beat position is in the s-th frame where “s” maximizes r(s).
Once the starting beat position is determined, subsequent beat positions are determined one by one (in step S108 in
A method therefor will be described with reference to
In the expression, “k” is a coefficient that is changed according to the value of “s” and is assumed to have a normal distribution such as that shown in
When the value of “s” that maximizes r(s) is found, the second beat position b1 is calculated by the following expression 8.
b
1
=b
0+τmax+s Expression 8
The third beat position and subsequent beat positions can be obtained in the same way.
In a musical piece where the tempo hardly changes, beat positions can be obtained to the end of the musical piece by this method. However, in an actual performance, in some cases, the tempo fluctuates to some extent or becomes slow in parts.
To handle such tempo fluctuation, the following method can be used.
In the method, the function M(t) shown in
τ1=τ2=τ3=τ4=τmax
where τ1, τ2, τ3, and τ4 indicate the time periods between pulses from the start, as shown in the figure. Row 2 indicates that the time periods τ1 to τ4 are equally made larger or smaller, that is,
τ1=τ2=τ3=τ4=τmax+s (−τmax×F≦s≦τmax×F)
With this approach, beat positions can be obtained for a case where the tempo suddenly changes. Row 3 is for ritardando (rit.: gradually slower) or for accelerando (accel.: gradually faster), and the time periods between pulses are calculated as follows:
τ1=τmax
τ2=τmax+1×s
τ3=τmax+2×s
τ4=τmax+4×s(−τmax×F≦s≦τmax×F)
The coefficients used here, 1, 2, and 4, are just examples and may be changed according to the magnitude of a tempo change. Row 4 indicates that the beat position currently to be obtained is set to any of the five pulse positions for rit. or accel. shown in Row 3.
When these are all combined and cross-correlation between L(t) and M(t) is obtained, beat positions can be determined from the maximum cross-correlation, even for a musical piece having a fluctuating tempo. When row 2 or row 3 is used, the value of the coefficient “k” used for correlation calculation also needs to be changed according to the value of “s”.
The magnitudes of the five pulses are currently set to be the same. The total of the incremental values in power of the notes in the scale may be enhanced at the position where a beat is obtained by setting the magnitude of only the pulse at the position of the beat (indicated by a tentative beat position in
When the position of each beat is determined in the manner described above, the results are stored in the buffer 30. At the same time, the results may be displayed so that the user can check and correct them if they are wrong.
When a “play” button is pressed, the current musical acoustic signal is D-A converted and played back from a speaker or the like. The current playback position is indicated by a play position pointer, such as the vertical line in the figure, and the user can check for errors in beat detection positions while listening to the music. Furthermore, when a sound such as that of a metronome is played back at beat-position timing in addition to the playback of the original waveform, checking can be performed not only visually but also aurally, facilitating determination of detection errors. As a method for playing back the sound of a metronome, for example, a MIDI unit can be used.
A beat-detection position is corrected by pressing a “correct beat position” button. When this button is pressed, a crosshairs cursor appears on the screen. If the starting beat position was erroneously detected, when the cursor is moved to the correct position and the mouse is clicked, all beat positions are cleared from a position a certain distance (for example, half of τmax) before the position where the mouse was clicked, the position where the mouse was clicked is set as a tentative beat position, and subsequent beat positions are detected again.
Next, detecting a meter and a bar will be described.
The beat positions are determined in the processing described above. The degree of change of all the notes for each beat is then obtained. The degree of a sound change for each beat is calculated from the power of each note in the scale for each frame interval, output from the scale-note-power detection section 2.
When the frame number of the j-th beat is called bj and the frames of the beats immediately therebefore and thereafter are called bj−1 and bj+1, the average of the powers of each note in the scale from frames bj−1 to bj−1 and the average of the powers of each note in the scale from frames bj to bj+1−1 are calculated to obtain the incremental value; the degree of change of each note in the scale for each beat is obtained from the incremental value; and the total of the degrees of changes of the notes in the scale is calculated, which equals the degree of change of all the notes for the j-th beat.
In other words, when the power of the i-th note in the scale at frame time “t” is called Li(t), since the average of powers of the i-th note in the scale for the j-th beat, Lavgi(j), is expressed by the following expression 9, the degree of a change of the i-th note in the scale for the j-th beat, Baddi(j), is expressed by the following expression 10.
Therefore, the degree of change of all the notes for the j-th beat, B(t), is expressed by the following expression 11, where T indicates the total number of notes in the scale.
In
The meter is obtained from the autocorrelation of the degree of a change in sound for each beat. Generally, it is thought that musical pieces have a sound change at the first beat. Therefore, the meter can be obtained from the autocorrelation of the degree of a change in sound for each beat. For example, by using the following expression 12, the autocorrelation φ(τ) of the degree B(j) of a change in sound for each beat is obtained while the delay “τ” is changed in the range from 2 to 4, and the delay “τ” which maximizes the autocorrelation φ(τ) is used as the number of beats per measure:
where N indicates the total number of beats, the autocorrelation φ(τ) is obtained while the delay “τ” is changed in the range from 2 to 4, and the delay “τ” which maximizes the autocorrelation φ(τ) is used as the number of beats per measure.
Next, the first beat is obtained. The position where the degree B(j) of a change in sound for each beat is maximum is set as the first beat. In other words, when “τ” that maximizes φ(τ) is called “τmax” and “k” that maximizes X(k) shown in the following expression 13 is called “kmax”, the kmax-th beat indicates the position of the first beat, and the positions indicated by adding “τmax” successively to the kmax-th beat are the positions of subsequent beats:
where nmax is the maximum “n”, provided that τmax×n+k<N.
When the meter and the position of the first beat (the position of a bar line) are determined in the manner described above, the results are stored in the buffer 40. At the same time, it is desired that the results be displayed on the screen to allow the user to change them.
Since this method cannot handle musical pieces having a changing meter, it is necessary to ask the user to specify a position where the meter is changed.
With the foregoing structure, from the acoustic signal of a human performance of a musical piece having a fluctuating tempo, the average tempo of the entire piece of music and the correct beat positions, as well as the meter of the musical piece and the position of the first beat, can be detected.
In the figure, the chord-name detection apparatus includes an input section 1 for receiving an acoustic signal; a scale-note-power detection section 2 for beat detection for applying FFT to the received acoustic signal at predetermined time intervals (frames) by using parameters suited to beat detection and for obtaining the power of each note in a scale at each frame interval from the obtained power spectrum; a beat detection section 3 for summing up, for all the notes in the scale, an incremental value of the power of each note in the scale at each frame interval to obtain the total of the incremental values of the powers, indicating the degree of change of all the notes at each frame interval, and for detecting an average beat interval and the position of each beat, from the total of the incremental values of the powers; a bar detection section 4 for calculating the average power of each note in the scale for each beat, for summing up, for all the notes, an incremental value of the average power of each note in the scale for each beat to obtain a value indicating the degree of change of all the notes at each beat, and for detecting a meter and the position of a bar line, from the value indicating the degree of change of all the notes at each beat; a scale-note-power detection section 5 for chord detection for applying FFT to the received acoustic signal at predetermined time intervals (frames) different from those used for beat detection described above, by using parameters suited to chord detection, and for obtaining the power of each note in the scale at each frame interval from the obtained power spectrum; a bass-note detection section 6 for setting several detection zones in each bar and for detecting a bass note in each of the detection zones from the power of a low note in the scale at a portion corresponding to a first beat in each of the detection zones among the detected power of each note in the scale; a first bar-division determination section 7 for determining whether the bass note is changed according to whether the detected bass note in each of the detection zones is different and for determining whether it is necessary to divide the bar into a plurality of portions according to whether the bass note is changed; a second bar-division determination section 8 for setting several chord detection zones in the bar, for averaging the power of each note in the scale for each frame interval in each of the chord detection zones in a chord detection range specified as a range where chords are mainly performed, for summing up the averaged power of each note in the scale for each of 12 pitch notes in the scale, for dividing the total for each of the 12 pitch notes by the number of summed-up powers to obtain the average power of each of the 12 pitch notes in the scale, for re-arranging the powers in descending order of strength, for determining whether a chord is changed according to whether C notes or more of the top M strongest notes, M being three or more, in the scale in a detection zone are included in the top N strongest notes, N being three or more, in the scale in the detection zone immediately therebefore, and for determining whether it is necessary to divide the bar into a plurality of portions according to the degree of change in the chord; and a chord-name determination section 9 for determining, when the first bar-division determination section 7 and/or the second bar-division determination section 8 determine that it is necessary to divide the bar into several chord detection zones, a chord name in each of the chord detection zones according to the bass note and the power of each note in the scale in each of the chord detection zones and for determining, when the first bar-division determination section 7 and the first and second bar-division determination section 8 determine that it is not necessary to divide the bar into several chord detection zones, a chord name in the bar according to the bass note and the power of each note in the scale in the bar.
The input section 1 receives a musical acoustic signal from which the chord is to be detected. Since the basic structure thereof is the same as the structure of the input section 1 of the previously developed apparatus, a detailed description thereof is omitted here. If a vocal sound, which is usually localized at the center, disturbs subsequent chord detection, the waveform at the right-hand channel may be subtracted from the waveform at the left-hand channel to cancel the vocal sound.
A digital signal output from the input section 1 is input to the scale-note-power detection section 2 for beat detection and to the scale-note-power detection section 5 for chord detection. Since these scale-note-power detection sections are each formed of the sections shown in
A waveform pre-processing section 20, which is used as a component thereof, has the same structure as described above and down-samples the acoustic signal sent from the input section 1, at a sampling frequency suited to the subsequent processing. The sampling frequency after downsampling, that is, the down-sampling rate, may be changed between beat detection and chord detection, or may be identical to save the down-sampling time.
In beat detection, the down-sampling rate is determined according to a range used for beat detection. To use the performance sounds of rhythm instruments having a high range, such as cymbals and hi-hats, for beat detection, it is necessary to set the sampling frequency after down-sampling to a high frequency. To mainly use the bass note, the sounds of musical instruments such as bass drums and snare drums, and the sounds of musical instruments having a middle range for beat detection, the same down-sampling rate as that employed in the following chord detection may be used.
The down-sampling rate used in the waveform pre-processing section for chord detection is changed according to a chord-detection range. The chord-detection range means a range used for chord detection in the chord-name determination section. When the chord-detection range is the range from C3 to A6 (C4 serves as the center “do”), for example, since the fundamental frequency of A6 is about 1,760 Hz (when A4 is set to 440 Hz), the sampling frequency after down-sampling needs to be 3,520 Hz or higher, and the Nyquist frequency is thus 1,760 Hz or higher. Therefore, when the original sampling frequency is 44.1 kHz (which is used for music CDs), the down-sampling rate needs to be about one twelfth. In this case, the sampling frequency after down-sampling is 3,675 Hz.
Usually in down-sampling processing, a signal is passed through a low-pass filter which removes components having the Nyquist frequency (1,837.5 Hz in the current case), that is, half of the sampling frequency after down-sampling, or higher, and then data in the signal is skipped (11 out of 12 waveform samples are discarded in the current case). The same reason applies as that described above.
When down-sampling is finished in this way in the waveform pre-processing section 20, an FFT calculation section 21 applies a fast Fourier transform (FFT) to the output signal of the waveform pre-processing section at predetermined time intervals.
FFT parameters (number of FFT points and FFT window shift) are set to different values between beat detection and chord detection. If the number of FFT points is increased to increase the frequency resolution, the FFT window size is enlarged to use a longer time period for one FFT cycle, reducing the time resolution. This FFT characteristic needs to be taken into account. (In other words, for beat detection, it is better to increase the time resolution with the frequency resolution suppressed.) There is a method in which, instead of using a waveform having the same length as the window length, waveform data is specified only for a part of the window and the remaining part is filled with zeros to increase the number of FFT points without suppressing the time resolution. However, the number of waveform samples needs to be set up to a certain point in order to also detect low-note power correctly in the case of the present embodiment.
The above points have been taken into account. In the present embodiment, in beat detection, the number of FFT points is set to 512, the window shift is set to 32 samples (window overlap is 15/16), and filling with zeros is not performed; and, in chord detection, the number of FFT points is set to 8,192, the window shift is set to 128 samples (window overlap is 63/64), and 1,024 waveform samples are used in one FFT cycle. When the FFT calculation is performed with these settings, the time resolution is about 8.7 ms and the frequency resolution is about 7.2 Hz in beat detection; and the time resolution is about 35 ms and the frequency resolution is about 0.4 Hz in chord detection. Since each note in the scale of which the power is to be obtained falls in the range from C1 to A6, a frequency resolution of about 0.4 Hz in chord detection is sufficient because the smallest frequency difference in fundamental frequency, which is between C1 and C#1, is about 1.9 Hz. A time resolution of 8.7 ms in beat detection is sufficient because the length of a thirty-second note is 25 ms in a musical piece having a tempo of 300 quarter notes per minute.
The FFT calculation is performed in this way in each frame interval; the squares of the real part and the imaginary part of the FFT result are added and the sum is square-rooted to calculate the power spectrum; and the power spectrum is sent to a power detection section 22.
The power detection section 22 calculates the power of each note in the scale from the power spectrum calculated in the FFT calculation section 21. The FFT calculates just the powers of frequencies that are integer multiples of the value obtained when the sampling frequency is divided by the number of FFT points. Therefore, the same process as that described above is performed to detect the power of each note in the scale from the power spectrum. Specifically, the power of the spectrum having the maximum power among power spectra corresponding to the frequencies falling in the range of 50 cents (100 cents correspond to one semitone) above and below the fundamental frequency of each note (from C1 to A6) in the scale is set to the power of the note.
When the powers of all the notes in the scale have been detected, they are stored in buffers. The waveform reading position is advanced by a predetermined time interval (one frame, which corresponds to 32 samples for beat detection and to 128 samples for chord detection in the previous case), and the processes in the FFT calculation section 21 and the power detection section 22 are performed again. This set of steps is repeated until the waveform reading position reaches the end of the waveform.
With the above-described processing, the power of each note in the scale for each frame interval for the acoustic signal input to the input section 1 is stored in a buffer 23 and a buffer 50 for beat detection and chord detection, respectively.
Next, since the beat detection section 3 and the bar detection section 4 in
The positions of bar lines (frame number of each bar) are determined in the same procedure by the same structure as described above. Then, the bass note in each bar is detected.
The bass note is detected from the power of each note in the scale for each frame interval, output from the scale-note-power detection section 5 for chord detection.
In the previously developed apparatus, since it is possible that the bass note differs between a first half and a second half in a bar, each bar is divided into a first half and a second half; a bass note is detected in each half; and when different bass notes are detected in the first half and the second half, the chord is also detected in each of the first half and the second half. In that method, however, when different chords are used but an identical bass note is detected, for example, when the C chord is used in the first half of a bar and the Cm chord is used in the second half, since the bass note is identical, the bar is not divided and the C chord is detected in the whole bar.
In addition, in the above apparatus, the bass note is detected in the entire detection zone. In other words, when the detection zone is a bar, a strong note in the entire bar is detected as the bass note. In jazz music where the bass note changes frequently (the bass note changes in units of quarter notes or the like), however, the bass note cannot be detected correctly with this method.
Therefore, in the structure of the present embodiment, when the bass-note detection section 6 detects a bass note, several detection zones are specified in each bar, and the bass note in each detection zone is detected from the power of a low note in the scale corresponding to the first beat in each detection zone among the detected powers of the notes in the scale. This is because the root notes of the chord are played at the first beat in many cases even when the bass note changes frequently, as described above.
The bass note is obtained from the average strength of the powers of notes in the scale in a bass-note detection range at a portion corresponding to the first beat in the detection zone.
When the power of the i-th note in the scale at frame time “t” is called Li(t), the average power Lavgi(fs, fe) of the i-th note in the scale from frame fs to frame fe can be calculated by the following expression 14:
The bass-note detection section 6 calculates the average powers in the bass-note detection range, for example, in the range from C2 to B3, and determines the note having the largest average power in the scale as being the bass note. To prevent the bass note from being erroneously detected in a musical piece where no sound is included in the bass-note detection range or in a portion where no sound is included, an appropriate threshold may be specified so that the bass note is ignored if the power of the detected bass note is equal to or smaller than the threshold. When the bass note is regarded as an important factor in subsequent chord detection, it may be determined whether the detected bass note continuously keeps a predetermined power or more during the bass-note detection zone for the first beat, in order to select only a more reliable one as the bass note. Further, instead of determining the note having the largest average power in the scale in the bass-note detection range as being the bass note, the bass note may be determined such that the average power for each note is used to calculate the average power for each of 12 pitch names, the pitch name having the largest average power is determined to be the base pitch name, and the note having the largest average power in the scale among the notes included in the bass-note detection range, having the base pitch name is determined as being the bass note.
When the bass note is determined, the result is stored in a buffer 60. The bass-note detection result may be displayed on the screen to allow the user to correct it if it is wrong. Since the base range may change, depending on the musical piece, the user may be allowed to change the bass-note detection range.
Next, the first bar-division determination section 7 determines whether the bass note changes according to whether the detected bass note differs in each detection zone and determines whether it is necessary to divide the bar into a plurality of portions according to whether the bass note changes. In other words, when the detected bass note is identical in each detection zone, it is determined that it is not necessary to divide the bar; in contrast, when the detected bass note differs in each detection zone, it is determined that it is necessary to divide the bar into a plurality of portions. In the latter case, it may be determined again whether it is necessary to divide each half of the plurality of portions further.
The second bar-division determination section 8 first specifies a chord detection range. The chord detection range is a range where chords are mainly played and is assumed, for example, to be in the range from C3 to E6 (C4 serves as the center “do”).
The power of each note in the scale for each frame interval in the chord detection range is averaged in a detection zone, such as half of a bar. The averaged power of each note in the scale is summed up for each of 12 pitch notes (C, C#, D, D#, . . . , and B), and the summed-up power is divided by the number of powers summed up to obtain the average power of each of the 12 pitch notes.
The average powers of the 12 pitch notes are obtained in the chord detection range for the first half and second half of the bar and are re-arranged in descending order of strength.
As shown in
When the three notes (this number is called “C”) or more are included (that is, all three are included), the second bar-division determination section 8 determines that the chord does not change between the first half and the second half of the bar and further determines that the division of the bar due to the degree of change in chord need not be performed.
Changing the values of “M”, “N”, and “C” used in the second bar-division determination section 8 changes how the bar is divided depending on the degree of change in the chord. In the foregoing example, where “M”, “N”, and “C” are all set to “3”, a change in the chord is rather strictly checked. When “M” is set to “3”, “N” is set to “6”, and “C” is set to “3” (which means determining whether the top three notes in the second half are all included in the top six notes in the first half), for example, it is determined that pieces of sound similar to each other to some extent have an identical chord.
A description has been given in which the first half and the second half are each further divided into two halves to have four divisions in the bar in the quadruple meter. A more correct determination suited to actual general music can be made when “M” is set to “3”, “N” is set to “3”, and “C” is set to “3” to determine whether to divide the bar into the first half and the second half and when “M” is set to “3”, “N” is set to “6”, and “C” is set to “3” to determine whether to divide each of the first half and the second half into two further halves.
The chord-name determination section 9 determines the chord name in each chord detection zone according to the bass note and the power of each note in the scale in each chord detection zone when the first bar-division determination section 7 and/or the second bar-division determination section 8 determine that it is necessary to divide the bar into several chord detection zones, or determines the chord name in the bar according to the bass note and the power of each note in the scale in the bar when the first bar-division determination section 7 and the second bar-division determination section 8 determine that it is not necessary to divide the bar into several chord detection zones.
The chord-name determination section 9 actually determines the chord name in the following way. In the present embodiment, the chord detection zone and the bass-note detection zone are the same. The average power of each note in the scale in a chord detection range, for example, in the range from C3 to A6, is calculated in the chord detection zone, the names of several top notes in average power are detected, and chord-name candidates are selected according to the names of these notes and the name of the bass note.
Since a note having a large power is not necessarily a component of the chord, several notes, such as five notes, are detected, all combinations of at least two of those notes are found, and chord-name candidates are selected according to the names of the notes in all the combinations and the name of the bass note.
Also in chord detection, notes having average powers which are not larger than a threshold may be ignored. In addition, the user may be allowed to change the chord detection range. Furthermore, instead of extracting chord-component candidates sequentially from the note having the largest average power in the scale in the chord detection range, the average power of each note in the chord detection range may be used to calculate the average power for each of 12 pitch names to extract chord-component candidates sequentially from the pitch name having the largest average power.
To extract chord-name candidates, the chord-name determination section 9 searches a chord-name data base which stores intervals from chord types (such as “m” and M7”) and the root notes of chord-component notes. Specifically, all combinations of at least two of the five detected note names are extracted; it is determined whether the intervals among these extracted notes match the intervals among chord-component notes stored in the chord-name data base, one by one; when they match, the root note is found from the name of a note included in the chord-component notes; and a chord symbol is assigned to the name of the note of the root note to determine the chord name. Since a root note or a fifth note of a chord may be omitted in a musical instrument that plays the chord, even if these types of notes are not included, the corresponding chord-name candidates are extracted. When the bass note is detected, the note name of the bass note is added to the chord names of the chord-name candidates. In other words, when a root note of a chord and the bass note have the same note name, nothing needs to be done. When they differ, a fraction chord is used.
If too many chord-name candidates are extracted in the above-described method, a restriction may be applied according to the bass note. Specifically, when the bass note is detected, if the bass-note name is not included in the root names of any chord-name candidate, the chord-name candidate is deleted.
When a plurality of chord-name candidates are extracted, the chord-name determination section 9 calculates a likelihood (how likely it is to happen) in order to select one of the plurality of chord-name candidates.
The likelihood is calculated from the average of the strengths of the powers of all chord-component notes in the chord detection range and the strength of the power of the root notes of the chord in the bass-note detection range. Specifically, when the average of the average powers of all component notes of an extracted chord-name candidate in the chord detection zone is called Lavgc and the average power of the root notes of the chord in the bass-note detection zone is called Lavgr, the likelihood is calculated as the average of these two averages as shown in the following expression 15. According to another method, the likelihood may be calculated as the ratio in (average) power between a chord tone (chord-component notes) and a non-chord tone (note other than chord-component notes) in the chord detection range.
When a plurality of notes having the same pitch name is included in the chord detection range or in the bass-note detection range, the note having the strongest average power among them is used in the chord detection range or in the bass-note detection range. Alternatively, the average power of each note in the scale may be averaged for the 12 pitch names to use the average power for each of the 12 pitch names in each of the chord detection range and the bass-note detection range.
Further, musical knowledge may be introduced into the calculation of the likelihood. For example, the power of each note in the scale is averaged in all frames; the averaged power of each note in the scale is averaged for each of the 12 pitch names to calculate the strength of each of the 12 pitch names, and the tune of the musical piece is detected from the distribution of the strength. The diatonic chord of the tune is multiplied by a prescribed constant to increase the likelihood. Or, the likelihood is reduced for a chord having a component note(s) which is outside the notes in the diatonic scale of the tune, according to the number of the notes outside the notes in the diatonic scale of the tune. Further, patterns of common chord progressions may be stored in a data base so that the likelihood for a chord candidate which is found, in comparison with the data base, to be included in the patterns of common chord progressions is increased by being multiplied by a prescribed constant.
The name of the chord having the greatest likelihood is determined to be the chord name. Chord-name candidates may be displayed together with their likelihood to allow the user to select the chord name.
In either of these cases, when the chord-name determination section 9 determines the chord name, the result is stored in a buffer 90 and is also displayed on the screen.
According to the configuration of the present embodiment described above, even persons other than professionals having special musical knowledge can detect chord names in an input musical acoustic signal in which the sounds of a plurality of musical instruments are mixed, such as those in music CDs, from the overall sound without detecting each piece of musical-note information.
Further, according to the configuration of the present embodiment, chords having the same component notes can be distinguished. Even if the performance tempo fluctuates, or even for a sound source that outputs a performance whose tempo is intentionally fluctuated, the chord name in each bar can be detected.
In particular, in the configuration of the present embodiment, since the bar is divided according to not only the bass note but also the degree of change in the chord to detect the chord, even if the bass note is identical, when the degree of change in the chord is large, the bar is divided and the chords are detected. In other words, if the chord changes in a bar with an identical bass note being maintained, for example, the correct chords can be detected. The bar can be divided in various ways according to the degree of change in the bass note and the degree of change in the chord.
A second embodiment of the present invention differs from the first embodiment in that the Euclidean distance of the power of each note in the scale is calculated to determine the degree of change in the chord to divide a bar and to detect chords.
In that case, however, if the Euclidean distance is simply calculated, it becomes large at a sudden sound increase (at the start of a musical piece or the like) and a sudden sound attenuation (at the end of a musical piece or a break), causing the risk of dividing the bar just due to magnifications of the sound even though the chord actually has no change. Therefore, before the Euclidean distance is calculated, the power of each note in the scale is normalized as shown in
The Euclidean distance of the power of each note in the scale is calculated according to the above-described expression 16. When the Euclidean distance is larger than the average of the powers of all notes in all frames, for example, the first bar-division determination section 7 determines that the bar should be divided.
To be more detailed, when the Euclidean distance is larger than “T” multiplied by the average of the powers of all the notes in all the frames, it is necessary to divide the bar. When the value “T” is changed, the bar-division threshold can be changed (adjusted) to a desired value.
The chord-name detection apparatus and the chord-name detection program according to the present invention are not limited to those described above with reference to the drawings, and can be modified in various manners within the scope of the present invention.
The chord-name detection apparatus and the chord-name detection program according to the present invention can be used in various fields, such as video editing processing for synchronizing events in a video track with beat timing in a musical track when a musical promotion video is created; audio editing processing for finding the positions of beats by beat tracking and for cutting and pasting the waveform of an acoustic signal of a musical piece; live-stage event control for controlling elements such as the color, brightness, direction and special lighting effect in synchronization with a human performance and for automatically controlling audience hand clapping time and audience cries of excitement; and computer graphics in synchronization with music.
The entire disclosure of Japanese Patent Application No. 2006-216361, filed on Aug. 9, 2006, including specification, claims, drawings and summary, is incorporated herein by reference in its entirety.
Number | Date | Country | Kind |
---|---|---|---|
2006-216361 | Aug 2006 | JP | national |