Patent Application
20110160887
The present invention relates to the technical field of an information generating apparatus, an information generating method and an information generating program. More specifically, it relates to the technical field of an information generating apparatus, an information generating method and an information generating program for generating a sound producing signal indicating a sound producing position used to detect a type or the like of a musical instrument playing a musical composition.
In recent years, like a so-called home server or portable audio device, there is widely used a system in which many items of musical composition data corresponding to musical compositions are electronically recorded and reproduced for enjoying music. For enjoying the music, it is desirable to rapidly retrieve a desired musical composition from among many musical compositions.
One of various retrieving methods for the retrieval is a method for retrieving a musical composition by using a musical instrument used for playing the musical composition as a keyword such as “musical composition containing piano playing” or “musical composition containing guitar playing”, for example. In order to realize the retrieving method, it is necessary to rapidly and accurately detect a type of musical instrument playing a musical composition recorded in the home server or the like.
On the other hand, for detecting the type of the musical instrument, sound producing positions of the sounds of the musical composition are detected, respectively, and the musical composition signals detected at the sound producing positions are analyzed to specify the type of the musical instrument sound-produced from the sound producing position.
The “sound producing position” refers to a timing at which one sound is produced by its musical instrument in the musical composition configured with multiple consecutive sounds on a temporal axis. Specifically, for example, it refers to, in the case of the piano, a timing at which a player's finger presses a key of the piano and accordingly a corresponding hammer hits a string so that a corresponding sound is produced, or in the case of the guitar, a timing at which a string is picked by a player's finger and accordingly a corresponding sound is produced.
There are the following conventional technique for detecting the sound producing position from a signal corresponding to a musical composition:
(1) Method for detecting a sound producing position by utilizing a temporal change in an acoustic power value of a sound of the signal (see Patent Literature 1),
(2) Method for detecting a sound producing position by utilizing a temporal change in linear predictive power value obtained by analyzing a sound of the signal by the linear predictive coding (LPC) method, or
(3) Method for detecting a sound producing position by obtaining a frequency gravity center of a sound of the signal by Fourier transform method and utilizing a change in frequency gravity center (see Non-Patent Literature 1).
The LPC method is a method for, assuming that a musical composition signal corresponding to a musical composition is an output of an articulation filter having an all-pole transfer function, modeling a spectrum density function of the musical composition signal and thereby efficiently obtaining an outline of the spectrum of the musical composition signal using the so-called linear predictive concept.
However, in the conventional technique described in the aforementioned patent literature or non-patent literature, a speed (so-called “tempo”) of a musical composition to be analyzed is not considered at all. Consequently, in the above conventional technique, there is a problem that an accuracy of detecting a sound producing position of a musical composition decreases and thus an accuracy (detection rate) of detecting a type of a musical instrument also decreases.
The present invention has been made in terms of the above problem, and one exemplary object is to provide an information generating apparatus, an information generating method and an information generating program capable of further improving an accuracy of detecting a sound producing position in a musical composition and enhancing a rate of detecting a type of a musical instrument than ever before.
In order to solve the above problem, the invention according to claim 1 relates to an information generating apparatus for generating type detection information used to detect a type of a musical instrument playing a musical composition, comprising:
a dividing unit which divides a musical composition signal corresponding to the musical composition into frame signals per preset unit time;
a power value calculating unit which performs a linear predictive analyzing processing on the divided frame signals and calculating a power value of a residual signal according to the linear predictive analyzing processing per frame signal;
a power value difference detecting unit which calculates a difference between the power value corresponding to one frame signal and the power value corresponding to the other frame signal position immediately before the one frame signal in the musical composition signal;
a threshold value calculating unit which calculates a threshold value for the difference used to detect a sound producing position of the musical instrument in the musical composition based on the calculated difference;
a sound producing position detecting unit which compares the calculated threshold value with each difference corresponding to each frame signal, and detects that the sound producing position is contained in a section of the frame signal having the larger difference than the threshold value; and
a generating unit which generates the type detection information corresponding to the section containing the sound producing position based on the detected sound producing position.
In order to solve the above problem, the invention according to claim 10 relates to an information generating method for generating type detection information used to detect a type of a musical instrument playing a musical composition, comprising:
a process of dividing a musical composition signal corresponding to the musical composition into frame signals per preset unit time;
a process, of calculating a power value, of performing a linear predictive analyzing processing on the divided frame signals and calculating a power value of a residual signal according to the linear predictive analyzing processing per frame signal;
a process, of detecting a power value difference, of calculating a difference between the power value corresponding to one frame signal and the power value corresponding to the other frame signal positioned immediately before the one frame signal in the musical composition signal;
a process, of calculating a threshold value, of calculating a threshold value for the difference used to detect a sound producing position of the musical instrument in the musical composition based on the calculated difference;
a process, of detecting a sound producing position, of comparing the calculated threshold value with each difference corresponding to each frame signal, and detecting that the sound producing position is contained in a section of the frame signal having the larger difference than the threshold value; and
a process of generating the type detection information corresponding to the section containing the sound producing position based on the detected sound producing position.
In order to solve the above problem, the invention according to claim 11 relates to an information recording medium in which an information generating program causing a computer to function as the information generating apparatus according to claim 1 is computer-readably recorded.
The best modes for carrying out the present invention will be described below with reference to
At first, a structure of a musical composition reproducing apparatus according to the embodiment will be described with reference to
As shown in
The operations will be described below.
Musical composition data corresponding to a musical composition to be subjected to a musical instrument detecting processing according to the embodiment is output from the musical DVD or the like and is output as musical composition data Sin to the single musical instrument's sound section detecting part 2 via the data input part 1.
Thereby, the single musical instrument's sound section detecting part 2 extracts, from the entire original musical composition data Sin, the musical composition data Sin belonging to a single musical instrument's sound section which is a temporal section of the musical composition data Sin which can be aurally considered as configured with either a single musical instrument's sound or single singer's voice by the method described later. Then, the extraction result is output as single musical instrument's sound data Stonal to the musical instrument detecting part D. The single musical instrument's sound section includes not only a temporal section in which a musical instrument such as piano or guitar is being played solo but also a temporal section in which the guitar is being played mainly while a drum is accompanying with small rhythm.
Additionally, the single musical instrument's sound section detecting part 2 uses a processing of analyzing the musical composition data Sin by the conventional method such as LPC method to output the analysis data Sa as the result of the analyzed musical composition data Sin to the musical instrument detecting part D. The analysis data Sa includes a residual value Slpc which is a LPC residual value calculated by the processing of analyzing the musical composition data Sin using the LPC method and single musical instrument's sound section information Sta indicating the single musical instrument's sound section described later.
Then, the musical instrument detecting part D detects a musical instrument which is playing a musical composition in the temporal section corresponding to the single musical instrument's sound data Stonal based on the single musical instrument's sound data Stonal and the analysis data Sa input from the single musical instrument's sound section detecting part 2, and generates a detection result signal Scomp indicating the detected result and outputs it to the result storing part 7.
Thereby, the result storing part 7 stores the musical instrument detection result output as the detection result signal Scomp together with the information indicating a musical composition title and a player name of the musical composition corresponding to the original musical composition data Sin in a nonvolatile manner. The information indicating the musical composition title and the player name is obtained via a network (not shown) in correspondence to the musical composition data Sin to be subjected to the musical instrument detection.
Next, the condition input part 6, which is operated by a user who desires to reproduce a musical composition, generates condition information Scon indicating a retrieval condition of a musical composition including a user-desired musical instrument name in response to the operation and outputs it to the result storing part 7.
The result storing part 7 compares a musical instrument indicated by the detection result signal Scomp per musical composition data Sin output from the musical instrument detecting part D with a musical instrument included in the condition information Scon. Thus, the result storing part 7 generates reproduction information Splay including the musical composition name and the player name of the musical composition corresponding to the detection result signal Scomp including a musical instrument matching with the musical instrument included in the condition information Scon, and outputs it to the reproducing part 8.
Finally, the reproducing part 8 displays contents of the reproduction information Splay on the display part (not shown). Thus, when a musical composition to be reproduced by the user (a musical composition including a user-desired musical instrument playing portion) is selected, the reproducing part 8 acquires the musical composition data Sin corresponding to the selected musical composition via a network (not shown) or the like and reproduces/outputs it.
Next, the operations of the musical instrument detecting part D will be described with reference to
As shown in
The sound producing position detecting part 3 detects a timing at which the musical instrument whose playing is detected as the single musical instrument's sound data Stonal produces a sound corresponding to one musical note in the musical composition corresponding to the single musical instrument's sound data Stonal, and a time for which the sound is being produced with the timing as the starting point, respectively, based on the single musical instrument's sound section information Sta and the residual value Slpc included in the analysis data Sa by the method described later. The detection result is output as a sound producing signal Smp to the characteristic amount calculating part 4.
Thus, the characteristic amount calculating part 4 calculates the acoustic characteristic amount of the single musical instrument's sound data Stonal per sound producing position indicated by the sound producing signal Smp by the conventionally-known characteristic amount calculating method, and outputs it as a characteristic amount signal St to the comparing part 5. At this time, the characteristic amount calculating method needs to correspond to a model comparing method in the comparing part 5. The characteristic amount calculating part 4 generates a characteristic amount signal St per sound (sound corresponding to one musical note) in the single musical instrument's sound data Stonal.
Then, the comparing part 5 compares the acoustic characteristic amount per sound indicated by the characteristic amount signal St with an acoustic model per musical instrument which is accumulated in the model accumulating part DB and is output as a model signal Smod to the comparing part 5.
Data corresponding to a musical instrument's sound model using, for example, HMM (Hidden Markov Model) is accumulated per musical instrument in the model accumulating part DB and is output as a model signal Smod per musical instrument's sound model to the comparing part 5.
Then, the comparing part 5 performs a processing of recognizing a musical instrument's sound per sound by using, for example, so-called Viterbi algorithm. More specifically, a log likelihood of the characteristic amount per sound relative to a musical instrument's sound model is calculated and the musical instrument's sound model whose log likelihood is maximum is assumed as a musical instrument's sound model corresponding to a musical instrument playing the sound so that the detection result signal Scomp indicating the musical instrument is output to the result storing part 7. In order to exclude a recognition result having low reliability, there may be configured such that a threshold value is set for the log likelihood and a recognition result with a log likelihood having a threshold value or less is excluded.
Next, the operations of the single musical instrument's sound section detecting part 2 will be described more specifically.
The single musical instrument's sound section detecting part 2 according to the embodiment, though detailed below, detects the single musical instrument's sound section on the principle that a so-called (single) voice generating mechanism model is applied to a musical instrument generating mechanism model.
In other words, typically, in a struck string instrument such as piano or a plucked string instrument such as guitar, when a vibration is given to a string as sound source, a sound power immediately attenuates and then ends with resonance. Consequently, in the struck string instrument or plucked string instrument, a linear predictive (LPC) residual power value calculated by the formula of residual power value=(corresponding residual value Slpc)2 using the residula value Slpc is small (the linear predictive (LPC) residual power value is simply called residual power value below).
To the contrary, when multiple musical instruments are being played at the same time, the musical instrument generating mechanism model to which the above voice generating mechanism model is applied cannot be adapted, and thus the residual power value becomes larger.
The single musical instrument's sound section detecting part 2 judges that the temporal section of the musical composition data Sin having a residual power value larger than the experimentally-preset threshold value of the residual power value is not the single musical instrument's sound section of a struck string instrument or plucked string instrument based on the magnitude of the residual power value in the musical composition data Sin, and ignores it. To the contrary, it is judged that the temporal section of the musical composition data Sin having a residual power value not exceeding the threshold value is the single musical instrument's sound section. Thus, the single musical instrument's sound section detecting part 2 extracts the musical composition data Sin belonging to the temporal section which is judged to be the single musical instrument's sound section, and outputs it as the single musical instrument's sound data Stonal to the musical instrument detecting part D.
The operations of the single musical instrument's sound section detecting part 2 described above correspond to the contents of the international application having the application number PCT/JP2007/55899 by the applicant, and more specifically the techniques described in
Along with this, the single musical instrument's sound section detecting part 2 divides the musical composition data Sin into frames each having the following preset information amount, generates the single musical instrument's sound section information Sta indicating the temporal section judged to be the single musical instrument's sound section per frame, configures the analysis data Sa together with the residual value Slpc, and outputs it to the musical instrument detecting part D.
Specifically, the single musical instrument's sound section information Sta includes start timing information indicating a start timing of a temporal section judged to be the single musical instrument's sound section, and end timing information indicating an end timing of the temporal section.
At this time, the start timing information and the end timing information indicate which samples among the samples constituting one musical composition are a start sample and an end sample of the single musical instrument's sound section.
More specifically, for example, it is assumed that in a 10-second musical composition, the start timing of the single musical instrument's sound section is three seconds from the beginning and the end timing of the section is seven seconds from the beginning. In this case, the start sample information is expressed by start sample information=fs×3 samples, where the sampling frequency in the musical composition data Sin is assumed as “fs”, while the end sample information is expressed by end sample information=fs×7 samples. The temporal section of the “fs×7−fs×3” samples is the single musical instrument's sound section, and the single musical instrument's sound section detecting part 2 divides the section into frames as described above. Thus, one single musical instrument's sound section is configured with one or multiple frames. The information amount per frame is 512 samples (11.6 msec in time) when the sampling frequency is 44.1 kHz.
Next, the detailed structure and operations of the sound producing position detecting part 3 will be described more specifically with reference to
As shown in
With the configuration, the sound producing characteristic amount calculating part 3A calculates a differential value relative to a residual power value (residual power value calculated using the residual value Slpc of the immediately previous frame) of the single musical instrument's sound data Stonal in the immediately-previous frame per residual power value corresponding to the single musical instrument's sound data Stonal corresponding to each frame based on the single musical instrument's sound section information Sta and the residual value Slpc, and outputs it as a differential value Sdiff to the threshold value judging part 3B.
Thereby, the threshold value judging part 3B compares a threshold value of the differential value Sdiff sequentially updated by the threshold value updating part 10 as described later (which is simply called threshold value below) with the differential value Sdiff, and when the differential value Sdiff is the threshold value or more, judges that the a sound producing position is present within a period corresponding to the frame corresponding to the differential value Sdiff, and assumes the frame as sound producing position candidate. Thereafter, candidate data Sp indicating the sound producing position candidate is generated and output to the sound producing position correcting part 3C.
Finally, the sound producing position correcting part 3C extracts a sound producing position candidate which is estimated to include a true sound producing position through the operation described later from the sound producing position candidates indicated by multiple items of candidate data Sp, and outputs the extracted sound producing position candidate as the sound producing signal Smp to the characteristic amount calculating part 4.
As is clear from the operations of the threshold value judging part 3B and the sound producing position correcting part 3C described above, the minimum unit in detecting a sound producing position according to the embodiment is a frame. In other words, the sound producing position detecting part 3 detects a sound producing position with one fram as minimum unit in time, and outputs the result as the sound producing signal Smp.
Then, the sound producing position detecting operation by the sound producing position detecting part 3 according to the embodiment will be described in more detail with reference to
At first, the entire sound producing position detecting operation will be described with reference to
As shown in
Then, the single musical instrument's sound section detecting part 2 subtracts the result of the linear predictive analyzing processing from the original musical composition data Sin of the corresponding frame and calculates the residual value according to the embodiment (the residual value on which the calculation of the residual power value is based) Slpc for each frame. Thereafter, the calculated residual value Slpc is temporarily stored in a memory (not shown) (step S3).
Next, the single musical instrument's sound section detecting part 2 confirms whether the operations of steps S1 to S3 have been completed for the entire segment configured of multiple frames (step S4). The concept of segment is similar to the conventional one like the concept of frame.
When an unprocessed frame for the operations of steps S1 to S3 is present within the target segment in the judgment of step S4 (step S4; NO), the processing returns to step S1 for performing the operations of steps S1 to S3 on the musical composition data Sin contained in the unprocessed frame.
On the other hand, when the operations of steps S1 to S3 have been performed on all the frames within the target segment in the judgment of step S4 (step S4; YES), the single musical instrument's sound section detecting part 2 performs an operation of detecting a single musical instrument's sound section on the musical composition data Sin within one segment by the above method (step S5), and temporarily stores the result as single musical instrument's sound section information Sta in the memory (not shown) (step S6).
Thereafter, the single musical instrument's sound section detecting part 2 confirms whether the operations of steps S1 to S6 have been performed on all the musical composition data Sin corresponding to one musical composition (step S7), and when the operations of steps S1 to S6 have not been terminated for all the data (step S7; NO), the processing returns to step S1 for performing the operations of steps S1 to S6 on the remaining musical composition data Sin.
On the other hand, when the operations of steps S1 to S6 have been performed on all the data in the judgment of step S7 (step S7; YES), the operations by the single musical instrument's sound section detecting part 2 are terminated and then the processing proceeds to the operations by the sound producing position detecting part 3 (steps S10 to S21).
In other words, at first the residual value per frame which is stored in the memory as a result of the operation of step S3 is sequentially output as the residual value Slpc to the sound producing characteristic amount detecting part 3A in the sound producing position detecting part 3. The single musical instrument's sound section information Sta per segment which is stored in the memory as a result of the operation of step S6 is also sequentially output.
Then, the sound producing characteristic amount detecting part 3A having acquired the data initially reads the single musical instrument's sound section information Sta output from the single musical instrument's sound section detecting part 2, and sets an analysis section which is the section of the musical composition data Sin for which the sound producing position is to be detected (step S10). Then, the sound producing characteristic amount detecting part 3A reads the residual value Slpc corresponding to each frame contained in the analysis section among the residual values Slpc output from the single musical instrument's sound section detecting part 2 (step S11).
A specific length of the analysis section according to the processing of step S10 is set by the preset conventional method using timing information and time information contained in the single musical instrument's sound section information Sta. In the operation of step S10, a frame to be contained in the analysis section is set. The threshold value is set to be variable as described later according to the length of the analysis section.
When the residual value Slpc corresponding to the analysis section is read (step S11), the sound producing characteristic amount detecting part 3A uses the residual values Slpc per read frames (multiple frames belonging to one analysis section) to calculate a residual power value per frame, and temporarily stores the obtained residual power value in the memory (not shown) (step S12). Then, the sound producing characteristic amount detecting part 3A calculates an average residual power value which is obtained by averaging the calculated residual power values for all the respective frames contained in one analysis section, and temporarily stores it in the memory (step S13).
Along with the processing of step S13, the sound producing characteristic amount detecting part 3A reads the residual power value per frame calculated by the operation of step S12 from the memory (not shown) (step S14), and compares the read residual power value with the average residual power value calculated by the operation of step S13 (step S15). Then, for the frame having the residual power value less than the average residual power value (step S15; NO), the sound producing characteristic amount detecting part 3A sets the residual power value for the frame at “0” (step S16), and proceeds to the operation of subsequent step S17.
To the contrary, for the frame having the residual power value equal to or more than the average residual power value in the judgment of step S15 (step S15; YES), the sound producing characteristic amount detecting part 3A calculates a differential value between the residual power value corresponding to the frame and the residual power value corresponding to a frame positioned immediately before the frame (step S17), and outputs it as the differential value Sdiff to the threshold value judging part 3B.
Next, the threshold value judging part 3B having received the value compares the threshold value sequentially updated by the threshold value updating part 10 as described later with the obtained differential value Sdiff (step S18). Then, when the differential value Sdiff is the threshold value or more at that time (step S18; YES), the threshold value judging part 3B assumes the frame corresponding to the differential value Sdiff as a sound producing position candidate, and generates candidate data Sp indicating the sound producing position candidate and outputs it to the sound producing position correcting part 3C.
Since the start sample information of the single musical instrument's sound section is previously found as described above, the sound producing time as the sound producing position candidate is calculated by adding the number corresponding to the frame number detected as the sound producing position (more specifically, “number of frame detected as the sound producing position−1” samples) to a value of a starting point with a value of the start sample as the starting point. In other words,
sound producing time as sound producing position candidate=start sample value (number)+{(number of frame detected as sound producing position−1) frames×number of samples for one frame}/sampling frequency fs is assumed.
For example, when the frames detected as the sound producing positions are the second frame and the fifth frame, assuming that the sampling frequency is 44.1 kHz, one frame has 512 samples, and further the start sample value is “1”,
the sound producing time corresponding to the second frame is expressed as the sound producing time=[1+{(2−1) frame×512}]/44100=22.6 microseconds.
In other words, the timing at which 22.6 microseconds have elapsed from the start of the single musical instrument's sound section is the sound producing time corresponding to the second frame. On the other hand, the sound producing time corresponding to the fifth frame is expressed by the sound producing time=[1+{(5−1)−frames×512}]/44100=46.4 microseconds.
In other words, the timing at which 46.4 microseconds have elapsed since the start of the single musical instrument's sound section is the sound producing time corresponding to the fifth frame.
Next, the sound producing position correcting part 3C extracts a sound producing position candidate estimated to include a true sound producing position based on the sound producing times which are the sound producing position candidates indicated by multiple items of candidate data Sp corresponding to the analysis section, and outputs the extracted sound producing position candidate as the sound producing signal Smp to the characteristic amount calculating part 4 (step S19), and proceeds to the operation of step S20 described later.
On the other hand, when the differential value Sdiff is less than the threshold value in the judgment of step S18 (step S18; NO), the frame corresponding to the differential value Sdiff is not assumed as the sound producing position candidate, and then the threshold value judging part 3B confirms whether the operations of steps S14 to S19 have been performed on all the frames contained in one analysis section set in step S10 (step S20). When the operations of steps S14 to S19 have not been terminated for all the frames (step S20; NO), the threshold value judging part 3B returns to step S14 for performing the operations of steps S14 to S19 on the remaining frames in the analysis section.
On the other hand, when the operations of steps S14 to S19 have been performed on all the frames in the judgment of step S20 (step S20; YES), the threshold value judging part 3B then confirms whether the operations of steps S10 to S20 have been performed on all the musical composition data Sin corresponding to one musical composition (step S21), and when the operations of steps S10 to S20 have not been terminated for all the data (step S21; NO), returns to step S10 for performing the operations of steps S10 to S20 on the remaining musical composition data Sin in the musical composition.
On the other hand, when the operations of steps S10 to S20 have been performed on all the musical composition data Sin in one musical composition in the judgment of step S21 (step S21; YES), the threshold value judging part 3B terminates the operations of the threshold value judging part 3B and the threshold value updating part 10.
Next, the operations of the threshold value updating part 10 according to the embodiment will be described in more detail with reference to
As shown in
Next, the threshold value updating part 10 performs the operation of reading an average residual power value obtained by the operation of step S13 in
The operation of calculating the median according to the operation of step S33 is specifically the operation of extracting a residual power value positioned on the center of the time series from the residual power values for ±N frames including the target frame.
In the operation of setting a correction value, the threshold value updating part 10 confirms whether the analysis section length is set at the preset number of frames M1 or more (step S34), and when the length is set at the M1 frames or more (step S34; YES), assumes the correction value as the value “C_High” preset when the analysis section length is set at M1 (M1>1) frames or more (step S36). On the other hand, when the analysis section length is not set at M1 frames or more in the judgment of step S34 (step S34; NO), the threshold value updating part 10 confirms whether the analysis section length is set at the number of frames M2 which is preset and is between “1” and M1 (step S35), and when the length is set at M2 frames or more (step S35; YES), assumes the correction value as the value “C_Middle” preset when the analysis section length is set between M2 frames and M1 frames (step S37). On the other hand, when the analysis section length is not set at M2 frames or more in the judgment of step S35 (step S35; NO), the threshold value updating part 10 assumes the correction value as the value “C_Low” preset when the analysis section length is set at less than M2 frames (step S38).
The threshold value updating part 10 then uses the value calculated or set by the operations of steps S32 to S38 to perform the operation of calculating a new threshold value (step S39). Thereafter, the threshold value updating part 10 gives the calculated threshold value to the operation of step S18.
The threshold value Td according to the embodiment is specifically a threshold value which is updated each time the operation by the sound producing position detecting part 3 is started for the target frame, assuming Td=δ+λ×(residual power value as median) . . . (1) (step S39).
At this time, the constant λ is a preset fixed value and λ=1 is assumed experimentally, for example.
Here, the constant λ is used for the formula (1) in order to correct an influence of a transit section from a small residual power value to a large residual power value, and an influence of a transit section from a large residual power value to a small residual power value, respectively.
Specifically, when the median is calculated in the section ranging from a frame having a small residual power value and a frame having a large residual power value, the threshold value Td increases due to the residual power value and consequently the sound producing time may not be detected (may be erroneously detected) in the frame having a small residual power value. The constant λ is used for alleviating the possibility, and thus the constant λ is reduced so that the threshold value Td can be reduced. Thereby, the possibility of erroneously detecting the sound producing time in the frame having a small residual power value can be reduced.
Further, the value δ is calculated each time by the formula (2) except for the frame having the residual power value of “0” depending on the analysis section length through steps S36 to S38:
δ=(correction value set by one of steps S36 to S38)+(residual power values corresponding to all frames in analysis section/total number of frames in analysis section) (2)
Furthermore, the length (the number of frames) of the analysis section which is the threshold value for the correction value switching (see steps S36 to S38) is experimentally preset at M1=400 (frames) and M2=300 (frames) according to the embodiment, and further the correction value to be switched is assumed as C_High=0, C_middle=0.05, and C_Low=0.1 according to the embodiment.
The length (the number of frames) of the analysis section which is the threshold value for the correction value switching is set at “M1” or “M2” as described above because the longer the analysis section length (analysis time length) is, the smaller the correction value is (see steps S34 to S38), thereby alleviating an influence on the update of the threshold value Td of the time length of the analysis frame. For the parameter N, in the embodiment, the minimum detection sound length is assumed as the time corresponding to a sixteenth note (that is, 125 msec) and thus its value is set at “5”.
At last, the sound producing position correcting operation by the sound producing position correcting part 3C (see step S18 of
As shown in
Then, the sound producing position correcting part 3C calculates a time difference between a sound producing position candidate to be corrected for a current sound producing position (which will be called current sound producing position candidate below) and an immediately-previous sound producing position candidate (which will be called previous sound producing position candidate below) among the sound producing position candidates indicated by multiple items of candidate data Sp (whose differential value Sdiff is the threshold value Td or more, of course) input from the threshold value judging part 3B (step S180). Next, the sound producing position correcting part 3C confirms whether the obtained time difference is the minimum detection sound length (indicated by numeral TTH in
Consequently, when the obtained time difference is the minimum detection sound length or more (step S181; YES), the sound producing position correcting part 3C judges that a sound producing position is included in the section of the frame corresponding to the previous sound producing position candidate, outputs the position as the sound producing signal Smp to the characteristic amount calculating part 4 (step S182, see numeral t1 in
On the other hand, when the obtained time difference is less than the minimum detection sound length in the judgment of step S181 (step S181; NO), the sound producing position correcting part 3C then retrieves a sound producing position candidate where the time difference calculated by the operation of step S180 is the minimum detection sound length or more in comparison with the previous sound producing position candidate (step S183, see numerals t1 to t4 in
When multiple sound producing position candidates can be retrieved (step S183; YES, see numerals t1 to t4 of
Next, a variant according to the present invention will be described with reference to
In the embodiment described above, the threshold value Td is calculated based on the residual power value corresponding to the frame signal, and additionally the threshold value Td can be calculated based on the differential value Sdiff between the residual power value corresponding to the immediately-previous frame and the residual power value corresponding to the target frame.
In this case, instead of the formula (1), the threshold value Td is calculated using the formulas (1)′ and (2)′:
Td=δ+λ×(differential value Sdiff as median in analysis section) (1)′
δ=(correction value set by one of steps S36 to S38)+(differential value Sdiff corresponding to all frames in analysis section/total number of frames in analysis section) (2)′
The values of “δ” and “λ” in the formula (1)′ are similar to those in the formula (1).
Next, the sound producing position detecting operation and the threshold value calculating operation according to the variant will be described in detail.
At first, for the sound producing position detecting operation according to the variant, specifically the operations of steps S1 to S7 similar to the entire sound producing position detecting operation according to the embodiment shown in
Next, the sound producing characteristic amount detecting part according to the variant uses the calculated residual power value to calculate the differential value Sdiff for all the frames contained in one analysis section, and temporarily stores it in the memory (not shown) (step S112).
Thus, the sound producing characteristic amount detecting part according to the variant calculates an average differential value obtained by averaging the calculated differential values Sdiff for all the frames contained in one analysis section (step S113).
The sound producing characteristic amount detecting part according to the variant reads the differential value Sdiff per frame calculated by the operation of step S112 from the memory (not shown) (step S114), and compares the read differential value Sdiff with the average differential value calculated by the operation of step S113 (step S115) in parallel with the processing of step S113. Then, the sound producing characteristic amount detecting part according to the variant sets the differential value Sdiff for the fram at “0” (step S116) for the frame having the differential value Sdiff less than the average residual value (step S115; NO), and proceeds to the operation of subsequent step S18.
To the contrary, for the frame with the differential value Sdiff equal to or more than the average differential value in the judgment of step S115 (step S115; YES), the sound producing characteristic amount detecting part according to the variant outputs the differential value Sdif to the threshold value judging part according to the variant as it is.
Next, the threshold value judging part according to the variant which receives the value performs the operations of steps S18 and S19 similar to the threshold value judging part 3B according to the embodiment, and then the threshold value judging part according to the variant confirms whether the operations of steps S114 to S116 as well as S18 and S19 have been performed on all the frames contained in one analysis section set in step S10 (step S117). Then, when the operations of steps S114 to S5116 as well as S18 and S19 have not been terminated for all the frames (step S117; NO), the threshold value judging part according to the variant returns to step S114 for performing the operations of steps S114 to S5116 as well as S18 and S19 for the remaining frames in the analysis section.
On the other hand, when the operations of steps S114 to S116 as well as S18 and S19 have been performed on all the frames in the judgment of step S117 (step S117; YES), the threshold value judging part according to the variant performs the operation of step S21 similar to the threshold value judging part 3B according to the embodiment, and terminates the operations of the threshold value judging part and the threshold value updating part according to the variant.
Next, for the threshold value calculating operation according to the variant, specifically the threshold value updating part according to the variant first performs the operation of step S30 similar to the threshold value calculating operation according to the embodiment shown in
Next, the threshold value updating part according to the variant performs the operation of reading the average differential value obtained in the operation of step S113 of
Here, the operation of calculating the median in step S133 is specifically an operation of extracting the differential value Sdiff positioned on the center of the time series from the differential values Sdiff for ±N frames including the target frame.
Then, the threshold value updating part according to the variant uses the value calculated or set by the operations of steps S132 and S5133 as well as S34 to S38 to perform the operation of calculating anew threshold value (step S139). Thereafter, the threshold value updating part according to the variant gives the calculated threshold value to the operation of step S18.
Here, the threshold value Td according to the variant is specifically calculated by using the formulas (1)′ and (2)′.
In the operations according to the variant described above, the formulas (1)′ and (2)′ are used so that the two operations of calculating the residual power and calculating the differential value Sdiff are not required, thereby simplifying the structure of the sound producing position detecting part.
Next, actual experimental values are exemplified in
As is clear from
As described above, through the operations of the sound producing position detecting part according to the embodiment, variant and example, the threshold value Td used for detecting a sound producing position of a musical instrument is calculated based on a differential value Sdiff of a residual power value for the liner predictive analyzing processing per frame, and compares the calculated threshold value Td with the differential value Sdiff to detect the sound producing position. This reflects the speed of the musical composition on the sound producing position detection since typically the higher the residual power value is, the faster the speed (tempo) of the musical composition is, and the lower the residual power value is, the slower the speed of the corresponding musical composition is. Thus, an accuracy of detecting the sound producing position of the musical instrument per frame is enhanced, thereby generating a sound producing signal Smp.
Therefore, the accuracy of detecting a sound producing position of a musical instrument is enhanced and consequently the rate of detecting the type of the musical instrument can be enhanced.
Since the differential value Sdiff is used to detect the sound producing position only when the differential value Sdiff is larger than its average value (see steps S15 to S18 of
Furthermore, when multiple sound producing position candidates are detected and a candidate having less than the minimum detection sound length is contained in a time interval between the sound producing position candidates, the sound producing position correcting part 3C detects that a sound producing position is contained in the section of the sound producing position candidate having the maximum differential value Sdiff among the sound producing position candidates contained in the time having the minimum detection sound length (see step S184 of
The threshold value Td is calculated based on the formula (1) and the formula (2) (or the formula (1)′ and the formula (2)′) so that the smaller the differential value Sdiff is, the smaller the threshold value Td is, and the larger the differential value Sdiff is, the larger the threshold value Td is, thereby detecting the sound producing position more accurately.
Further, the threshold value Td is calculated by using the number of frames in one analysis section given to detect the sound producing position (see steps S34 to S38 of
A program corresponding to the flowcharts shown in
| Filing Document | Filing Date | Country | Kind | 371c Date |
|---|---|---|---|---|
| PCT/JP2008/064832 | 8/20/2008 | WO | 00 | 2/22/2011 |
