Patent Application
20080236364
The present invention relates to techniques for selectively using a plurality of pre-recorded tone data to generate a tone.
Heretofore, there have been proposed techniques for selectively outputting any one of a plurality of tone data in response to operation of an operator member (e.g., key) of an electronic musical instrument. Japanese Patent Application Laid-open Publication No. 2006-106754 (corresponding to European Patent Application Publication No. EP 1646035), for example, discloses a system where is defined in advance correspondence relationship (or mapping) between tone data of individual sections of music pieces and operator members of an electronic musical instrument. Once a user of the disclosed system operates a particular one of the operator members, tone data corresponding to the operated operator member is reproduced.
However, with the technique disclosed in the No. 2006-106754 publication, it is necessary to perform in advance a huge number of processes in order to define relationship between various inputs to the electronic musical instrument and many tone data. Particularly, because a sufficient number of tone data have to be secured in order to diversify tones to be generated in response to operation by the user, the foregoing problem would become very serious.
In view of the foregoing, it is an object of the present invention to reduce a load involved in defining relationship between inputs and relationship between various inputs and many tone data.
In order to accomplish the above-mentioned object, the present invention provides an improved tone processing apparatus, which comprises: a storage section that, for at least one music piece, stores tone data of each of a plurality of fragments segmented from the music piece and stores a first descriptor indicative of a musical character of each of the fragments in association with the fragment; a descriptor generation section that receives input data based on operation by a user and generates a second descriptor, indicative of a musical character, on the basis of the received input data; a determination section that determines similarity between the second descriptor and the first descriptor of each of the fragments; a selection section that selects the tone data of at least one of the fragments from the storage section on the basis of a result of similarity determination by the determination section; and a data generation section that, on the basis of the tone data of the fragment selected by the selection section, generates tone data to be output.
With the arrangement that a second descriptor is generated on the basis of input data based on user's operation and tone data of an appropriate fragment is selected from the storage section in accordance with the determined similarity of the first descriptor of each of the fragments to the second descriptor, the present invention can eliminate the need for defining in advance relationship (or mapping) between input data and first descriptors stored in the storage section; thus, the cumbersome advance definition (mapping) processing can be dispensed with. Besides, because tone data to be output is generated on the basis of the tone data of the thus-selected fragment, the present invention can generate tone data appropriately reflecting therein an intension of the user.
In an embodiment, the determination section calculates a similarity index value indicative of a degree of similarity, to the second descriptor, of the first descriptor of each of the fragments, and the selection section selects, from the storage section, the tone data of at least one of the fragments on the basis of the similarity index value calculated by the determination section. With such an arrangement, the similarity determination result is quantified as a similarity index value indicative of a degree of the first descriptor of each of the fragments, and thus, the selection process based on the similarity determination result can be facilitated.
In an embodiment, the descriptor generation section receives input data from a performance device that generates the input data in response to performance operation by the user. With such an arrangement, the present invention can generate tone data reflecting therein real-time performance operation by the user.
In the present invention, each “fragment” is one of a plurality of sections or segments obtained by dividing (i.e., divided from) a music piece. In a more preferred implementation, each fragment is one of segments obtained by dividing a music pieces every one or a predetermined number of beats, or one of segments obtained by dividing an interval between successive beat points (e.g., segment of a time length corresponding to a ½ or ¼ beat). Using such fragments synchronous with beat points of a music piece, the present invention can generate tones with a natural rhythm feeling.
In the present invention, specific content of the first and second descriptors may be chosen as desired, and the following are examples of forms of the first and second descriptors that may be used in the invention.
For example, the first descriptor may include a combination of information indicative of a pitch of a tone included in the fragment associated therewith and information indicative of a volume of the tone (e.g., HPCP (Harmonic Pitch Class Profile) and LPF-HPCP in
The first descriptor may include information indicative of a volume of a tone included in the fragment associated therewith. In this case, the input data includes intensity data indicative of intensity of operation, by the user, of a performance operator member of the performance device, and the descriptor generation section generates the second descriptor that includes information indicative of a volume corresponding to the intensity data. Here, a fragment with the first descriptor including a tone volume similar to a tone volume designated by the user through intensity of operation on the performance device, for example, is selected for generation of the tone data to be output.
Further, the first descriptor may include information indicative of a spectral center of gravity of a chord included in the fragment associated therewith. In this case, the descriptor generation section determines a chord on the basis of the input data, determines a spectral center of gravity of the determined chord and generates the second descriptor including information indicative of the determined spectral center of gravity of the determined chord. Here, a fragment with the first descriptor including a spectral center of gravity of a chord similar to a spectral center of gravity of a desired chord designated by the user using input data, for example, is selected for generation of the tone data to be output.
Further, the first descriptor may include information indicative of a degree of variation of a tone within the fragment associated therewith. In this case, the input data includes after-touch data indicative of a sustained operating state of a performance operator member of the performance device following operational triggering, by the user, of the performance operator member, and the descriptor generation section generates the second descriptor that includes degree-of-variation information corresponding to the after-touch data. Here, a fragment with the first descriptor including degree-of-variation information which is similar to a degree of variation in a sustained operating state following operational triggering (i.e., degree of variation of after-touch data), for example, is selected for generation of the tone data to be output.
Furthermore, the first descriptor may include information indicative of a tempo in the fragment associated therewith. In this case, the input data includes beat point data indicative of a beat point synchronous with an input motion of the user, and the descriptor generation section generates the second descriptor that includes information indicative of a tempo corresponding to the beat point indicated by the beat point data. Here, a fragment with the first descriptor including a tempo similar to a tempo designated by the user through input operation, for example, is selected for generation of the tone data to be output.
In a preferred embodiment of the present invention, the input data includes beat point data indicative of a beat point synchronous with an input motion of the user, and the data generation section includes a connecting section that time-serially arranges the tone data of the fragments, sequentially selected by the selection section, in such a manner that the tone data synchronizes with the beat point indicated by the beat point data. Because the tone data to be output is generated by time-serially arranging the tone data of each of the fragments in such a manner that the tone data synchronizes with the beat point indicated by the beat point data, the user can advantageously control a tone tempo of the tone data to be output. In a case where the tone data time length differs among the fragments, the data generation section preferably further includes a processing section that processes a respective time length of the tone data of each of the fragments, sequentially selected by the selection section, in such a manner that the tone data has a time length corresponding to a time interval between the beat points indicated by the beat point data. With this arrangement, the present invention can achieve generation of natural tones where tones of individual fragments are smoothly interconnected.
In a preferred embodiment, the first descriptor and the second descriptor each include a character value of each of a plurality of types of musical character elements, and the tone processing apparatus further comprises a setting section that sets weighting, as desired by the user, individually for each of the plurality of types of musical character elements, the determination section using a character value, weighted for each of the musical character elements, to determine similarity between the second descriptor and the first descriptor of each of the musical character elements. Because the similarity is determined after the individual character values are weighted independently of each other, the present invention can select fragments with similarity of a particular character value given priority consideration.
The present invention may be constructed and implemented not only as the apparatus invention as discussed above but also as a method invention. Also, the present invention may be arranged and implemented as a software program for execution by a processor such as a computer or DSP, as well as a storage medium storing such a software program. Further, the processor used in the present invention may comprise a dedicated processor with dedicated logic built in hardware, not to mention a computer or other general-purpose type processor capable of running a desired software program.
The following will describe embodiments of the present invention, but it should be appreciated that the present invention is not limited to the described embodiments and various modifications of the invention are possible without departing from the basic principles. The scope of the present invention is therefore to be determined solely by the appended claims.
For better understanding of the objects and other features of the present invention, its preferred embodiments will be described hereinbelow in greater detail with reference to the accompanying drawings, in which:
A. Construction of Tone Processing Apparatus:
The control device 10 is an arithmetic processing unit (CPU) that controls various sections of the tone processing apparatus 100 by executing programs. The storage device 30 stores a program to be executed by the control device 10 and various data to be used for processing by the control device 10. Any of a semiconductor storage device, magnetic storage device and optical storage device, for example, is suitably used as the storage device 30. The storage device 30 stores one or more music piece data sets of one or more music pieces.
Each music piece is segmented into a plurality of sections or segments (hereinafter referred to as “fragments”) with time points synchronous with beat points used as dividing boundaries. In the instant embodiment, the fragment is a section of a time length corresponding to one beat of a music piece. In the case of a quadruple-time music piece, for example, each segment obtained by dividing a measure of the music piece into four equal sections is set as a “fragment”. Alternatively, each segment obtained by dividing a beat of a music piece (e.g., each segment of a time length corresponding to a ½ or ¼ beat) may be set as a “fragment”. In another alternative, each segment obtained by segmenting a music piece independently of the beat (e.g., each segment of a fixed time length unrelated to the beat) or each segment of a predetermined time length from a sounding start point of a tone (e.g., high-volume tone, such as a drum tone) may be set as a “fragment”.
In the instant embodiment, the descriptor (first descriptor) P comprises respective character values of a plurality of types of character elements, including HPCP (Harmonic Pitch Class Profile), LPF (Low Pass Filter)-HPCP, volume (average energy value) of a tone within the fragment, spectral center of gravity, tempo, degree of variation, etc.
As shown in
The input device 40 shown in
The control device 10 generates output data O using a group of a plurality of fragment data DS (hereinafter referred to as “candidate data group G”) extracted in advance from a plurality of music piece data sets stored in the storage device 30. One or more conditions with which fragment data DS are selected as a candidate data group G are determined in accordance with operation performed by the user on the input device 40. For example, as the user designates a particular character value by operating the input device 40, the control device 10 selects, as a candidate data group G, a plurality of fragment data DS of which the particular character value in the descriptor P is above a predetermined value. As an example, bibliographic data, such as a genre and name of a singer, may be added in advance to each music piece data set stored in the storage device 30 so that the control device 10 can select, as a candidate data group G, only fragment data DS of a music piece matching bibliographic data designated or input by the user via the input device 40.
The detector 64 shown in
As shown in
As shown in
In the case where the input data IA is event data, the generation section 12A generates an HPCP indicative of a combination of a tone designated by a note number of the input data IA and tone volume corresponding to velocity of the input data IA. Further, the generation section 12A determines a spectral center of gravity on the basis of an average value of a plurality of note numbers (tone pitches) of the input data IA and determines a tone volume on the basis of velocity. Further, in the case where the input data IA is after-touch data, the generation section 12A determines a degree of variation on the basis of key depression pressure designated by the input data IA. For example, the generation section 12A sets a degree of variation such that the degree increases as the key depression pressure value increases. In a similar manner to the generation section 12A, the generation section 12B determines an LPF-HPCP and tone volume on the basis of the input data IB (event data) including a note number and velocity and determines a degree of variation on the basis of the input data IB (after-touch data) including key depression pressure.
The beat point data B output from the detector 64 is supplied not only to the data generation section 18 but also to the generation section 12C of the descriptor generation section 12 as input data IC. As shown in
As shown in
The calculation section 14A calculates a similarity index value RA on the basis of each character value weighted with the weighting value set by the setting section 14C. For example, if N (N=5 in the illustrated example of
R
A=α1·γ1+α2·γ2+ . . . +αN·γN (1)
where the degree of similarity γi (i is an integral number satisfying a condition “1≦i≦N”) is a numerical value indicative of a degree of similarity between an i-th character value included in the descriptors QA and QC and a character value, which is of the same type as the i-th character value, in the descriptor P. In the illustrated example of
Further, the weighting value αi in Mathematical Expression (1) above is a weighting value set by the setting section 14C for an i-th character value. As seen from Mathematical Expression (1), the similarity index value RA takes a greater value as the descriptors QA and QC and the descriptor P approximate each other more closely in character value. Also, the similarity index value RA takes a greater value as the descriptors QA and QC and the descriptor P approximate each other more closely in character value to which is assigned a great weighting value αi.
The calculation section 14B calculates a similarity index value RB in a similar manner to the aforementioned. Namely, the similarity index value RB takes a greater value as individual character values of the descriptors QA and QC and the same types of character values of the descriptor P approximate each other, and it also takes a greater value as the descriptors QA and QC and the descriptor P approximate each other more closely in character value to which is assigned a great weighting value set by the weighting section 14C.
The fragment selection section 16 of
Similarly, the selection section 16B selects, from the candidate data group G, a predetermined number of fragment data DS in decreasing order of the similarity index values RB, reads out the tone data M (MB) of each of the selected fragment data DS from the storage device 30 and sequentially outputs the read-out music piece data to the data generation section 18. Thus, the tone data MB of fragments similar in musical character to a performance executed by the user with respect to the lower-pitch section 622 of the electronic musical instrument 62 and detector 64 are selectively output to the data generation section 18. Note that the tone data M (MA, MB) may be output to the data generation section 18 either in the increasing order of the similarity index values R (RA, RB) or in arranged order in the candidate data group G.
The data generation section 18 of
Further, the processing section 181A varies the pitch of the tone data MA in accordance with pitch-bend data PB supplied from the electronic musical instrument 62. Further, the processing section 181A performs tone volume adjustment and equalization on the tone data MA and then performs a filter process on the tone data MA for cutting off frequency components lower than a predetermined frequency (that is, for example, the frequency corresponding to the pitch of the dividing boundary point between the higher- and lower-pitch sections 621 and 622 of the electronic musical instrument 62). Similarly, the processing section 181B performs time length adjustment responsive to the beat point data B and pitch adjustment responsive to the pitch-bend data PB on the tone data MB and then performs a filter process on the tone data MB for cutting off frequency components lower than the predetermined frequency.
The connecting section 183A of
In a similar manner to the connecting section 183A, the connecting section 183B generates output data OB by interconnecting the individual tone data MB processed by the processing section 181B. Adder section 183A adds together the output data OA generated by the connecting section 183A and output data OB generated by the connecting section 183B, to thereby generate output data O. Tone is reproduced by the thus-generated output data O being supplied to the output device 50.
In the instant embodiment, as explained above, fragments (fragment data DS) are selected in accordance with similarity index values R between the descriptors Q and P, and thus, there is no need to define in advance (correspondence) relationship between the input data I and the descriptors P. Further, because fragments closely related to content of a user's performance in terms of musical characters are selectively used for generation of output data O, the instant embodiment can reproduce tones appropriately reflecting therein an intension of the user's performance. Further, because the similarity index values R are calculated on the basis of character values weighted in accordance with user's instructions, the instant embodiment can generate a variety of tones preferentially reflecting therein a character value to which the user attaches particular importance.
In the instant embodiment, there are individually set the channel for generating higher-pitch-side output data OA and lower-pitch-side output data OB, so that fragment data DS of different music pieces can be selected on the higher-pitch side and lower-pitch side. In this way, the instant embodiment can generate a variety of tones with melody tones (of higher pitches) and bass tones (of lower pitches) clearly differentiated.
Tone data M are sequentially reproduced in synchronism with beat points designated by beat point data B. Thus, the user can set a desired tempo of tones by controlling a cyclic period of his or her body motion. Further, because tone data M is generated for each fragment obtained by segmenting a music piece on a beat-by-beat basis and because the tone data M is adjusted by the data generation section 18 to have a time length synchronous with the beat point designated by the beat point data B, the instant embodiment can generate tones that progress naturally with a unified rhythm feeling.
Only one or more fragment data DS, extracted as a candidate data group G from among all of the fragment data DS stored in the storage device 30, are used for generation of output data O. Thus, the above-described embodiment is advantageous in that processing loads on the control device 10 (e.g., the load involved when the similarity calculation section 14 calculates a similarity index value R and the load involved when the fragment selection section 16 selects a fragment) can be reduced as compared to the conventional construction where all of fragment data DS are processed. Further, because fragment data DS satisfying user-designated conditions are extracted as a candidate data group G, the above-described embodiment can advantageously generate tones conforming to a user's intention.
B. Modification:
The above-described invention may be modified variously, and the following paragraphs describe specific examples of modifications; note that the following modifications may be combined as necessary.
(1) Modification 1:
Character values included in the descriptors P and Q are not limited to the aforementioned. For example, the descriptors P may include pitches of tones included in corresponding fragments of a music piece, and the descriptors Q may include pitches of tones corresponding to note numbers of input data I (event data). Tone data M of a fragment including a pitch similar to that of a key operated by the user on the electronic musical instrument 62 is used for generation of output data O.
Further, a numerical value indicative of a degree of tone color complexity (hereinafter referred to simply as “degree of complexity”) of tones within a fragment may be included in the descriptor P. For example, the more the number of types of tones in the fragment, the greater value the degree of complexity takes. The descriptor Q generated on the basis of the input data I too includes such a degree of complexity. The descriptor generation section 12 calculates the degree of complexity of the descriptor Q such that the degree of complexity takes a greater value if a range of tone pitches designated by note numbers of input data I is greater (i.e., value of a difference between the highest-pitch tone and the lowest-pitch tone is greater). As seen from the foregoing, the descriptor P only has to include at least one character value indicative of a musical character of a tone within a fragment, while the descriptor Q only has to include at least one character value indicative of a musical character of a performance executed by the user.
(2) Modification 2:
Whereas the foregoing have described the construction where the time length of tone data M is adjusted to synchronize with a beat point designated by beat point data B, it is not necessarily necessary to expand or contract tone data M. For example, there may be employed an alternative construction where tone data M are arranged in synchronism with beat points or in succession (without being necessarily synchronized with beat points) regardless of excess/deficiency of the time length. Further, in the case where tone data M are expanded or contracted too, specific contents of the processing may be modified as necessary. For example, the time length (tempo) of tone data M may be adjusted in such a manner that the tone pitch too is changed with the adjusted time length (tempo), or the time length (tempo) of tone data M may be adjusted by interpolating, as necessary, a waveform of a particular tone included in tone data M. If tone data M are reproduced for a time period, shorter than the beat-to-beat interval, from a particular beat point, rhythmical tones, rich in auditory interest or attractiveness, can be generated because a silent section is set to an interval between beat points.
(3) Modification 3:
Whereas the foregoing have described the construction where a predetermined number of fragment data DS are selected in decreasing order of similarity index values R, the way of selecting fragment data DS may be modified as necessary. For example, fragment data DS of which the similarity index value R exceeds a predetermined threshold value may be selected, or only one fragment data DS of which the similarity index value R is the greatest may be selected. In another alternative, there may be employed a construction where fragment data DS that can be candidates of selection by the fragment selection section 16 are not narrowed down to a candidate data group G; namely, the similarity index value calculation and selection by the fragment selection section 16 may be performed for every fragment data DS stored in the storage device 30.
(4) Modification 4:
Whereas the foregoing have described the construction where output data O is generated in accordance with positional arrangement of individual tone data M, the way of generating output data O may be chosen as desired without being limited to the aforementioned example. For example, output data O may be generated by mixing a plurality of tone data, selected by the fragment selection section 16, at a predetermined ratio. Further, the pitch of a tone indicated by tone data M may be converted into a pitch of a note number included in input data I (IA, IB) and then used for generation of output data O.
(5) Modification 5:
Character value determined on the basis of after-touch data (key depression pressure) is not limited to one indicative of a degree of variation. For example, a time length of a fragment, corresponding to after-touch data, may be determined by the descriptor generation section 12 (12A, 12B) as a character value. More specifically, the greater the pressure designated by after-touch data, the shorter the time lengths of fragments included as character values in the descriptors Q (QA, QB). On the other hand, time lengths of individual fragments of a music piece are included in the descriptors P. With such an arrangement, a fragment of a time length corresponding to after-touch data is selected by the fragment selection section 16. Thus, if, for example, the user depresses more other keys after depression of a particular key on the electronic musical instrument 62, a fragment of a shorter time length is selected; in this way, it is possible to reproduce a music piece where frequent changes occur in tone and tune at short intervals.
Further, a numerical value functioning as an index indicative of a degree of tone color variation (e.g., amount or frequency of tone color variation) within a fragment may be included in the descriptor P so that the descriptor generation section 12 can determine, as a character value, a degree of tone color variation corresponding to after-touch data. For example, the descriptor generation section 12 may determine a degree of tone color variation such that the variation degree increases if a greater pressure is designated by after-touch data. With such an arrangement, it is possible to reproduce a music piece where greater tone color variation occurs if the user depresses more other keys after depression of a particular key on the electronic musical instrument 62 (e.g., where tone color changes frequently).
Furthermore, the number of fragments to be reproduced simultaneously (i.e., in parallel) may be controlled in accordance with after-touch data. For example, the data generation section 18 generates output data O by mixing a particular number of tone data M, corresponding to after-touch data (input data IA, IB) output from the electronic musical instrument 62, from among tone data M of a plurality of fragments selected by the fragment selection section 16. With such an arrangement, a greater number of fragments can be reproduced simultaneously if the user depresses more other keys after depression of a particular key on the electronic musical instrument 62.
Whereas the foregoing have described after-touch data indicative of intensity of key depression, an input to an assignable controller, such as a pitch-bend wheel or modulation wheel, may be used in place of the after-touch data.
(6) Modification 6:
The way of calculating the similarity index value R may be chosen as desired without being limited to the aforementioned. For example, whereas the similarity index value R has been described as increasing as similarity in character value between the descriptors P and Q increases (Mathematical Expression (1)), the similarity index value R may be a value decreasing as similarity in character value between the descriptors P and Q increases. Further, weighting by the setting section 14C may be dispensed with. For example, coordinates corresponding to the descriptors P and Q may be set in an N-dimensional space, and a distance between such coordinates (or inverse number of the distance) may be calculated as the similarity index value R.
(7) Modification 7:
Whereas the foregoing have described the construction where the control device 10 executes a software program to generate the output data O, the tone processing apparatus 100 may be implemented by hardware (electronic circuit), such as a DSP, that performs processing similar to that performed by the control device 10 of
This application is based on, and claims priority to, JP PA 2007-001058 filed on 9 Jan. 2007. The disclosure of the priority application, in its entirety, including the drawings, claims, and the specification thereof, is incorporated herein by reference.
| Number | Date | Country | Kind |
|---|---|---|---|
| 2007-001058 | Jan 2007 | JP | national |
