Patent Grant
8399757
The invention relates to automatic analysis and performance of music.
There is a recognized need to automatically perform music. Oftentimes music composers wish to hear their newly composed music performed without gathering an ensemble of musicians for playing the music. For someone who is not a professional musician, it is difficult, and in many cases impossible, to appreciate a musical score until it is performed, and engaging well-trained players is cost prohibitive, since extensive preparation, practice and rehearsals are required to achieve a reasonably expressive performance. There is therefore a need for automatic performance of music.
Conventional music notation was developed to allow human performers to perform the music in many cases according to their musical interpretation, rather than to indicate the exact expression of each note regarding parameters such as tempo and/or intensity. Human performers take into account, consciously or not, many parameters that are not explicit in musical scores. For instance, human performers usually play a first note in a measure louder than the other notes in the measure even in the absence of any explicit indications in the score for doing so. Similarly, human performers intuitively change the length and intensity of notes in accordance with the role these notes play in the harmonic flow of the music.
State of the art systems for automatic music performance perform music from notes by playing the notes as indicated in the score, yet fail to take into account many implicit factors considered by human performers, as data describing these factors, is absent from conventional scores. As a result, automatic performance of music is characteristically monotonic and devoid of musical expression. Most notation applications allow to input music in standard music notation and translate the notation file information to MIDI format, allowing users to change manually MIDI parameters such as Tempo, Duration and Velocity (the MIDI parameter that together with Volume determines the sound intensity) of each note at the user's will. This allows in principle creating a music notation file that is not different from human performance; but in practice, the task of assigning MIDI parameters to each note is too tedious, and requires exceptional skills.
Another option provided in the art is playing music by a human performer into a music notation file (for instance, in a piano with MIDI capabilities, such as the Yamaha Disklavier™), and then “correcting” the performance by manipulating tempo and/or velocity values of some of the notes played. However, this solution does not free the user from actually performing the music in the first place.
In accordance with an aspect of some embodiments of the present invention, notes are automatically played with deviations from exact execution of the way they are written in the score; notes also are automatically assigned parameter values that are not explicit in the score; and these deviations and/or assignments are determined by the musical context in which the notes appear.
In the context of the present invention, “musical context” relates to at least one of the two following positions of a note:
(1) the position of a note in the harmonic flow of music; and
(2) the position of a note in the rhythmic flow of music.
It is generally known in the art of human music performance that notes of different positions in a harmonic flow are played differently. For instance, it is a common practice (at least in some musical cultures and styles) to slow down when a cadence begins and speed up after a cadence ends. It is also a common practice to play the resolution note more softly than the dissonant note, which precedes it.
Thus, a broad aspect of some embodiments of the present invention relates to automatically analyzing the flow of harmonic changes in music. Such automatic analysis may be very beneficial in automatic music performance, as well as in other fields. For instance, determining flow of harmonic changes in musical scores is a task that music students and teachers are involved with, in the process of understanding the structure of music. Having an automatic tool that provides a reliable analysis of such flow will help in teaching and/or learning music. Automatic determination of flow of harmonic changes is also important for automatic composition of original musical scores based on input scores. Such original scores may be variations of the original score, improvisations thereon, continuation thereof, or the like.
In accordance with an embodiment of the invention, an automatic music performer plays notes with deviations from their exact annotation, and assignments of values to note parameters that are not explicit in the annotation and these deviations and assignments depend on the position of these notes in a harmonic flow of the music.
It is also generally known in the art of music performance that notes of different position in a rhythmic flow are played differently. In accordance with an aspect of some embodiments of the present invention, an automatic music performer plays notes with performance parameters that deviate from the exact way indicated in the score, and/or assign to notes performance parameters that are not explicit in the score and these deviations and/or assignments depend on the position of these notes in a rhythmic flow of the music.
In a preferred embodiment of the invention, both the position in the harmonic flow and the position in the rhythmic flow are taken into account in automatically performing a musical score. For instance, in an exemplary embodiment, if a note is to be slowed down by 10% due to its position in the harmonic flow, and speeded up by 5% due to its position in the rhythmic flow, this note is slowed down by 5%.
In an embodiment of the invention, a user of the automatic performer may associate performance deviations and/or assignments with one or more of the above-mentioned positions of the note, and the automatic music performer identifies automatically when each deviation and/or assignments are to be applied, in accordance with harmonic and/or rhythmic analysis of the music.
One broad aspect of some embodiments of the present invention relates to assigning chords to a musical score, such that the chords will be recognized by a musician to reflect the flow of harmonic changes of the score. The musical score may be, for example, a score of monophonic music with accompaniment, or polyphonic music.
An aspect of some embodiments of the invention relates to optimizing a cost function over possible chord assignments. Optionally, the chord assignment is done by applying an optimization method.
Musical fitness is a function of the notes of the score and the assigned chords, and it has a maximal (or minimal) value when the chord assignment is such, that a musician finds that it most satisfactorily reflects the flow of harmonic changes in the score. The satisfaction level of a musician with a suggested chord assignment is referred herein as “musical correctness”.
In an exemplary embodiment, the musical fitness function is used as a cost (or value) function, where assigning each chord to a segment of the score has a cost (or value), and the assignment of minimal cost (or maximal value) is searched for. In the following description and claims we refer to cost function and value function interchangeably and refer to a minimum and a maximum by the general term “optimum”.
An aspect of some embodiments of the invention relates to providing as a musical fitness function a cost function that has a minimum at relatively high musical correctness.
According to an exemplary embodiment of the invention, the musical fitness function has at least two components: a local component and a contextual component.
The local component depends on a relation between a chord assigned to a particular segment of the score and the notes of the same segment. For example, a local component of the musical fitness function may depend on the number of notes that are common to a chord assigned to a score segment, and to the notes actually present in the same score segment.
The contextual component depends on a relation between a chord assigned to a particular segment of the score and notes of neighboring segments. For example, if there is a dissonant between a note in a score segment and the chord assigned to the same score segment, the contextual component of the musical fitness function optionally depends on the existence of resolutions of the dissonant.
Optionally, the function also has a global component that depends on the overall chord assignment. For instance, the musical fitness function may have higher value if longer sequences of time segments are assigned the same chord.
In exemplary embodiments of the invention, the score is considered to be made of time segments, each of which begins with the appearing of a first note and ends with the appearing of another note. Such a time segment is termed herein a moment.
In exemplary embodiments of the invention, chords are assigned to moments responsive to the key environment of the moments. The key environment is defined by the accidentals (sharps, flats, naturals, etc.) the notes that appear in the moment have, and the accidentals that the notes that do not appear in the moment were expected to have, considering the accidentals that appeared in the score before.
Another broad aspect of some embodiments of the invention relates to automatically performing music scores in a ‘more rhythmically musical’ manner, by providing rules, by which tempo parameters, and optionally also velocity parameters are assigned to moments and/or notes. The parameters assigned by these rules define deviations from the values explicitly indicated in the score. As a result, the timing of the moments is different from that explicitly defined in the score, and the music performance is more rhythmically correct and sounds more ‘musical’.
The rules are generally score-independent, although some users may prefer using one set of rules for one piece of music and another set of rules for another piece of music. The set of rules are referred herein as ‘pre-sets’.
The terms moment and tempo are explained below.
Tempo parameter is a parameter that defines a deviation between the tempo with which a moment is to be played according to the rule and the tempo with which the same moment is to be played without the rule (hereinafter ‘the nominal value’), for instance, the tempo assigned to it explicitly in the score, or a default Tempo (usually: a quartertone=120).
Optionally, the tempo parameter is expressed in multiples of the nominal tempo, for instance, if a nominal tempo is 100, a parameter of 1.1 indicates a tempo of 110. In this option, a parameter of 1.0 indicates no deviation from the nominal value.
In another option, the tempo parameter is expressed in MIDI tempo units to be added to the nominal tempo value. For instance, if the nominal tempo is 100, a parameter of −5 indicates a tempo of 95. In this option, a parameter of 0 indicates no deviation from the nominal value.
Optionally, the tempo parameter is expressed in the final tempo value, for instance, a tempo parameter of 110 indicates a tempo of 110. In this option, there is no parameter value that indicates 0 deviation from the nominal value. The invention may be worked with any kind of parameter that defines deviations from the nominal tempo values.
In an embodiment of the invention, if no tempo is assigned explicitly to the score, it is possible to assign a tempo to the score by the user (or by default). In this case, the tempo parameter defines a deviation (for instance, a ratio) between the tempo with which a note is to be played according to the rule and the user-defined (or default) tempo.
In an embodiment of the invention, tempo parameters are assigned to a moment responsive to rhythmic characteristics of the moment. These rhythmic characteristics may include, for instance, the time signature, the subdivision of the moment and the position of the moment in the measure (that is, if the moment begins at the beginning of the measure, some subdivision away from the beginning, at the end of the measure, etc.).
In an embodiment of the invention, the performance is automatic in the sense that tempo and/or velocity parameters that result in high ‘rhythmical correctness’ may be assigned without requiring any input from a user. However, the user may be allowed to input preferences for the automatic performance, optionally by choosing between several optional pre-sets or by changing pre-sets or creating pre-sets, optionally within some predetermined guidelines. Optionally, the user may define one set of preferences for the entire score. Alternatively, a user may define different performance preferences to different portions of the score.
Velocity parameter is a parameter that defines a deviation between the velocity with which a moment is to be played according to the rule and the velocity with which the same moment is to be played without the rule (hereinafter ‘the nominal velocity), for instance, the velocity assigned to the moment explicitly in the score, or a default velocity (usually 64).
Optionally, the velocity parameter is expressed in multiples of the nominal velocity, for instance, if a nominal velocity is 100, a parameter of 1.1 indicates a velocity of 110. In this option, a parameter of 1.0 indicates no deviation from the nominal value.
In another option, the velocity parameter is expressed in MIDI velocity units to be added to the nominal velocity. For instance, if the nominal velocity is 100, a parameter of −5 indicates a velocity of 95. In this option, a parameter of 0 indicates no deviation from the nominal value.
Optionally, the velocity parameter is expressed in the final velocity value, for instance, a parameter of 110 indicates a velocity of 110. In this option, there is no fixed value to a parameter that indicates no deviation.
The invention may be worked with any kind of parameter that defines deviations from the nominal velocity value.
In an embodiment of the invention, if no velocity is assigned explicitly, it is possible to assign a general velocity to the score by the user (or by default). In this case, the velocity parameter defines the deviation (for instance, the ratio) between the velocity with which a note is to be played according to the rule and the user-defined (or default) velocity.
An aspect of the invention relates to automatically performing musical scores, and allowing manual control over deviations of the automatic performance from tempi and/or velocities explicitly indicated in the score. Optionally, the manual control is by applying rules, by which tempo and/or velocity parameters are automatically assigned to notes and moments.
An aspect of some embodiments of the invention relates to an apparatus for receiving a first music notation file, and creating based on the received file, a second music notation file, the realization of which sounds more rhythmically correct, than that of the first. The second file includes at least tempi that are different from the tempi as marked in the score.
Optionally, the apparatus plays the music from the second music notation file. Optionally, instead of creating a second music notation file, the apparatus plays the first music notation file in a non-standard manner. For instance, the apparatus may play music from the first music notation file according to data stored in a performance instructions file, comprising instructions to play music in a manner different than is explicitly written in the first music notation file.
Thus, according to some embodiments of the present invention, there is provided a method for assigning chords to a musical score indicative of a sequence of time segments, each time segment comprising at least one note, the method comprising assigning a chord to each time segment responsive to consonance/dissonance relationships between the assigned chord and the notes of at least two of the time segments.
Preferably, each of the time segments is a moment that begins with the appearing of a note and ends with the appearing of another note.
Preferably, the consonance/dissonance relationships include existence or absence of a dissonant between a chord assigned to a time segment and a note of the time segment, and/or existence of resolution to said dissonant.
Optionally, the score comprises monophonic music with accompaniment, alternatively or additionally, the score comprises monophonic music without accompaniment, alternatively or additionally, the score comprises polyphonic music.
In an embodiment of the invention, the method comprising considering each note of the score, in determining the assigned chords.
Preferably, the assigned chords reflect the flow of harmonic changes in the score.
In an exemplary embodiment of the invention, the method comprising:
Preferably, (b) comprises selecting one of the trial chords for a first time segment, responsive to consonance/dissonance relationships between the trial chord and notes of one or more time segments neighboring the first time segment. Optionally, at least one of the neighboring time segments appears in the score later than the first time segment.
In an exemplary embodiment, selecting and/or assigning comprises evaluating a function, the value of which is responsive to dissonance/consonance relationships between chords assigned to time segments and notes of time segments in the score, the function being a cost function or a value function.
In an exemplary embodiment of the invention, the method includes a chord determination procedure, wherein a trial chord is determined to each moment, responsive to consonance/dissonance relationships between the determined chord and the notes present in the moment. Optionally, the method further comprises a chord combination procedure, wherein at least one of the trial chords determined in the chord determination procedure is changed responsive to notes of an adjacent moment, thereby increasing the value function. Additionally or alternatively, the method includes a chord reduction procedure, wherein at least one of the chords assigned in a preceding procedure is changed to reduce the number of chord changes along the score and increase the value function.
Optionally, each suggested trial chord has a root note in the time segment to which it is suggested. Optionally an accidental of a note in a trial chord is the same as the accidental of the same note in its last appearance in the score. Alternatively, the accidental assigned to a virtual note, not appearing in the moment to which the trial chord is assigned, is the accidental the virtual note had in a list of scales assigned to the nearest previous moment where there was no accidental conflict for this note. Optionally, the list of scales includes scales wherein the accidentals of the notes that exist in the moment, to which the list is assigned, are the same as those that appear in said moment.
In an embodiment of the invention, the value of the function used for evaluating musical correctness of a certain chord assignment is responsive to the number of notes common to the trial chord and to the moment. Optionally, the value of the function is responsive to existence in the moment of a note that is a dissonant 7th of the trial chord. Optionally, existence in a moment of a dissonant 7th of a chord brings the function closer to its optimum. Optionally, the value of the function is responsive to a resolution of a dissonant. Optionally, the dissonant is a 7th. Optionally, the value of the function is responsive to a quality of the resolution.
In an embodiment of the invention, the value of the function is responsive to a distance between a dissonant and its resolution. In an embodiment of the invention, the value of the function is responsive to the number of times that the chord's root note appears in the moment. In an embodiment of the invention value of the function is responsive to the presence of a bass dissonant over the 7th, optionally, the value of the function is also responsive to a resolution of a bass dissonant over the 7th.
In an embodiment of the invention, the function is responsive to the quality of the chord, the position of the chord, presence of sustained notes in the bass, and/or presence of sustained notes in parts other than the bass.
An aspect of some embodiments of the invention provides a method for assigning chords to a musical score indicative of a sequence of moments, each moment beginning with an appearance of a note and ending with the appearance of another note, the method comprising assigning a chord to each moment; and amending the chord assignment to reduce the number of changes of chords along the score. Preferably, amending the chord assignment is to improve musical correctness of the chord assignment.
In an embodiment of the invention, the method includes assigning to each moment a chord responsive to the notes of the moment and moments following it and amending the chord assignment responsive to the notes of the moment, notes of moments following it and notes of moments preceding it.
In an embodiment of the invention, the assigning of a chord to a moment is responsive to predefined dissonance/consonance relationships between the chord assigned to the moment and notes of the moment and of following moments.
Optionally, amending the chord assignment comprises combining moments containing only single notes with a preceding moment responsive to a predefined set of parameters for each moment in a series of moments.
Optionally, the reduction of the number of chords in a sequence is responsive to a predefined set of parameters characterizing each chord in a series of chords.
Optionally, chords are reduced based on their position in a series of chords. There is also provided, in accordance with an exemplary embodiment of the invention, a method of assigning tempo parameters to moments in a musical score, the method comprising:
Optionally, the at least one rhythmical characteristic comprises a position of a moment in a measure which comprises said moment.
Optionally, said at least one rhythmical characteristic comprises a smallest subdivision in a measure comprising said moment.
Optionally, according to the rules, a moment appearing last in a measure is always assigned a tempo slower than the nominal tempo.
Optionally, according to said rules of said first set of rules, a first moment, appearing last in a measure is never assigned a faster tempo than that assigned to a second moment, appearing first in the following measure, unless said first moment is assigned a tempo that is slower than a moment preceding it, in which case said second moment may be slower.
Optionally, according to said rules of said first set of rules a moment of a weaker beat is never assigned a tempo faster than that of a stronger beat following said weaker beat, unless said weaker beat is also slower than a stronger beat that precedes said weaker beat.
Optionally, according to said rules of said first set of rules a weighted average over tempo assigned to moments in a measure of the score, is essentially the same for all measures, the average being weighted by a nominal length of each moment.
Optionally, the method further comprising:
There is also provided in accordance with an exemplary embodiment of the invention a method of assigning velocity parameters to moments in a musical score, the method comprising:
providing a second set of rules for assigning a velocity parameter to a note, a rule of said second set of rules being responsive to at least one rhythmical characteristic of said note,
extracting from the musical score said at least one rhythmical characteristic of at least one note; and
assigning to said at least one note a velocity parameter according to said second set of rules.
Optionally, at least one rule of said second set of rules is responsive to a time signature of the musical score.
Optionally, said at least one rhythmical characteristic, to which rules of said second set of rules are responsive, comprises a position of the note in a measure comprising the note.
Optionally, said at least one rhythmical characteristic, to which rules of said second set of rules are responsive, comprises a smallest subdivision in a measure comprising the note.
Optionally, according to said second set of rules a note of a weaker beat is never assigned a velocity value higher than that of a stronger beat following it.
There is also provided in accordance with an exemplary embodiment of the invention, an apparatus for automatically performing music from a music notation file, the apparatus comprising an input for receiving a music notation file, a processor for processing the input music notation file, and an output for playing the processed music notation file; wherein the processor is configured to assign to moments tempo parameters according to at least one embodiment of the invention.
Optionally, the processor is configured to assign to notes velocity parameters according to an embodiment of the invention.
In accordance with an exemplary embodiment of the invention, there is provided an apparatus for automatically writing a computer readable music file, the apparatus comprising an input for receiving a music notation file, a processor for processing the input music notation file into a computer readable music file, and an output for writing the computer readable music notation file; wherein the processor is configured to assign to moments tempo parameters according to an embodiment of the invention.
Optionally, a processor according to an embodiment of the invention is configured to assign to notes velocity parameters according to embodiments of the invention.
There is also provided in accordance with an exemplary embodiment of the invention, an apparatus for automatically performing music or writing a computer readable music file, the apparatus comprising:
Optionally, the processor is configured to assign velocity parameters to notes, in a method comprising extracting from the score at least one rhythmical characteristic of at least one note, and assigning to the at least one note a velocity parameter according to a rule of a second set of rules, said rule being responsive to the at least one rhythmical characteristic, and the user interface allows affecting the second set of rules by a user.
Optionally, affecting a set of rules comprises selecting a set of rules out of at least two sets of rules.
Optionally, at least one rule of said first set of rules is provided to the processor dependent on at least one variable, and affecting said at least one rule comprises determining the value of said at least one variable.
Optionally, said variable has a continuous value.
Optionally, the apparatus comprises an editor for editing said set of rules.
Non-limiting examples of embodiments of the present invention are described below with reference to figures attached hereto and listed below. The figures use standard music score notation; however, the invention may be implemented whatever notation is used for indicating the appearance and disappearance of sounds along the music time-line, for example MIDI notation, or frequency notation.
Under this heading, several terms that are used in the descriptions that follow are explained. Some of these terms are generally used in the field, and some were coined to communicate embodiments of the present invention.
Notations for Notes
In the following, we relate to notes using two notation systems, namely the common notation and the TCD-HT notation. These two notations are explained herein in relation to
The keyboard portion (2000) includes 14 white keys, marked with letters A-G (starting with C), which are common names for the notes played by pressing these keys.
The keyboard portion (2000) also includes 10 black keys, marked 2010, 2020 . . . 2100. Each of these keys may have two common names, and an explanation as to which name is used in each case is not required for understanding the present application. Each of the black keys 2010-2100 is commonly named by a white key adjacent to it, and an “accidental”, denoting if the black key is lower or higher than the adjacent white key. For instance, the black key 2010 may be denoted as C sharp or as D flat (with “sharp” and “flat” being the accidentals). The key 2020 may be denoted as D sharp or as E flat, etc. For completeness it should be noted that the note F may also be denoted as E sharp or G double-flat, etc.
In the TCD-HT notation, each key is denoted by two numbers: its displacement from a tone center (TCD stands for Tone Center Displacement) and its distance from the tone center, expressed in half-tones (HT stands for Half Tone). For example, if the tone center is at the C denoted as 2200, this C is denoted {0, 0}, and the key 2060 is denoted either {0, 1}, in equivalence to C sharp, or {1, 1} in equivalence to D flat. Another example: if the tone center is on the G key denoted as 2150, the key 2200 is denoted as {3, 5}.
In a score written in common notation default accidentals are defined by the key signature, which includes the sharps or flats placed after a clef to indicate the key. However, this indication is not perfect, as each major key has a corresponding minor key with the same flats and sharps, (for instance, when no sharps or flats are present, the key may be C major or A minor). In most notation software there is a requirement to mention a key, not only sharps and flats, and in the present description, a key signature refers to the strict mention of key, which implies the default flats and sharps. Accidentals that differ from the default are written in common notation next to the note, to which they refer.
Chord Notation
Generally, a chord is any combination of two or more notes, played at the same time. In the following, we shall concentrate on chords that have exactly three or four notes, referred to as triads and tetras, respectively. A triad has a root, which is the lowest note in the triad, a third, displaced two notes from the root, and a fifth, displaced four notes from the root. A tetra has a root, third, fifth, and seventh. Accordingly, in the TCD-HT notation a triad may be denoted as {0, 0}, {2, X}, {4, Y}, and a tetra as {0, 0}, {2, X}, {4, Y}, {6, Z}. Hence, if the root is at C natural, the third is E and the fifth is G and the tetra also includes B. Each of the E, G, and B that appear in the triad and tetra having the root in C natural may have five different accidentals, (natural, sharp, flat, double sharp and double flat). Out of the 25 triad combinations, the 9 triads summarized in Table I below are most useful. The right column in the table is the common name of the triads, which has two parts: the root note (C in the example) and the quality of the chord.
Chord Stability
In the present description and claims, a chord is considered stable if it passes the threshold of stability test, [or: stability threshold test] which happens in three cases:
When two notes are played together they may be considered to be dissonant or consonant. Consonants are frequently described by a wide variety of terms, such as: pleasant, euphonious, beautiful, smooth, fused, pure, diffuse, and relaxed. Dissonants, on the other hand, are frequently described as unpleasant, ugly, rough or tense. The question if a specific note combination is to be considered as a consonant or a dissonant may change with generations, culture, or musical style. For instance, many note combinations that were considered dissonants in the past are considered consonants in contemporary music.
In the description below, a note that is not part of the triad is said to be dissonant to that triad, and a note that is part of the triad is said to be consonant to the triad. Nevertheless, the invention is not limited to this definition of dissonance/consonance relationship.
According to an embodiment of the invention, it is preferred to assign chords that have minimal number of dissonants with the notes of the time segments of a score, and if there is no single chord with minimal number of dissonants, to assign chords, the dissonants of which find the quickest and/or best resolution.
The dissonant combinations of notes and their “resolutions”, as these terms are in ordinary use in Western music are listed in table II below, using the TCD-HT notation. The term “resolution” of a dissonant is used herein in the regular sense it has in the art, and this is sometimes explained as a note that eases the uneasy feeling that the dissonant caused.
Quality of Dissonant Seventh Resolution
In some embodiments of the invention, resolutions of 7th dissonant are defined as being of high or low quality as follows:
If the dissonant is a major seventh, which in TCD-HT notation format is ({0,0}, {6,11}), and the {6,11} is replaced with either a {7,12} or a {5,9}, the resolution is of high quality.
If the dissonant is a minor seventh ({0,0}, {6,10}) and the {6,10} is replaced with a {5,9} or with a {5,8}, the resolution if of high quality.
If the dissonant is a minor seventh ({0,0}, {6,10}) and the {6,10} is replaced with a {7,12}, the resolution is of low quality.
If the dissonant is a diminished seventh ({0,0}, {6,9}), there is only one (high quality) resolution, which is {5,8}.
All other possibilities are not considered a resolution.
Chord Relationships
In music theory it is common to define Dominant-Tonic relationships (and Tonic-Dominant relationships) between chords. Herein, we refer to these relationships as strict, and define broader relationships, referred herein as quasi Dominant-Tonic and quasi Tonic-Dominant. All these relationships are defined below:
A strict Dominant-Tonic relationship exists between two chords in each one of the following two cases:
a) there is a root relationship of {0,0}, {3,5} between the chords, the chord built on {0,0} is major and the chord built on {3,5} is either major on minor (for instance: C major-F major or C major-F minor);
b) there is a root relationship of {6,11}, {0,0} between the chords, the chord built on {6,11} is diminished and the chord built on {0,0} is either major on minor (for instance, B diminished-C major or B diminished-C minor).
A quasi Dominant-Tonic relationship exists between two chords in each one of the following cases:
a) there is a root TCD relationship of 0, 3 (the quality of the two chords doesn't matter);
b) there is a root TCD relationship of {0,0}, {1,1} in which the chord built on {0,0} is diminished or diminished-diminished, and the quality of the other chord doesn't matter.
A strict Tonic-Dominant relationship exists if there is a root relationship of {0,0}, {4,7} between the chords, the chord built on {0,0} is either major or minor and the chord built on {4,7} is major.
A Quasi Tonic-Dominant relationship exists only between chords in which the root TCD relationship is 0, 4, regardless of the quality of either chord.
Virtual Notes
To assign chords according to some embodiments of the invention, it is sometimes required to consider notes that are not present in the time segment to which a chord is being assigned. These notes are referred to herein as virtual notes. For instance, according to an exemplary embodiment of the invention, each time segment of the score is assigned a triad having a root in one of the notes that appear in the same time-segment. The triad notes that appear in the time segment are referred to as existing notes, and these that do not appear in the time-segment (if any) are referred to as virtual notes.
As described above, each root note may be associated with 9 different triads, depending on the accidentals of the third and the fifth. In one exemplary method, the accidentals of existing notes are those that appear in the moment, and the virtual notes are assigned accidentals based on accidentals that appeared in earlier time segments in the score, as described below.
It should be noted, that unless otherwise indicated in the score, accidentals of existing notes are determined in accordance with the key signature of the score.
Timing Markings
Timing markings are all the marks used to indicate when each note begins and when it ends. Some examples to such markings are subdivision (1/4, 1/8, etc.), tempo, and articulation markings such as ‘staccato’ ‘tenuto’, etc.
Subdivisions
Subdivisions indicate nominal ratios between the lengths of notes, for instance, a note written in a subdivision of 1/8 occupies half the time segment of a note written in a subdivision of 1/4. Human performers use subdivisions as general guidelines rather than as indications for mathematical relations between the lengths of different notes. Playing notes precisely according to their mathematical subdivision sounds musically ‘erroneous’. The invention concerns automatically deviating from the nominal ratios indicated by the subdivision in a way that makes the performance sound more rhythmically correct.
Tempo
Tempo (plural: tempi) is a measure of the swiftness by which a musical score or portion thereof is to be performed. The tempo is sometimes indicated in scores by means of general instructions, such as “fast”, “slow”, “lively”, etc. and sometimes by precise metronome indication, for instance: =60, which means that 60 1/8s notes are to be played per minute, or that each
note occupies a time segment of 1 second. In conventional music notation a single tempo is usually indicated for an entire score or a large portion thereof. In the context of the present application tempo is assigned to each moment, and thus, together with the general tempo indication and the note-length, determines the period of time from the beginning of the moment to the beginning of the next moment. For instance, if the score includes an explicit tempo indication of
=60 and a certain 1/8 note in the score is assigned according to the invention a tempo parameter of 120%, this 1/8 will be played for 1/(60*120%) minutes=1/72 minutes.
Time Signature
Time signature is the number of times the listed subdivision appears in the measure, for example, in 4/4, there are 4 quarter-notes in each measure.
Time signature unit—a subdivision of a length 1/X, with X being the denominator in the time signature.
Beat
Beat is used in the present application to indicate the degree to which a moment is accented in relation to another moment within a measure. Optionally, moments are grouped only in pairs or triplets. For example, if 4 quarter-notes are present in a measure in 4/4, the 1st and 3rd beats are more accented than the 2nd and 4th, thus the relation between the 1st and 2nd beat is referred to as strong/weak and the relationship between the 2nd and 3rd beats is weak/strong. Likewise, the 1st beat is stronger than the 3rd, and the 2nd stronger than the 4th. In a triple meter, (3/4, for example), the 1st beat is the strongest, the 2nd is weaker than the 1st and the 3rd is weaker than the 2nd. Thus, beats may be graded ‘higher’ or ‘lower’.
Automatic Analysis of Harmonic Changes
Overview
A musical score is a representation of appearing (and disappearing) of sounds along a time-line.
In an exemplary embodiment of the invention, the music timeline is first segmented into moments, infrastructure for finding accidentals of virtual notes is prepared, and then, chords are assigned to segments as described in
The procedure “chord combination” (204) is made to reduce the number of chord changes along the music timeline. It relates to two consecutive time segments, and optionally defines a common chord to both (referred to as a ‘merged chord’). Optionally, the common chord is determined using the chord determination procedure.
Optionally, in the chord combination stage (204) time segments are not combined if the chords assigned to them have strict or quasi Tonic-Dominant or Dominant-Tonic relationship. The detailed rules applied in the “chord combination” procedure according to a preferred embodiment are detailed below.
Optionally, the results of the “chord combination” procedure (204) are the final results of the method, as they are satisfactory for many musical scores. Alternatively, a reduction stage takes place.
In the reduction stage (206), the possibility to combine groups of time segments under a common chord is considered. The possibility to combine is usually turned off if it results in the elimination of a chord that has a strict Tonic-Dominant or Dominant-Tonic relationship with the chord of the following time segment. The detailed rules for chord reduction according to a preferred embodiment are shown in the look-up-table annexed to U.S. provisional patent application No. 60/821,744, incorporated herein by reference, as annex II.
Optionally, the chord reduction procedure is run more than once. In an exemplary embodiment it is run once and again until it does not change the chord assignment any more.
Results of chord combination and reduction are presented in
The method used for the analysis shown in
Timeline Segmentation
In an exemplary embodiment, the musical score is treated as being made of time segments, each of which begins with the appearing of a first note and ends with the appearing of another note, whether or not the first note disappeared. This kind of segment is referred herein as a moment. Some examples to the use of this term are provided in
As a result of the analysis, it may be found that several sequential moments are assigned a common chord, and a time segment consisting of such a sequence of moments is referred herein as a “super moment”. Two such examples are provided in
Determining Accidentals of Virtual Notes
According to a preferred embodiment of the present invention, to decide on an accidental of a virtual note, infrastructure for making the decision is prepared as shown in
Determining the Allowed Scales
There are many ways to decide on which scales are allowable, and each may be valid for a certain kind of music. For tonal music, it is a good option to include all the major and minor scales in the allowed scales, and in the minor scales to use the so-called “harmonic” scale, which contains a “raised” 7th note (or “leading tone”) instead of the “natural” 7th note. A complete list of the allowed scales in this case is provided in
Compiling the Actual Lists of Scales
An actual list of scales is associated with each moment as shown in the flow chart of
If none of the allowed scales match all the notes with their accidentals present in the moment (1604), actual lists are prepared for replacement moments, each having all the notes of the moment absent one (1606), and the actual list of the moment is the combination of the actual list of the replacement moments (1608). If all the replacement moments don't have a matching scale, the process is repeated (1610), this time omitting two of the notes in each of the replacement moment. Further repetitions may be carried out as required. A matching scale cannot be found when a moment includes the same note twice, each time with a different accidental (conflicting notes) or, more generally, when the moment includes notes that do not appear together in any of the allowed scales, for instance D sharp and F natural. Determining the existing lists of scales
To determine an existing list of scales, the actual list associated with a particular moment (number M in the score, see
For the first moment, the key signature is considered to be the existing list of the moment preceding it (1706).
Determining an Accidental to a Virtual Note
As described in
Thus, if the existing list of the (M-1)th moment has an accidental conflict for a Virtual Note (N) of the Mth moment (1804), the accidental of that note will be the one it has in the existing list of the (M-2)th moment (1806), unless this list also has a conflict for the note N, in which case the existing list of moment M-3 is checked, and so on.
Chord Determination (202 in the Flowchart of
In this procedure, a chord is assigned to each moment. The assigned chord is referred below as a moment's chord.
In the embodiment shown in
If all the factors were considered and no chord had a highest score, one of the higher scoring chords is assigned randomly. If the different terms are computed as described below, it is doubtful if such a case is possible theoretically, and if it is, it is extremely rare in practice.
The hierarchical arrangement of factors of the embodiment of
Other embodiments of the invention may take into consideration only some of the factors considered below. Alternatively or additionally, other factors may be considered. Optionally, the factors considered below may be weighted in a different manner.
Optionally, after a chord is assigned to a moment in the chord determination procedure, a chord is assigned to the next moment by the same procedure. Alternatively, after a chord is assigned to a moment in the chord determination procedure, the chord combination procedure is applied on the newly assigned chord and the chord of the preceding moment, and only then a chord is assigned to the next moment by a chord determination procedure.
Computing the Various Terms Mi
Optionally, the terms Mi are computed for all the allowed chords. Preferably, they are computed only for a limited number of trial chords. It is useful to compute it only for chords having their root in the moment. Thus, in a preferred embodiment of the invention, a ‘perfect’ triad (or tetra) is built on each one of the notes in the moment to provide trial chords. A ‘perfect’ triad or tetra is one that its virtual notes have accidentals that fit the key environment, for instance, in accordance with the procedure described above.
Computing M1
The term M1, named “note count”, is based on the number of notes the trial triad has in common with the notes of the moment, and in some cases is also responsive to the existence of seventh dissonants, and the existence and quality of their resolutions.
The note count term is optionally computed in two stages:
(1) first, M1 value of each trial chord is set to the number of notes common to the moment and to the trial chord. If one of the trial chords has a higher M1 value than all the other trial chords, this chord is assigned to the moment and the chord determination procedure is concluded for this moment.
(2) If further computation stages are needed, the value of M1 is increased according to the following rules:
As an example of a case in which the first stage suffices: if a moment appears in a key environment with no sharps or flats, and the notes of the moment are C, E, and G, the trial chords are C-E-G, E-G-B, and G-B-D, and their note count values (as computed in stage 1) are 3, 2, and 1 respectively. Therefore, the trial chord C-E-G is assigned to the moment.
As an example of a case in which the second stage is required: if a moment appears in an environment with no sharps or flats, and the notes of the moment are C, E, B and D, at the end of stage 1 the note count value is 2 for the chords built on C, E and B and the final note count values are decided by considering the dissonant 7th. Since the seventh is found for two chords: the one built on C and the one built on E, and both contain 2 triad members, the note-count values of the chords built on C and E are raised according to the quality of the resolution of the seventh, which determines the final value of the note count term. If either seventh has a higher-quality resolution, the chord with that higher-quality resolution is assigned to the moment. If neither seventh has a resolution quality advantage, or if both sevenths have no resolution at all, the method continues to step 310 and evaluation of the next term.
Computing M2
The term M2 is named “dissonant score”, and is higher as the number of unresolved dissonants decreases.
Optionally, in computing dissonant score, only dissonant notes other than the 7th are considered.
Optionally, the value of M2 is as follows:
M2=4, in absence of dissonant notes;
M2=3, in the presence of dissonant notes, all of which having resolutions;
M2=2, in the presence of one dissonant note without resolution;
M2=1, in the presence of two dissonant notes without resolution; and
M2=0, in the presence of three or more dissonant notes without resolution.
Resolutions are looked for within the following few moments so long as the dissonant note is present in these following moments and/or the following moments do not contain more than one additional dissonant note.
Computing M3
The term M3 is named “dissonant resolution distance”, and it is increases as the distance (in moments) between a dissonant note (including the 7th) and its resolution is reduced.
Computing M4
The term M4 is named “root in bass”, and is the number of repetitions of the root note in the moment. If the root is not in the bass, M4=0.
Computing M5
The term M5 is named “bass dissonant over the 7th” and optionally has the following values:
M5=2, if the moment has no bass over the trial chord's 7th;
M5=1, if the moment has a bass over the trial chord 7th that has a resolution; and
M5=0, if the moment has a bass over the trial chord 7th, and this bass has no resolution.
A moment is considered to have a bass over the 7th of a trial chord if the lowest note in the moment is not a member of the perfect tetra of the trial chord.
Computing M6
The term M6 is named “chord quality” and its value is as follows:
M6=0 if the third and/or the fifth are not present in the moment; otherwise:
M6=2, if the chord is a major chord;
M6=1, if the chord is a minor chord; and
M6=0, in all other cases.
Examples where the assigned score of the first moment is decided according to M6 are shown in
In the moment shown in
Computing M7
The term M7 is named “chord position” and it optionally has the following values:
M7=3, if the root is in the bass (that is, the lowest note in the moment is the root of the trial chord);
M7=2, if the third is in the bass;
M7=1, if the fifth is in the bass; and
M7=0, in all other cases.
Terms M1 to M7 are sufficient to decide a chord in almost any given situation. In some very rare cases, not yet encountered by the inventors, the following terms may be used.
Computing M8
The term M8, named “sustained notes in the bass” may be computed to prefer chords having sustained notes in the bass in at least three consecutive moments in which there is at least one new note in each moment. Optionally, the values of Mg are as follows:
M8=1, if there is a sustained note in the bass; and
M8=0, otherwise.
Computing M9
The term M9, called “sustained notes in parts other than the bass” may be computed to prefer chords having sustained notes over those that don't have them at all. Optionally, M9 is equal to the number of sustained notes that exist.
The above-described chord determination procedure finds a best-fitting chord for each moment; however, the chord assignment of highest musical correctness usually has less frequent chord changes. Therefore, musical correctness may be improved by merging moments into super-moments, to which a single chord is assigned. In an exemplary embodiment, this is done in two stages: first, it is checked whether each moment may be properly merged with the preceding one; then, it is checked if groups of moments (some of them already merged) may be further merged to further increase the musical correctness. The first stage is termed herein “chord combination” and the second “chord reduction”. In a preferred embodiment, the chord combination procedure is executed only once (between adjacent moments), and the chord reduction is executed once and again until the chord assignment doesn't change with additional rounds of reduction.
The chord determination and the chord reduction procedures are described in detail below.
Chord Combination
In the above-described chord determination procedure, each moment is assigned a chord responsive to the notes of the moment and optionally responsive to notes of following moments. In the chord combination procedure, the chord assigned to a moment is optionally changed to better fit to the chord(s) assigned to the preceding moment. There are two possible results for this procedure:
(1) The chord assignment is left unchanged
(2) The moments are combined to provide a super-moment, and a single chord is assigned to the super moment.
As there is no significance to the question whether a particular moment is a super-moment or not, all ‘moments’ and ‘super-moments’ will be referred to below as ‘moments’, unless a super-moment characteristic or single-moment characteristic needs to be emphasized.
In
In all other cases, the notes of the two consecutive moments are merged (410) to obtain a “merged moment”, and the chord that best fits the merged moment is determined in accordance with the chord determination procedure, to define a merged chord (412).
In
Otherwise, it should be decided if each one of the chords assigned to the two moments should be left unchanged, replaced with the chord assigned to the first moment, or replaced with the merged chord.
If the two chords are the same (416) and the second moment has only one new note (e.g. that was not present in the first moment), the two moments are combined, and the chords of both moments are replaced with the merged chord.
If the merged chord is the same as the chord assigned to the first moment (418), and there is more than one new note in the second moment, there are two conditions (420 and 422), under both of which the moments are combined and the second chord is replaced with the first one. In all other cases chord assignment is left unchanged. The conditions are:
The Special Case of Moments Having a Single Note
In an embodiment of the invention, if the second moment has only one note, the decision whether or not to combine it with the first, is taken by somewhat different rules, summarized in a look-up-table (500) such as that appearing in
Another difference between the table and the general rules is that the combined super-moment is always assigned the chord that was assigned to the first moment by the chord determination procedure.
Some examples of lines that are part of the single-note LUT in use by the inventors are shown in
The full table, which is currently used by the inventors, is annexed to U.S. provisional patent application No. 60/821,744, incorporated herein by reference, as Annex I.
The Notation Used in the LUT of
Action Column and Numbering of Moments
The action that the LUT allows is combining single notes into the preceding, non-single-note moment. The serial numbers of the single notes to be combined into the first, non-single-note, moment are listed as indexes of the letter C (Combine) in the action column (501) in the LUT. The combination means that the moments of all the combined notes together with the first moment, become part of one super-moment. The first, non-single-note, moment is numbered 1, therefore, the indexes of the single notes that may be combined are always larger than 1. The LUT considers up to 4 single notes at a time; therefore, the largest index may be 5.
If the action is C24, for instance, the second and fourth moments (which are the first and third single notes) are combined into the first moment, and the third moment (which is the second single note) is disregarded.
If the score contains a sequence of up to 4 single notes, the LUT is used once for this sequence. If the score contains a sequence with 5 or more single notes, the LUT is used again, this time, the 5th single note is considered as if it was the first, and the moment, into which single notes have been combined in the previous use of the table, is considered the first, non-single-note, moment.
If the last single note in a sequence is disregarded by the action column, and the following moment contains more than one note, the chord assignment of that last single note remains unchanged, and it is not disregarded.
In the chord combination stage, single notes are combined only with a preceding moment. Combining single notes with a following moment may occur in the chord reduction stage, described below.
Headings of Columns in the Table
The factors considered while deciding whether to combine single notes into a preceding moment are listed in the headings of the columns of the table of
The factors are as follows:
Characteristics of the First Chord:
Characteristics of the Second Chord
The column headed “Pattern” does not contain any additional information, but is a summary of the columns headed “TCD” and “HT” in the second to fifth chord columns.
Application Examples of the Single-Note LUT
Chord Reduction
In the above-described chord combination procedure, chords that were assigned to moments in the chord determination procedure are optionally changed to better fit to the chord(s) assigned to the preceding moment. In the chord reduction procedure, groups of moments (including super-moments) are optionally combined together with following and/or preceding moment(s) to produce a super-moment with one chord assigned to it.
The chord reduction procedure allows for disregarding non-single-note moments. The procedure may be run once and again until further reduction of the number of chord changes along the score is not obtained. As in the description of the chord combination procedure, the term moment is used herein to denote both single-moment and super-moment.
In a preferred embodiment, the chord reduction procedure is carried out in accordance with a LUT.
Other tables may be used as well, for instance, when different kinds of musical scores (classical, Jazz, ancient, etc.) are handled. It is also possible that some musicians would find another table to better suit their preferences. The table is provided to show one way to carry out chord reduction. In other embodiments, the chord reduction stage may be embodied in algorithm, rather than a LUT.
The LUT of
The action taken during the reduction process is indicated in the table of
The exemplary reduction LUT considers the reduction of a sequence of up to 5 chords at a time. The following factors, regarding each of the five chords, are considered.
As may be seen in the table, the columns relating to the interval are always empty, as the inventors have not yet found a case where this factor has to be considered.
The inventors found that the above-described chord assignment method provides flow of harmonic changes of high musical correctness for all tonal musical scores that were tested. One example to such high musical correctness chord assignment is provided in
Exemplary Result of an Exemplary Method
The second line (134) shows chords assigned by a method that also includes one run of a chord reduction process; and the third line (136) shows chords assigned by a method that includes two consecutive runs of the chord reduction procedure. Further repetitions of the chord reduction procedure do not change the chords assigned to the shown score-portion. The procedures used for the preparation of the chord assignments are those described in detail above. The chord notation used in the lines 132, 134, and 136 is that the first letter designates the root of the assigned chord (with # representing a sharp and b representing a flat), and the second letter designates its quality as follows: M=Major, m=minor, d=diminished, and A=augmented. Chords of other qualities, such as diminished-diminished, Augmented-Augmented, minor-Augmented, Major-diminished, and diminished-Augmented may be assigned by the method, but do not appear in the shown score-portion.
Options and Alternatives
In the preceding sections, preferred embodiments were referred to. Nevertheless, the invention is not limited to these embodiments, and alternatives and options do exist. Some such options and alternatives are described in the following sections.
Time Segmentation:
In some embodiments of the present invention the score is segmented along the timeline not by moments, as these were described above, but to other kinds of time segments, for instance time segments of constant duration. However, if the analysis relates only to time segments that are longer than moments, musical events that occur during the time segment might be ignored, and this might lead to analysis of lower musical correctness than is achieved with moments. Using time segments shorter than a moment seems not to improve the musical correctness of the analysis. Also the end of a moment can be defined as the disappearance of a note.
Determining Accidentals of Virtual Notes:
In some embodiments of the invention, each virtual note is assigned the accidental the same note had last time it existed in the score, or, if it never existed before in the score, in accordance with the key signature. In other embodiments, each virtual note is assigned the accidental the same note had last time it existed in the bar, or, if it never existed before in the bar, in accordance with the key signature. While these embodiments allow for easier determination of accidentals, the musical correctness achieved with them is often lower than when the accidentals are determined as in the preferred embodiment.
Chord Determination Rules:
While the inventors have found that the rules described above provide the most satisfactory musical analysis, of all rules they have attempted to apply, additional sets of rules may exist. Some modifications that may be made are: it is possible to consider all trial triads that have notes in the time segment, not necessarily the root. The various terms may be positioned in other places in the hierarchy, or completely ignored; lower quality dissonant resolutions may be treated as non-resolutions; resolutions of various qualities may be defined to dissonants other than the seventh, etc. Dissonant resolutions may be looked for up to a certain time of moments after the appearance of the dissonant, or certain duration after its appearance, etc. Also chords other than triads may be defined as allowed chords.
Chord Combination:
The stage of chord combination may be omitted, and the chords assigned by the chord determination process may be treated directly with a chord reduction process. However, this might require a much larger look-up-table for the reduction stage.
Chord Reduction:
The stage of chord reduction may be omitted, but in complex musical scores this might result in chord assignment characterized by too frequent chord changes along the score, and thus to lower the musical correctness of the chord assignment.
The Optimization Process
The preferred embodiment was described above as utilizing a “mount climbing” optimization process, however, every optimization method known in the art may be useful to find the chord sequence of optimal musical fitness.
Defining Trial Chords
The above described preferred embodiment, is one of a broad family of embodiments, where optimization is carried out by first assigning a best fitting chord to each time segment (in the preferred embodiment: by the chord determination procedure), and then amending the chord assigned to each segment is considered, responsive to chords assigned to its neighboring segments (by the chord combination and chord reduction procedures).
In other members of this embodiment family, the best fitting chord for a single time segment is determined by computing the musical fitness of all the allowed chords, and choosing the one that gives the highest value.
Alternatively, the musical fitness is computed only for some trial chords selected from the allowed chords. Optionally, the trial chords are selected responsive to the notes of the time segment. Additionally or alternatively, the trial chords are selected responsive to the notes of preceding time segments. Optionally, the trial chords are chosen such that each trial chord is a triad having its root in the time segment. Optionally, the accidentals of virtual notes in a trial chord are determined responsive to accidentals that appeared earlier in the score, including in the key-signature.
Alternatively or additionally, the trial chords are selected responsive to chords assigned to neighboring time segments.
Optionally, a first time segment is considered a neighbor of a second time segment if the number of time segments separating them is not larger than a defined number. In one example, only adjacent segments are considered neighbors (in which case the defined number is 0). Alternatively, segments that are further along the timeline, separated by 1, 2, 3, 4, 5, 10, or any other number of segments are considered neighbors. Alternatively or additionally, neighborhood relationships between time-segments end when a minimal number of new notes appear in the score; for instance, when two or more new notes appear.
Optionally, the optimization process accepts only changes associated with increase of the musical fitness. Alternatively, the process includes random acceptance of changes that result in decrease of the musical fitness, with a probability depending on the amount of decrease.
The above described methodology is an example of optimizing, in which a cost function (musical fitness) is used to select an optimal or near-optimal matching of chords to music. The above description relates generally to local optimization method; however, other optimization methods are known in the art and may be used, including, for example, hill climbing, simulated annealing, quantum annealing, tabu search, genetic algorithms, ant colony optimization, evolution strategy, differential evolution, and particle swarm optimization. The optimization methods may use various types of cost functions, including local functions (that depend on the relationships between a moment and a chord assigned to it), contextual components, that depend on the relationships between a chord assigned to a moment and the notes and/or chords of neighboring moments, and global functions, that depend on the chords assigned to the entire score.
Rhythmic Flow
Overview
Automatic Tempo Assignment
In an embodiment of the invention, the duration of a moment of a lower beat, which precedes a moment of a higher beat, is longer than the explicit duration as defined in the score. For instance, the beat of the last moment in a measure is lower than the beat of the first moment in the following measure. Accordingly, in an embodiment of the invention, a moment that appears at the end of a measure is assigned a tempo that is slower than the tempo assigned to it by the tempo marking in the score.
In an embodiment of the invention, a similar rule applies to moments of lower beat grade that precede moments of higher beat grade inside a measure. For instance, in a score written in 4/4 time, the moment preceding the moment that appears at the beginning of the third quarter is optionally assigned slower tempo.
A look-up-table with beat grades for various time signatures is provided in application 60/821,744 as annex III, which is incorporated herein by reference.
In an embodiment of the invention a tempo is “slow” if it is slower than the tempo indicated in the score. Alternatively or additionally, a moment is referred to as having a slower tempo when its tempo is slower than that of the preceding moment.
More generally, it is in accordance with embodiments of the invention to assign to each moment a tempo responsive to the time signature and the moment's position in the measure. Optionally, this is done by grading the beat of each moment in the measure, and assigning to each moment a tempo responsive to the moment's beat-grade.
The terms beat and time signature are explained in the glossary above.
Generally, there is no one correspondence rules-set governing the correspondence between beat-grade and tempo, and different rule-sets may be applicable to different kinds of music or different performance styles. Nevertheless, the following correspondence guidelines may be helpful in assigning tempi to moments in a manner that would result in a performance that in most cases will sound “rhythmically correct”:
In an embodiment of the invention, the velocity assigned to a moment of a lower beat, which precedes a moment of a higher beat, is lower than the velocity assigned to the moment of the higher beat. For instance, in an embodiment of the invention, a moment that appears at the end of a measure is assigned a velocity that is lower than the velocity assigned to the moment appearing in the beginning of the following measure.
In an embodiment of the invention, a similar rule applies to moments of lower beat grade that precede moments of higher beat grade inside a measure. For instance, in a score written in 4/4 time, the moment preceding the moment that appears at the beginning of the third quarter is assigned a lower intensity.
In an embodiment of the invention a velocity is “low” if it is lower than the velocity of the note that follows it. Alternatively or additionally, a moment is referred to as having a lower velocity when its velocity is lower than the velocity assigned to it by a specific dynamic marking in the score or default intensity.
It should be noted that music notation usually does not assign velocity values to each and every note in the score, such velocity assignment being left to the discretion of the performer. In an embodiment of the invention the performer is an apparatus, which assigns intensities to notes in accordance with the time signature and the position of the note in the measure.
More generally, it is in accordance with embodiments of the invention to assign to each note a velocity responsive to the time signature and the note's position in the measure. Optionally, this is done by grading the beat of each note in the measure, and assigning to each note a velocity responsive to the note's beat-grade.
The following correspondence guidelines may be helpful in assigning intensities to notes in a manner that would result in a performance that in most cases will sound “rhythmically correct”:
These guidelines may be followed with many different pre-sets, and they are all in accordance with embodiments of the invention. Specific numbers that provide most appropriate pre-sets for a specific musical score or performance style may be found by a skilled person in a trial and error process, and the pre-sets reproduced herewith provide good starting points.
An Automatic Music Performer
Optionally, input 102 is a notation application (that is a ‘word processor’ for writing music notes rather than alphabet letters), which allows a user to input music as notes and get a printable score as an output. Such notation applications are commercially available, for instance, from MakeMusic! by the trade name Finale™ or from Sibelius by the trade name Sibelius™. Symbolic music notation file is any computer-readable music representation file, such as a MIDI list or Music XML.
Additionally or alternatively, input 102 is configured to receive a music notation file created outside of performer 100.
Additionally or alternatively, input 102 is configured to receive a score in standard notation, and includes an optical reader for reading the score, and software for translating it into a music notation file. Such a system is described, for instance, in US Patent Application Publication No. 2006-254407, the entire contents thereof is incorporated herein by reference.
Additionally or alternatively, input 102 is configured to receive a score in any other format that includes the information conventionally included in a standard musical score.
Input 102 may also include a user interface, configured to allow a user to input performance preferences.
Storage 104 is configured to store the input music notation file. Additionally, storage 104 stores a software applications and data files for use by processor 106.
In an embodiment of the invention, processor 106 is configured to create a second music notation file, based on the file received through input 102 and the software application and/or data files stored on storage 104. Generally, processor 106 extracts at least one rhythmic characteristic of a moment, and assigns a tempo parameter to the moment, in accordance with the rules saved on storage 104 regarding tempo parameter of moments with the at least one parameter extracted. Some details of actions preformed by such software in an exemplary embodiment of the invention are provided below, under the title ‘assigning tempo and/or velocity parameters’.
In an embodiment of the invention, output 108 is a display, displaying the music notation file prepared by processor 106. Optionally, the display is visual, and the music notation file is displayed in a human-readable format. Optionally, the display has an audio output for playing back the music notation file as music performance. Preferably, such music performance sounds more “rhythmically correct” than performance of the input file.
Exemplary User Interfaces
In an exemplary embodiment of the invention, the user interface allows a user to choose among several different tempo pre-sets and several different velocity pre-sets. Optionally, the pre-sets are conjugated, such that each tempo pre-set is chosen by default with its conjugated velocity pre-set. Optionally, this default may be overridden.
In an embodiment of the invention, the user interface allows choosing different pre-sets to different portions of the score. For instance, the user is allowed to mark portions of the score with the mouse, and select for the marked portion a pre-set from a pre-set menu.
In an exemplary embodiment of the invention, the user interface allows for changing, creating, and saving pre-sets.
In an embodiment of the invention a pre-set may be graphically amended. Optionally, a graph such as that presented in
In an embodiment of the invention, the pre-sets are provided variable-dependent. For instance, a tempo parameter is provided as 1+0.1a, 1−0.1a and the like (rather than 1.1, 0.9, etc.). Optionally, the user is allowed setting the value of the variable. In the above example, setting a=1 would result in parameters 1.1, 0.9; while setting a=0.4 would result in 1.04, 0.96. Optionally, the user interface allows a user to set values of variables, on which pre-sets depend, with a slider.
Assigning ‘Rhythmically Correct’ Tempo and/or Velocity Parameters
Optionally, processor 106 reads from storage 104 tempo and/or velocity parameters associated with different measure-portions, and assigns to each moment the tempo and/or velocity parameters associated with the measure-portion occupied by the moment. A moment that occupies more than one measure-portion is optionally associated with tempo and/or velocity parameters, each being the average of the values assigned to the measure-portions occupied by the moment.
It is convenient to assign tempo and/or velocity parameters to measure portions corresponding to the shortest subdivision in the measure. This way, each moment occupies a whole number of measure-portions. Thus, in an embodiment of the invention, a list of beat-grades is prepared for each subdivision and a tempo and/or a velocity is assigned to each subdivision responsive to the subdivision's beat-grade. The lists of beat-grades and corresponding tempo and/or velocity parameters may be prepared in advance, and used for many different scores, as they are not score-specific. In performing a specific score, each moment is assigned a tempo and a velocity, where applicable (or an average of tempi and intensities, as the case may be) from the lists of tempi and velocities. Optionally tempo and/or velocity may be expressed in these lists as percentages of fixed tempo and/or velocity value.
Optionally, lists of tempo and/or velocity parameters are prepared for the shortest subdivision, and parameters of all moments in the score are assigned based on this list. Preferably, lists of tempo and/or velocity parameters are prepared for each subdivision, and each moment is assigned a parameter from the list prepared for the shortest subdivision appearing in the measure (that is, the shortest moment appearing in the measure).
In an embodiment of the invention, processor 106 assigns tempo parameters to each moment and/or velocity to each note in accordance with the explicit performance indications in the score (202′, 208′), time signature (210′) indicated in the input music notation file and the position of the moment/note in the measure.
Table III provides tempo and velocity parameters according to an exemplary embodiment of the invention, for time-signature 4/4, with the smallest subdivision being 1/4.
Table IV provides tempo and velocity parameters according to an exemplary embodiment of the invention, for time-signature 4/4, with the smallest subdivision being 1/16.
In operation, processor 106 identifies the shortest subdivision in each bar, and accordingly extracts parameters from table II (if the shortest subdivision is 1/4, such as in the first measure in
Similar tables may be provided for use in performing scores written in other time signatures and having measures with other shortest subdivisions. The values appearing in such tables in some exemplary embodiments of the invention are provided in
In all the figures, the horizontal axis show EDU (elapsed duration units) at which a subdivision begins. EDU is a measure of the nominal time-length of a score-portion, namely, the time-length between two points in a score, if played exactly as indicated by the subdivisions.
As the score of
Processor 106 reads the subdivision of each moment from the first music notation file stored in storage 104. If the subdivision is the shortest in the measure (such as in moments #2 or 3 in the first measure and 2 or 3 in the second measure in
If the moment is longer than the shortest subdivision (such as in moments #1, 4 or 7 in
Table V provides subdivisions of each moment appearing in the score reproduced in
Pre-Sets of Full Correspondence with Beat-Grade
In an embodiment of the invention, the tempo and optionally also the velocity parameters are set in full correspondence with the beat-grade, such that a moment (or note) having a higher beat-grade is associated with a higher velocity and faster tempo than a moment or note with a lower beat-grade.
Optionally, a maximal range is set for each parameter, and each beat-grade unit is associated with tempo parameter that corresponds to the maximal range, divided by the number of subdivisions in the measure. For instance, for time-signature 4/4, subdivision 1/32, and maximal range of velocity 16%, each beat-grade unit corresponds to 0.5% in the velocity parameters. For instance, if the beat-grades of the four first subdivisions are 32, 30, 31, 29 the corresponding velocity parameters may be 1.08, 1.07, 1.075, and 1.065 etc. Alternatively, the first subdivision may have another value and the differences remain the same, for instance: 1.00, 0.99, 0.995, and 0.985.
The full correspondence is easy to establish directly from the beat-grade tables, and provides improved musical correctness, however, the pre-sets described below provide even better results, as they suggest more variety between different measures.
Exemplary Tempo Pre-Sets
The bar-charts of
In the upper bar-chart (3A), tempo parameters are provided for quarters, and the quarters have the pattern: fast, intermediate, intermediate, slow. This is not in full correspondence with the beat-grade (4, 2, 3, 1, see appendix III in 60/821,744, which is incorporated herein by reference), however, it obeys guideline (a) above, as the tempo of the weak beats (34, 36) are not higher (faster) than that of the strong beats that follow them (38 and 40, respectively).
In the third bar-chart 3C providing tempo parameters for 1/16th, guideline (a) is obeyed, as all the weaker beats (62) have the same tempo parameter as the stronger beats that follow them, except for the last one in the second quarter (62′), and the last one in the forth quarter (62″), which are slower than the strong beat following them.
In the fourth bar-chart 3D, providing tempo parameters for 1/32nd, guideline (a) is strictly obeyed. It may be note-worthy that this does not stand in the way of providing a general slow-down beginning at the middle of the third quarter (72) and ending at the end of the measure.
Guideline (b) is kept all along the bar-charts of
Compliance with guideline (c) is illustrated in the 1/8th bar-chart, with the dashed lines showing that the average value over each of the quarters obeys the same general rule (fast, intermediate, intermediate, slow) as the time-signature unit.
These guidelines may be followed with many different pre-sets, and they are all in accordance with embodiments of the invention. Specific numbers that provide most appropriate pre-sets for a specific musical score or performance style may be found by a skilled person in a trial and error process or based on knowledge of certain styles (primarily dance forms), and the pre-sets reproduced herewith provide good starting points.
Exemplary Velocity Pre-Sets
The bar-charts of
In
In the velocity pre-set of
It is note-worthy that in the charts of 1/16s (
The inventors found that such crescendo makes the performance sound more rhythmically correct than many other arrangements, in particular when it is combined with a slow-down shown in the corresponding tempo charts, as discussed above.
Additional Pre-Sets
In an embodiment of the invention, music performer 100 may have additional pre-sets of tempo and/or velocity parameters. For instance, storage 104 may store pre-sets for phrase-endings. Such a pre-set may include, for instance, a slow down accompanied with a gradual reduction of velocity towards the end of a phrase, and gradual speed up accompanied with a gradual increase of velocity in the beginning of a new phrase. Borders between phrases may be assigned arbitrarily, say, each four measures; manually by a user through a user interface, or by any other means, such as thematic analysis.
In an embodiment of the invention, the processor is configured to identify phrase ends from the flow of harmonic changes in the musical score. The harmonic flow changes may be deduced from the music notation file by the musical performer, for instance, as described herein. Alternatively or additionally, the harmonic flow change may be provided to music performer 100 with the input music notation file, or in any other input means. In this embodiment, the processor identifies phrase ends automatically, whenever harmonic flow patterns corresponding to ‘authentic cadence’, ‘half cadence’, ‘incomplete cadence’ or any other cadence is found in the score.
In an embodiment of the invention, music performer 100 also includes a pre-set for dissonants and their resolutions. This may include, for instance, a slow tempo on the dissonant moment and a faster tempo on the resolution moment both being slower than the general tempo as well as lower velocity on the resolution note.
In an embodiment of the invention, music performer 100 also includes pre-sets for rhythmic events such as syncopation, melodic events such as melodic repetition or ‘sequence’ (meaning a repetition in a higher or lower pitch)
In an embodiment of the invention, music performer 100 also includes pre-sets for carrying out explicit performance indication that appear in scores. For instance, there may be a “crescendo” and “diminuendo” pre-sets for performing score-portions marked with crescendo or diminuendo marks. Such pre-sets may include, for instance a tempo slow down towards the end of the crescendo or the diminuendo, or a slow down before a “sforzando” or a “Fermata”, etc.
Pre-sets may be applied to articulation (duration), controllers such as the sustaining pedal of the piano, vibrato, pitch bend, instrument specific options such as bow direction in bowed string instrument, tonguing in woodwinds, etc.
Similarly to the pre-sets that are of pure rhythmic origin, also the punctuation pre-sets, the dissonant-resolution pre-sets, as well as any other pre-set is score-independent, and may be useful with practically any score, where using a specific pre-set for a specific score is mainly dependent on user preferences.
When more than one pre-set is in use, the deviations appearing in each of them optionally occumulate. For instance, if a certain moment has a tempo parameter of 110% in the rhythmic pre-set and tempo parameter of 110% in the end-of-phrase pre-set, and these are the only two active pre-sets, this moment will be played with a tempo parameter of 121%.
Optionally, pre-sets values may be modified to avoid exaggerated effects of multiple pre-sets working together. For instance, in an embodiment of the invention, the slow down associated with a cadence is reduced by 50% if the moments to be slowed down due to the cadence are also marked in the score with an explicit slow-down marking.
Optionally, when more than one pre-set is in use, the user-interface allows controlling the pre-sets in groups. For instance, there may be one control for all tempo pre-sets, which allows a user to set all tempo-changes dictated by the pre-sets to the same level, for instance, to maximal level. Additionally or alternatively, a single control may affect several pre-sets of similar origin, for instance, there may be one control for punctuation pre-sets, another for explicit performance indications, and another to pre-sets of purely rhythmic origin. This way a user may decide to make punctuation more (or less) pronounced than in the default, while keeping the rhythm at default, etc.
In an embodiment of the invention, many layers of pre-sets are provided, wherein each layer allows a different degree of control, where the deeper layer allows controlling each pre-set individually, and the upper layer has only small number of controls, for instance, 1, 2, 3, or 4, each controlling a large number of pre-sets.
Exemplary Uses of Automatic Music Performers
An automatic music performer as described above may be used by professional musicians wishing to automatically perform music with a good rhythmic sense. This may help a musician evaluate in advance how a certain piece of music is going to sound when performed. If sufficient pre-sets are provided, for instance, as described above under the heading of ‘additional pre-sets’, the music performance may be good enough to allow replacing a human performance. In such a case, the musician may use the automatic music performer for preparing low cost high quality performances of freshly composed music.
Another use of an automatic music performer according to some embodiments of the present invention is in musical education. Music students (in the university, high school, or even kindergarten), may play around with the pre-sets to learn about the effect of rhythm (or any other factor for which pre-sets are provided) on the general impression made by a musical performance.
General
The present invention has been described using non-limiting detailed descriptions of embodiments thereof that are provided by way of example and are not intended to limit the scope of the invention. It should be understood that features and/or steps described with respect to one embodiment may be used with other embodiments and that not all embodiments of the invention have all of the features and/or steps shown in a particular figure or described with respect to one of the embodiments. Variations of embodiments described will occur to persons of the art. Furthermore, the terms “comprise,” “include,” “have” and their conjugates, shall mean, when used in the disclosure and/or claims, “including but not necessarily limited to.”
It is noted that some of the above described embodiments may describe the best mode contemplated by the inventors and therefore may include structure, acts or details of structures and acts that may not be essential to the invention and which are described as examples. Structure and acts described herein are replaceable by equivalents, which perform the same function, even if the structure or acts are different, as known in the art. Therefore, the scope of the invention is limited only by the elements and limitations as used in the claims.
This application is a divisional of U.S. patent application Ser. No. 12/376,438 filed on Feb. 5, 2009, which is a National Phase of PCT Patent Application No. PCT/IL2007/000961 having International filing date of Jul. 31, 2007, which claims the benefit of priority under Section 119(e) of U.S. Provisional Patent Application Nos. 60/821,602 filed on Aug. 7, 2006, 60/821,744 filed on Aug. 8, 2006, and 60/884,447 filed on Jan. 11, 2007. The contents of all of the above applications are incorporated by reference as if fully set forth herein.
| Number | Name | Date | Kind |
|---|---|---|---|
| 3916754 | Thiel | Nov 1975 | A |
| 4002097 | Leonard | Jan 1977 | A |
| 4295402 | Deutsch et al. | Oct 1981 | A |
| 4345501 | Nakada et al. | Aug 1982 | A |
| 4419918 | Dyck et al. | Dec 1983 | A |
| 4539882 | Yuzawa | Sep 1985 | A |
| 4864907 | Oguri | Sep 1989 | A |
| 4941387 | Williams et al. | Jul 1990 | A |
| 4951544 | Minamitaka | Aug 1990 | A |
| 5085118 | Sekizuka | Feb 1992 | A |
| 5218153 | Minamitaka | Jun 1993 | A |
| 5218157 | Akagawa et al. | Jun 1993 | A |
| 5220122 | Shibukawa | Jun 1993 | A |
| 5221801 | Bruti et al. | Jun 1993 | A |
| 5260510 | Shibukawa | Nov 1993 | A |
| 5300728 | Shimada | Apr 1994 | A |
| 5302776 | Jeon et al. | Apr 1994 | A |
| 5418325 | Aoki et al. | May 1995 | A |
| 5424486 | Aoki | Jun 1995 | A |
| 5429023 | Imaizumi | Jul 1995 | A |
| 5455379 | Kim et al. | Oct 1995 | A |
| 5491298 | Eiichiro | Feb 1996 | A |
| 5510572 | Hayashi et al. | Apr 1996 | A |
| 5629491 | Usa | May 1997 | A |
| 5736666 | Goodman et al. | Apr 1998 | A |
| 5753843 | Fay | May 1998 | A |
| 5760325 | Aoki | Jun 1998 | A |
| 5942710 | Hayakawa et al. | Aug 1999 | A |
| 6060655 | Minamitaka | May 2000 | A |
| 6618336 | Kanou et al. | Sep 2003 | B2 |
| 6791021 | Aoki | Sep 2004 | B2 |
| 7183479 | Lu et al. | Feb 2007 | B2 |
| 20060060067 | Lu et al. | Mar 2006 | A1 |
| 20100175539 | Silbert et al. | Jul 2010 | A1 |
| 20110036231 | Nakadai et al. | Feb 2011 | A1 |
| 20120118128 | Silbert et al. | May 2012 | A1 |
| Number | Date | Country |
|---|---|---|
| 1533786 | May 2005 | EP |
| WO 2008018056 | Feb 2008 | WO |
| Entry |
|---|
| Communication Relating to the Results of the Partial International Search Dated Nov. 26, 2007 From the International Searching Authority Re.: Application No. PCT/IL2007/000961. |
| International Preliminary Report on Patentability Dated Feb. 19, 2009 From the International Bureau of WIPO Re.: Application No. PCT/IL2007/000961. |
| International Search Report and the Written Opinion Dated Feb. 22, 2008 From the International Searching Authority Re. Application No. PCT/IL2007/000961. |
| Notice of Allowance Dated Sep. 20, 2011 From the US Patent and Trademark Office Re. U.S. Appl. No. 12/376,438. |
| Official Action Dated Nov. 12, 2010 From the US Patent and Trademark Office Re. U.S. Appl. No. 12/376,438. |
| Official Action Dated Feb. 18, 2011 From the US Patent and Trademark Office Re. U.S. Appl. No. 12/376,438. |
| Response Dated Dec. 13, 2010 to Official Action of Nov. 12, 2010 From the US Patent and Trademark Office Re. U.S. Appl. No. 12/376,438. |
| Response Dated Jun. 20, 2011 to Official Action of Feb. 18, 2011 From the US Patent and Trademark Office Re. U.S. Appl. No. 12/376,438. |
| Barthélemy et al. “Figured Bass and Tonality Recognition”, 2nd Annual International Symposium on Music Information Retrieval, ISMIR, Indiana University, Bloomington, IN, USA, 8 P., 2001. Retrieved From the Internet: <URL: http://ismir2001.ismir.net/pdf/barthelemy.pdf>. Abstract, Figs.1-7, Introduction, 4. Description of Harmonic Reduction. |
| Pardo et al. “Algorithms for Chordal Analysis”, Computer Music Journal MIT Press, 26(2): 27-49, 2002. Abstract, Figs.1-4, Tables 1-3, p. 27-32, p. 35, r-h col.—p. 37, r-h col. |
| Pardo et al. “Algorithms for Chordal Analysis”, Database INSPEC [Online], Database Accession No. 7376008, 2002. |
| Pardo et al. “Automated Partitioning of Tonal Music”, Database INSPEC [Online], Database Accession No. 7372830, 2000. Abstract. |
| Pardo et al. “Automated Partitioning of Tonal Music”, Proceedings of the 13th International Florida Artificial Intelligence Research Society Conference, FLAIRS, Orlando, FL, USA, p. 23-27, 2000. |
| Number | Date | Country | |
|---|---|---|---|
| 20120118128 A1 | May 2012 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60821602 | Aug 2006 | US | |
| 60821744 | Aug 2006 | US | |
| 60884447 | Jan 2007 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 12376438 | US | |
| Child | 13355578 | US |
