Patent Grant
7653534
1. Field of the Invention
The present invention relates to the technical field of musical harmony recognition, and in particular, the present invention relates to an apparatus and a method for determining a type of chord using a reference vector for a key of the type of chord.
2. Description of Prior Art
In the last ten years, due to the clearly improved storage and sound optimization of recorded pieces of music, the importance of classifying these pieces of music into styles of music has increased. However, it has had to be taken into account that in the last few years a multitude of subtypes of styles of music have formed, a classification into, for example, the styles of music of “classical music”, “jazz”, “rock music”, . . . having proven to be no longer sufficient. In addition, one has had to bear in mind/to note that due to the considerable increase of pieces of music published, the formation of different tastes in music has also been boosted. The increase in the number of different tastes in music, and the marked increase of pieces of music published necessitated also a pre-classification of the pieces of music which is then enclosed, in an electronic form, as meta data (i.e. data about data) with the piece of music which was mostly also stored in an electronic form. Since the classification of the piece as may be taken from the meta data is often made obsolete due to the change in tastes of music, it was necessary to provide a possibility of generating meta data directly from the musical properties of a piece of music shortly before classification, wherein a classification of the piece of music then need not be stored in the meta data, but wherein only the musical and/or the music-theoretical properties of the piece of music are recognized from the piece of music itself, and wherein the style of music or subclass of the style of music of the piece of music may be inferred therefrom.
As an evident characterizing feature of a piece of music, the classification of occurring types of chords, such as a D major or a G minor chord in a piece of music, could serve as an important feature in classifying the piece of music to belong to a style of music or subclass of a style of music. In a first approach of recognizing a key of a piece of music, David Temperley suggested, in his document “The Recognition of Basic Musical Structures”, The MIT Press, 2001, pages 173 to 187, to establish key profiles by means of an empirical psycho-acoustic reference model formation. These key profiles then indicate a frequency of how often a certain note occurs, for example in a piece of music in C major, in relation to other notes. Such key profiles are represented, for example, for the C major key in
However, such an approach has the disadvantage that a temporally long segment of a piece of music is to be used for creating a meaningful histogram as a meaningful test signal vector), and that the key of the piece of music may change within this time segment of the piece of music to be examined. This will subsequently lead to an inaccurate classification of the piece of music. In addition, it is only the key of the piece of music (and/or of the examined time segment of the piece of music) that can be recognized by the above-described method; the recognition of the key of an individual chord is thus not ensured. This primarily results from the fact that, for short time segments, no meaningful histogram may be prepared due to the short time duration of the segment. Thus, the temporal resolution behavior of the above-described method is limited.
Thus, it is the object of the present invention to provide a possibility of determining a type of chord underlying a test signal, it being intended for the determination of the chord type underlying the test signal to provide a temporal resolution which is better than has been possible in the prior art.
In accordance with a first aspect, the invention provides an apparatus for determining a type of chord underlying a test signal, the type of chord being defined by an occurrence of predetermined frequencies in a frequency range of the test signal, and the predefined frequencies in the frequency range of the test signal corresponding to tones in a predetermined spectral margin, wherein a first type of chord has at least one predefined significant tone in the spectral margin and wherein a second type of chord has a second predefined significant tone in the spectral margin, wherein the first significant tone differs from the second significant tone, and wherein the apparatus for determining has:
a provider for providing a reference vector for the type of chord from a plurality of different reference vectors, the reference vector having a plurality of reference vector elements associated with one tone, respectively, in the spectral margin, and wherein at least one significant reference vector element is provided for each reference vector for a significant tone of an associated type of chord;
a provider for providing a test signal vector from the test signal, wherein the test signal vector has a plurality of test signal vector elements associated with one tone, respectively, in the spectral margin, wherein a test signal vector element is dependent on whether the tone associated with the test signal vector element occurs in the test signal, wherein the provider for providing a test signal vector is configured to allocate a value of one to a test signal vector element if the tone corresponding to the test signal vector element has an amplitude value which exceeds a predetermined threshold value, and wherein the provider for providing a test signal vector is configured to allocate a value of zero to the test signal vector element if the tone which corresponds to the test signal vector element has an amplitude value falling below the predetermined threshold value; and
a comparator for comparing the reference vector with the test signal vector, the comparator being configured to compare the reference vector with the test signal vector or with versions of the test signal vector which are cyclically shifted by various shift values in the frequency range, in order to obtain various comparative results allocated to the test signal vector or shift values in order to determine the type of chord on the basis of an extreme comparative result and of the shift value associated with same.
In accordance with a second aspect, the invention provides a method for determining a type of chord underlying a test signal, the type of chord being defined by an occurrence of predetermined frequencies in a frequency range of the test signal, and the predefined frequencies in the frequency range of the test signal corresponding to tones in a predetermined spectral margin, wherein a first type of chord has at least one predefined significant tone in the spectral margin and wherein a second type of chord has a second predefined significant tone in the spectral margin, wherein the first significant tone differs from the second significant tone, and wherein the method for determining includes the steps of:
providing a reference vector for the type of chord from a plurality of different reference vectors, the reference vector having a plurality of reference vector elements associated with one tone, respectively, in the spectral margin, and wherein at least one significant reference vector element is provided for each reference vector for a significant tone of an associated type of chord;
providing a test signal vector from the test signal, wherein the test signal vector has a plurality of test signal vector elements associated with one tone, respectively, in the spectral margin, wherein a test signal vector element is dependent on whether the tone associated with the test signal vector element occurs in the test signal, wherein the step of providing a test signal vector includes allocating a value of one to a test signal vector element if the tone corresponding to the test signal vector element has an amplitude value which exceeds a predetermined threshold value, and wherein the step of providing a test signal vector includes allocating a value of zero to a test signal vector element if the tone which corresponds to the test signal vector element has an amplitude value falling below the predetermined threshold value; and
comparing the reference vector with the test signal vector, the comparator being configured to compare the reference vector with the test signal vector or with versions of the test signal vector which are cyclically shifted by various shift values, in order to obtain various comparative results allocated to the test signal vector or shift values in order to determine the type of chord on the basis of an extreme comparative result and of the shift value associated with same.
In accordance with a third aspect, the invention provides a computer program having a program code for performing the method for determining a type of chord underlying a test signal, the type of chord being defined by an occurrence of predetermined frequencies in a frequency range of the test signal, and the predefined frequencies in the frequency range of the test signal corresponding to tones in a predetermined spectral margin, wherein a first type of chord has at least one predefined significant tone in the spectral margin and wherein a second type of chord has a second predefined significant tone in the spectral margin, wherein the first significant tone differs from the second significant tone, and wherein the method for determining includes the steps of:
The present invention is based on the findings that by using a reference vector and a test signal vector determined from the test signal, a comparison of the reference signal vector with the test signal vector may be effected, and that the type of chord may be derived directly from the result of the comparison. Unlike the prior art, it is here no longer required to provide a large number of reference vectors. Rather, for a class of chord types such as the major chords, a reference value may be provided, and from the knowledge of a distance between two tones occurring in a chord, a fundamental tone of the type of chord to be determined may be ascertained by means of a cyclically shifting the test vector and comparing the shifted version of the test vector with the reference vector.
The inventive approach offers the advantage that a statistical distribution of the occurrence of tones, or halftones, (histogram) in the test signal is no longer required for determining the type of chord. Rather, the type of chord underlying the test signal may be determined in a simple manner by providing the reference vector and the test signal vector, derived from the test signal, with a subsequent simple cyclic shift of the elements of the test signal vector. Unlike the prior art, one does not need to fall back on a large number of reference vectors (which have been determined in a psycho-acoustic manner). Also, compared to the prior art, the presence of a long-duration test signal is not required for determining the type of chord. This means that the test signal which has the type of chord underlying it may be clearly shorter than in conventional approaches. In particular, this results from the fact that, in the inventive approach, it is only the occurrence of a tone within the test signal that is detected in an element of the test signal vector, and that thus, for example, a simultaneous sounding of different tones within a short period of time (for example a quarter note) is sufficient to detect the type of chord underlying the test signal by means of the spectral distance of the tones. The inventive approach thus offers the advantage, over the prior art, of being able to examine clearly shorter time segments for types of chords, and thus of achieving a clearly higher level of granularity of the chord-type determination within a test signal.
These and other objects and features of the present invention will become clear from the following description taken in conjunction with the accompanying drawing, in which:
In the subsequent description of the preferred embodiments of the present invention, identical or similar reference numerals will be used for those elements depicted in the various drawings which have similar actions; repeated descriptions of these elements being dispensed with.
The method of chord determination performed in means 106 for comparing is fundamentally based on a pattern recognition process. Tonal events (i.e. the occurrence of tones in the test signal) are compared with one (or several) reference vectors representing various types of chords and/or chord-type classes. These reference vectors may include a number of, e.g., 12 reference vector elements corresponding to the 12 different halftones in an octave of the Western scale of tones. In addition, those reference vector elements wherein respective tones or tones occur in the respective type of chord or the class of chord types may then be set to a value of 1, wherein the other reference vector elements may then be set to a value of 0. The tones occurring in a chord are significant for the type of chord and will be referred to as significance reference vector elements in the description which follows. For example, the reference vector for a major chord may contain the value of 1 in the first, fifth and eighth (significance) reference vector elements, or the reference vector for a minor chord may contain the value of 1 in the first, fourth and eighth reference vector elements, whereas the other reference vector elements have a value of 0 in the respective reference vectors.
As an alternative to major-key or minor-key reference vectors, other reference vectors may also be used, as are roughly reproduced in the tabular representation in
In addition, for example a matrix with 12 rows may be prepared in means 104 for providing as is represented in
By means of such cyclic shifting of the elements of the test signal vector, all possible chord inversions may thus be verified. The class of chord types identified by that reference vector which resulted in the highest scalar product in the comparison, for example, by the formation of the scalar product, will then be output as the class of the chord type. In addition, the respective fundamental tone of the chord type recognized is determined from the number of cyclic shifts performed which led to the highest scalar product.
To prevent such an apparatus for determining a type of chord from identifying incomplete chords, an additional criterion may be applied. The results of the scalar-product formation for reference vectors which contain the value of 1 in one component of the reference vector, and exhibit the value of 0 at the respective location of the input vector (test vector) are disregarded in determining the type of chord. In the event that only such reference vectors are available which, in connection with an input vector, lead to such a disregard of the results for the chord recognition, the algorithm suggested herein may be configured such that it outputs the individual notes within the respective time frame or time segment.
In the event that the input vector comprises more elements than the reference vector, the closest match between the input vector and the reference vector is selected. This may be effected, for example, in that in the input vector individual elements are taken out, whereby the input vector is reduced in length. This reduced input vector may then be compared with the respective reference vector(s), and/or the scalar product may be formed, and the result which provides the best comparison value or the highest scalar product may be output as the final result. In this case, a value which is dependent on the amplitudes of the respective notes becomes particularly relevant, since loud tones then obtain more importance in the calculation of the scalar product. As a last step, a text file may be generated which contains all identified chords and/or chord types in a chronological order.
In addition, harmonies whose time durations are too short for them to be taken into account may be removed from the list, since a safe assumption may be seen in that matching chords do not change within intervals of milliseconds.
One major advantage of the previously suggested algorithm thus is the fact that it exhibits an upward compatibility for determining further chord types, which is possible simply by adding new reference vectors to the chord matrix.
For investigating the chord types occurring in the individual time segments of the piece of music to be examined, the above-described matrix is passed through on a column-by-column basis, and thus the current column, respectively, is examined as an input vector 302, as is depicted in
In the inventive approach, the input vector 302 is initially shifted, if required, such that the first element (i.e. that element which is at the foremost position of the reference vector) does not have the value of 0. Since in
For performing the first inversion, the input vector shifted in row 306 is then cyclically shifted to the left by 4 further elements, so that the value of 1 again occurs at a foremost, or first, position of the new shifted input vector. Such a shift by further 4 elements is depicted in
The fundamental tone of the chord type may then be determined by evaluating that shift value by which the input vector 302 was shifted to obtain the shifted input vector used to calculate, with the reference vector, the maximum scalar product. For the embodiment of input vector 302 which is selected in
As an alternative to the above-mentioned possibility of summing amplitudes of tones occurring in various octaves, a deletion of overtones may also occur. Each music note—except for pure synthetic sinus signals—consists of more than one frequency component. These additional components are known as overtones or harmonics and appear in the spectral representation at multiples of their fundamental frequencies. The difficulty with obtaining a reliable harmonics analysis from a frequency-domain representation of a music signal predominantly consists in identifying these overtones and using only the fundamental tone (in one octave), if possible, for finding the chord. Here, two further assumptions have been made for further chord recognition:
Particularly the second assumption has proven to be problematic, since in polyphonic music it is quite common for two fundamental tones to occur at the same time, the higher fundamental tone having a lower level than the lower fundamental tone, and the higher fundamental tone being an overtone of the lower fundamental tone. According to the above assumption, the higher fundamental tone would then have to be deleted. In chord recognition this may lead to the fact that when determining a type of chord, those tones which lie outside of an octave contemplated are not used for determining the chord type.
Depending on the circumstances, the inventive method for determining a chord type which underlies a test signal may be implemented in hardware or in software. The implementation may be effected on a digital storage medium, in particular a disc or CD with electronically readable control signals which can cooperate with a programmable computer system such that the respective method is performed. Generally, the invention thus also consists in a computer program product having a program code, stored on a machine-readable carrier, for performing the inventive method, when the computer program product runs on a computer. In other words, the invention may thus be realized as a computer program having a program code for performing the method, when the computer program runs on a computer.
In summary, it may thus be stated that a matrix may be formed, from a piece of music to be examined, which in the rows contains the halftones of an octave, and in the columns contains the time segments, i.e. the time frames of the piece of music to be examined, and that such values which correspond, with respect to their amplitude values, to those amplitudes which occur at the respective time segments and halftones may be entered into the respective matrix elements. To perform the chord recognition, the matrix may now be passed through on a column-by-column basis, and the respectively current column may be examined as a vector. The chord recognition is then based on comparing this vector, in all possible inversions, with reference vectors representing the various types of chords.
Initially, the input vector, i.e. a column of the matrix, may be shifted, if necessary, such that the first element has no value of 0. Then, the scalar products are formed with all reference vectors. Subsequently, this is also calculated for all inversions of the input vector. In a specific form, no evaluation will then need to be performed for all comparisons wherein the reference vector exhibits ones in elements in which the input vector has zeros. If this is true for all comparisons, the individual tones, for example, will be output from the algorithm introduced here. Otherwise, that reference vector which has the highest scalar product will “win”. The fundamental tone results from the number of elements by which the input vector has been shifted for the comparison with the highest result. In
While this invention has been described in terms of several preferred embodiments, there are alterations, permutations, and equivalents which fall within the scope of this invention. It should also be noted that there are many alternative ways of implementing the methods and compositions of the present invention. It is therefore intended that the following appended claims be interpreted as including all such alterations, permutations, and equivalents as fall within the true spirit and scope of the present invention.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10 2004 028 693 | Jun 2004 | DE | national |
This application is a continuation of copending International Application No. PCT/EP2005/004669, filed Apr. 29, 2005, designating the United States, and is not published in English, which claims priority of German Patent Application 10 2004 028 693.0-51, filed on Jun. 14, 2004, and is incorporated herein by reference in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4142433 | Gross | Mar 1979 | A |
| 4184401 | Hiyoshi et al. | Jan 1980 | A |
| 4300430 | Bione et al. | Nov 1981 | A |
| 4354418 | Moravec | Oct 1982 | A |
| 4412473 | Laflamme | Nov 1983 | A |
| 4467689 | Stier et al. | Aug 1984 | A |
| 4694280 | Rollhaus et al. | Sep 1987 | A |
| 5117727 | Matsuda | Jun 1992 | A |
| 5442129 | Mohrlok | Aug 1995 | A |
| 5459281 | Shibukawa | Oct 1995 | A |
| 5486647 | Kay et al. | Jan 1996 | A |
| 5756918 | Funaki | May 1998 | A |
| 5760325 | Aoki | Jun 1998 | A |
| 5864631 | Shutoh | Jan 1999 | A |
| 6057502 | Fujishima | May 2000 | A |
| 6541691 | Tolonen et al. | Apr 2003 | B2 |
| 7027983 | Puterbaugh et al. | Apr 2006 | B2 |
| 7342167 | Cremer et al. | Mar 2008 | B2 |
| 7346500 | Puterbaugh et al. | Mar 2008 | B2 |
| 7353167 | Puterbaugh et al. | Apr 2008 | B2 |
| 7460994 | Herre et al. | Dec 2008 | B2 |
| 7580832 | Allamanche et al. | Aug 2009 | B2 |
| 20060075884 | Streitenberger et al. | Apr 2006 | A1 |
| 20060075886 | Cremer et al. | Apr 2006 | A1 |
| 20080115658 | Fujishima et al. | May 2008 | A1 |
| 20090095145 | Streich et al. | Apr 2009 | A1 |
| 20090100990 | Cremer et al. | Apr 2009 | A1 |
| Number | Date | Country |
|---|---|---|
| 3023578 | Jun 1980 | DE |
| 1278182 | Jan 2003 | EP |
| 2003-263155 | Sep 2003 | JP |
| WO0104870 | Jan 2001 | WO |
| WO0188900 | Nov 2001 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20070144335 A1 | Jun 2007 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | PCT/EP2005/004669 | Apr 2005 | US |
| Child | 11607659 | US |
