Patent Grant
10614785
A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in a published Patent and Trademark Office patent file or record, but otherwise reserves all copyrights whatsoever.
This application claims the benefit of U.S. Provisional Application No. 62/563,669, filed Sep. 27, 2017, which is herein incorporated by reference in its entirety for all purposes. This application is also related to U.S. Pat. No. 5,606,144, which issued on Feb. 25, 1997, U.S. Pat. No. 9,286,876 which issued on Mar. 15, 2016, and U.S. Pat. No. 9,286,877, which also issued on Mar. 15, 2016. All of these patents are herein incorporated by reference in their entirety for all purposes, including all the computing devices, platforms, and applications disclosed in U.S. Pat. Nos. 9,286,876 and 9,286,877.
The invention relates to methods for creating mash-up variations of sequences of symbols, and more particularly, to methods for creating a mash-up variation of a piece of music or another symbol sequence whereby the mash-up variation differs from the original sequence(s) but nevertheless retains features of the original sequence(s). The term “mash-up” as used herein denotes a mixture or fusion of disparate elements, in accord with the generally accepted definition of the term (e.g., Oxford English Dictionary, Oxford University Press, 2018).
In my prior patents, U.S. Pat. Nos. 5,606,144, 9,286,876, and 9,286,877, and in other previous work a chaotic mapping technique is provided for generating variations on an existing work, for example, musical variations of a given musical work. It should be noted that the term “variation” is used herein to refer to any process by which some features of a work (such as a musical piece) change while other features remain the same. The term “variation technique” is used herein to refer to (1) any modification scheme, (2) any compositional procedure, or (3) any combination of elements (1) and (2).
An “improved chaotic mapping technique,” as described in my prior patents, produces musical variations of MIDI and MP3 files. The mapping in embodiments of my prior patents utilizes two chaotic trajectories from the Lorenz equations—a system comprising three nonlinear first order differential equations
dx/dt=σ(y−x) (1)
dy/dt=rx−y−xz (2)
dz/dt=xy−bz, (3)
where σ=10, b=8/3, and r=28 (E. N. Lorenz, “Deterministic nonperiodic flow,” J. Atmos. Sci. 20, 130-141 (1963)). Other embodiments use at least one scheme selected from the group consisting of (1) other chaotic systems, (2) probabilistic methods, (3) pattern matching, (4) machine learning, and (5) signal processing.
The Lorenz equations arise in applications ranging from lasers to private communications, and have also served as generators of “chaotic” music, where a chaotic system is allowed to free-run and its output converted into a series of notes, rhythms, and other musical attributes in order to create a piece from scratch. However, these approaches did not generate variations or mash-ups on an already completed piece.
The improved chaotic mapping of my prior patents utilizes a mapping strategy in conjunction with designated variation procedures to produce musical variations of MIDI (Musical Instrument Digital Interface) songs, as well as audio recordings, e.g., WAV and MP3. According to the improved chaotic mapping approach, chaotic algorithms are used to identify hole elements within an input series of notes or other elements. Algorithms are also used to identify substitution elements that are substituted for the hole elements.
The improved chaotic mapping of my prior two patents produces a rich array of MIDI and audio variations and mash-ups. Some embodiments from those two patents include:
Similarly, for j=9, first deduce the value of x2,9=15.24 from part D; find the least x1,i≥15.24 which is x1,9=15.26; take the value of the index i of x1,9 which is 9 and assign it to g(9)→g(9)=9. Applying the ‘hole-generating’ function to x2,1 and x2,9 yields blackened holes since j=g(j) in both cases. On the other hand, open holes denote those j for which j≠g(j). For example, for j=4, apply the function w(x2,j)=g(j) to determine the value of g(4): first deduce the value of x2,4=3.68 from part D; find the least x1,i≥3.68 which is x1,5=6.40; take the value of the index i of x1,5 which is 5 and assign it to g(4)→g(4)=5. Since j≠g(j), i.e., since 4≠g(4), an open hole results for location 4 in the Hole Template. Likewise, for Hole Template locations 5-7 and 16, it can be verified that j≠g(j), and so open holes arise in these locations as well.
Blackened holes indicate no change from the source Bach, and open holes indicate changes will occur in those locations. The notes that will change—Nc,4, Nc,5, Nc,6, Nc,7, and Nc,16—are indicated above the open holes. Each Nc,j denotes the pitch number of the jth pitch of the source piece for which j≠g(j). The unchanging notes of the Bach and the prior event pitches Ep,3 and Ep,15, which will serve as the reference pitches for the Dynamic Inversion process explained in parts F and G, are marked above the blackened holes. For example, prior pitch event Ep,3 represents the pitch immediately prior to four ‘open holes’ in the variation; here Ep,3 corresponds to the third blackened hole=G4. This prior pitch event Ep,3=G4 functions as the reference pitch for the Dynamic Inversion of those Bach input file pitches 4-7, i.e., Nc,4, Nc,5, Nc,6, and Nc,7, which will all be inverted about the G4. Thus, the Nc,j's will be inverted about the pitch occupying the previous blackened hole—here, G4. In the Dynamic Inversion procedure, reference pitches are defined by, and change according to the Hole Template of improved chaotic mapping. Thus, each differs from any other reference pitch, e.g., Ep,3≠Ep,15 so that the reference pitches are dynamic rather than fixed or static. By contrast, in past and current commercial computer practice, inversions are taken about a reference pitch that is fixed, e.g., middle C (C4) or a user-supplied reference pitch.
The process of Dynamic Inversion is set up in
Since j=g(j) for the first three events of the variation, as already determined by the Hole Template of part E, the original Bach pitches will occur in locations 1-3 of the variation. The same is true for the 8th-15th pitches of the variation. However, since j≠g(j) for j=4, . . . , 7, 16, the variation will change from the source according to the expression for improved chaotic Mapping working in conjunction with said Dynamic Inversion procedure:
where g(j) is assigned the value of the index i of the least x1,j for which x2,j≤x1,i. Here, Dynamic Inversion is used to calculate each new pitch PN of the variation using the expression N=−(Nc,j−Ep,j−1)MOD12+Ep,j−1, where Nc,j denotes the pitch number of the jth pitch of the source piece for which j≠g(j), and Ep,j−1 is defined in terms of Nc,j as the pitch number of the (j−1) pitch of the original pitch sequence that occurs before one or more consecutive Nc,j's. As stated earlier, the prior event Ep,j−1 serves as the dynamic reference pitch about which the inversion of one or more consecutive Nc,j's will occur.
To find p′4, the fourth pitch of the variation, calculate PN where N=−(Nc,4−Ep,3)MOD12+Ep,3=−(72−67)+67=62→P62 D4=p′4, the 4th pitch of the variation. Pitches p′5, p′6, and p′7 result from a similar procedure. The mod 12 option did not apply in determining p′4. Though it was not necessary here, it can be invoked to preclude inversions that move up/down from any reference pitch by more than an octave. To see how the G #4 is assigned to p′16, again calculate PN where N=−(Nc,16−Ep,15)MOD12+Ep,15=−(76−72)+72=68→P68=G #4=p′16.
Applying improved chaotic Mapping in conjunction with Dynamic Inversion yields the variation sequence displayed in
where E′j represents any event of the variation, ei=g(j) denotes any of pitch, chord, phrase, beat, note rhythmic value, note-group, musical event from the source work, and combinations thereof, that will appear unchanged in the variation as a result of the condition j=g(j), and Ej represents any musical event in the variation produced by improved chaotic mapping in conjunction with a designated variation procedure whenever j≠g(j). The term g(j) is assigned the value of the index i of the least x1,i, for which x2,j≤x1,i.
In Block [1], the first chaotic trajectory {x1,i, y1,i, z1,i}, indexed on i, with initial conditions (x1,1, y1,1, z1,1) is launched. A second chaotic trajectory {x2,j, y2,j, z2,j}, indexed on j, with initial conditions (x2,1, y2,1, z2,1) is simulated in Block [2]. The hole-generating function w(x2,j)=g(j) of Block [3] takes each x-value of the second chaotic trajectory (possibly including y-, z-values of same) and determines g(j), where g(j) is assigned the value of the index i of the least x1,i such that x2,j≤x1,i. The hole-generating function creates the Hole Template of Block [4] according to whether or not j=g(j). If so, a blackened hole appears at the appropriate j; if not, an open hole occurs.
Block [4] shows a hypothetical example of a Hole Template (resulting from Block [3]), where M represents any given integer. Here, M is the hypothetical value of the leftmost j in the plotting. Suppose that applying the hole-generating function, w(x2,j)=g(j), to x2,j results in j=g(j), and equivalently M=g(M) for the j=M location in the drawing. Thus, for j=M, a blackened hole results. Suppose further that the same process applied to the next x-value x2,j, where j=M+1, also results in j=g(j)→M+1=g(M+1). Then for j=M+1, a blackened hole appears. As stated earlier, blackened holes denote those j for which j=g(j). Here, blackened holes result from applying the hole-generating function to the two x-values x2,M and x2,M+1, as well as to the x-values x2,M+5 and x2,M+6. But now suppose that for x2,j where j=M+2, the hole-generating function returns a value for g(j) that does not equal j. Thus an open hole occurs for j=M+2. Ditto for j=M+3 and j=M+4.
Block [5A] supplies the event sequence {ei} of a source work-which can include MIDI events, audio events, or both—to improved chaotic mapping of Block [5B]. Note that if the event list consists of more than one musical attribute (e.g., MIDI pitch, onset time, and velocity), each can be varied separately, or together, by applying this exemplary algorithm to one or more axes in 3-space, or to n axes in n-space, e.g., where n axes result from harnessing additional chaotic systems.
Improved chaotic Mapping is applied in Block [5B] whenever j=g(j), i.e., at the blackened holes. This results in events ei=g(j) occurring in the same spot in the variation as in the source piece. Thus for this hypothetical example, original events eM, eM+1, eM+5, eM+6 fill the blackened holes E′j=M, E′j=M+1, E′j=M+5, and E′j=M+6 in the variation shown in Block [7].
Block [6A] provides a designated variation procedure which will work in tandem with improved chaotic Mapping of Block [6B] to generate new events Ej whenever j≠g(j), i.e., at the open holes. Thus, for this hypothetical example, new events EM+2, EM+3, and EM+4 fill the open holes and are equivalent to E′j=M+2, E′j=M+3, and E′j=M+4, in the variation shown in Block [7].
Block [7] contains the variation's event sequence {E′j} which comprises the sum of Blocks [5B]+[6B]. The contents of the blackened holes remain unchanged from the source event sequence. The open holes are filled with new events Ej supplied by improved chaotic mapping in conjunction with a designated variation procedure(s). The variation is produced by merging the contents of the blackened and open holes to give the variation's final event sequence {E′j}={ . . . , eg(M), eg(M+1), EM+2, EM+3, EM+4, eg(M+5), eg(M+6), . . . }.
In the embodiments described above, the hole-generating function of improved chaotic mapping produces a Hole Template where the number of open vs. blackened holes can vary depending on the initial conditions for the chaotic trajectories 1 and 2. In other embodiments, other schemes such as probabilistic methods can produce a Hole Template capable of receiving new events via, for instance, dynamic inversion. One inherent advantage of improved chaotic mapping over a probabilistic scheme lies in the fact that improved chaotic mapping has several built-in ‘controls’ or ‘sliders’ that determine the amount of variability—all arising from a natural mechanism for variability present in chaotic systems, i.e., the sensitive dependence of chaotic trajectories to initial conditions. Thus, the degree to which the initial conditions chosen for the second chaotic trajectory differ from those assigned to the first trajectory will directly affect the amount of variability present in a variation.
Parsing
Where the input to the improved chaotic mapping is a continuous function, it can be automatically divided or “parsed” into a sequential series of original elements which are segments of the input that can include (1) eighth note beats, (2) quarter note beats, (3) groups of eighth note beats, (4) groups of quarter note beats, and/or (5) combinations thereof, through automated detection of any of time signature beats, boundaries between musical phrases, and repetitive structures.
Automatic parsing of an input into segments or events such as segments comprising specified numbers of eighth-note or quarter-note beats, e.g., groups of 4 quarter note beats, can be accomplished by any number of audio beat detection methods. These give a sequence of timings that can then be used to cut the audio file into desired groupings, also referred to herein as “parses”, “events”, and elements (said terms being used herein as synonyms).
For example, detection of quarter note divisions allow groups of 2, 4, 8 or more quarter note divisions to function as parsed events. Each of the parses can be delineated by a start time plus a duration. A number of papers have been written on beat detection, e.g., “Multi-Feature Beat Tracking” [J. Zapata, M. Davies, and E. Gómez, IEEE/ACM Transactions on Audio, Speech, and Language Processing, Vol. 22, No. 4, April 2014]. There are many sites online that feature various open-source algorithms, e.g., Essentia and Queen Mary vamp plugins for Audacity.
Musical Variation, Remixes, and Mash-Ups.
Musical variation occupies a storied place in music, from the lutenists of 16th c. Spain to the remixes and mash-ups of today. In every society past and present, remix and mash-up variations spin contemporary songs into fresh ones.
For example, whenever Bach and his large musical family got together, they sang popular songs both comic and lewd-all at the same time. In composing his Goldberg Variations, Bach combined several German folksongs to create his final variation.
Today DJs carry on the tradition, remixing and mashing White Albums with Black Albums to create something new, like the Grey Album by Danger Mouse.
Many of today's songs are made by laying down tracks that are then combined to produce a full song. Thus each track is a “component” of the full song. First the instrumental track might be recorded. Then the vocalist(s) sings the song melody and records the solo vocal track. Finally, the component tracks, i.e. the solo vocal track and the instrumental track, combine to make the full song. But once the full song has been recorded, the vocal track by itself and the instrumental track by itself have little value, except for DJs and remix/mash-up aficionados. They take tracks and re-combine them in new and interesting ways, while adding effects like filtering, flanging, scratching, beat repeating, echo/delay, reverb, etc. In short, they take audio tracks and use digital audio workstations (DAWs) to create new takes, interpretations, and versions of the original song(s). But DJ skill requires time, effort, and money.
For the purposes of this description and the accompanying claims, “component”, also referred to as “component track”, means a track that is included in a musical composition, such as (1) an instrumental track, (2) a vocal track, (3) a percussion track, (4) any associated track contributing to the song, 5) any added track to a song, or (6) any combination(s) thereof. A “composition” means a work such as (1) a written musical score, (2) sound recording, (3) written text, (4) spoken text, (5) any component track of a song, or (6) any combination(s) thereof.
Prior work offered methods and apparatus for generating musical variations as shown in “Method and Apparatus for Computer-Aided Variation of Music and other Sequences, including variation by Chaotic Mapping” U.S. Pat. No. 9,286,876 and U.S. Pat. No. 9,286,877 (CIP). These methods enabled the process of creating song variations with the click of a few buttons, thus opening up the creative process to anyone with a digital device, computer, or mobile phone.
In accordance with prior work methods, a mash-up variation was made by beat-matching two different songs, concatenating both, then parsing the concatenated file, after which the improved chaotic mapping is applied, thus producing holes which are replaced with elements of both songs, some of which elements may have undergone a designated variation procedure. But these prior work methods do not teach application of the improved chaotic mapping method to the process of structuring the mash-up in the first place.
Accordingly, despite the successes of my improved chaotic mapping method, the mash-ups produced according to these prior work strategies can sometimes be limited in structure.
What is needed, therefore, is a method that emphasizes structure amidst variation when producing a mash-up of ordered inputs such as musical inputs, so as to produce a mash-up with discernible structure that enables it to be perceived as a song in its own right.
A method is disclosed for varying musical compositions and other ordered inputs to create hybrid variations, whereby at least two songs and/or other inputs such as sound tracks are combined using the improved chaotic mapping of my prior art, in tandem with new methods of the present invention to create a new song or other output that is a so-called “mash-up.” Embodiments present the resulting mash-up to a user in a printed score, recorded audio format, or both. The disclosed method emphasizes structure amidst variation when producing the mash-ups, so as to produce mash-ups with the discernible structure of a song. Some of these new methods make use of the by-products of the song production process-vocal and instrumental tracks-thus allowing artists and record companies additional revenue streams from what were formerly cast-off component tracks.
Embodiments of the invention make use of the improved chaotic mapping method and other prior work described above, but applied in novel ways.
The disclosed method creates a mash-up of at least two inputs, which are generically referred to herein as songA and songB, although they need not be musical inputs. According to the disclosed method, songA and the at least one songB are first “parsed,” i.e., divided into distinct elements, which can be elements of equal length, such as beats in a musical input. In embodiments, an input song that is a continuous function is automatically divided or “parsed” into a sequential series of elements which are segments of the input such as (1) eighth note beats, (2) quarter note beats, (3) groups of eighth note beats, (4) groups of quarter note beats, and (5) combinations thereof, through automated detection of any of time signature beats, boundaries between musical phrases, and repetitive structures.
The songs are then “beat matched,” by shrinking or stretching each beat of songB so that it is equal in length to the corresponding beat of songA, beat-by-beat in sequence. In some embodiments, the song tracks are aligned by performing a null period process on a selected one of the inputs, where the null period process can be (1) adding a null period to the selected input; (2) deleting a null period from the selected input; and (3) any combination(s) thereof. For example, the vocal track of songB could be aligned so that it enters where the vocal track of songA would have entered, or if an upbeat is present, the downbeat of the vocal track of songB would occur where the initial downbeat of the vocals in songA would have occurred. Finally, the songs are combined to form a mash-up using any of several mash-up algorithms disclosed herein. In some embodiments, improved chaotic mapping is used to vary textures by which the parsed elements of the songs are combined, where texture in general “refers to the many [features] of music, including register and timbre of instrumental combinations, but in particular, it refers to music's density (e.g., the number of voices and their spacing)” as described in (S. Laitz. The Complete Musician, 2nd edition. Oxford, Oxford University Press, 2008).
In one general aspect of the present invention, referred to herein as the MUiMUv family of algorithms, the mash-up algorithm comprises using the improved chaotic mapping method to substitute elements selected from the components of at least one songB in place of selected elements from the components of songA.
In embodiments, after parsing and beat matching of the songs, improved chaotic mapping is used to select elements from the component tracks of songA and songB, whereby selected “hole” elements in the component tracks of songA are replaced by selected replacement elements from the component tracks of one or more songB's to produce modified component tracks of songA which have been infused with elements from the component tracks of songB. These infused component tracks of songA combine to produce a mash-up. Depending on the embodiment, any of several methods can be applied to modify the replacement elements before the substitution is made. In embodiments, the song tracks are aligned according to any of several criteria before the substitution elements are selected.
In embodiments, each of songA and songB comprises a plurality of component tracks. For example, suppose songA and songB are each a musical input that includes a vocal track and at least one instrumental track. In some of these embodiments improved chaotic mapping is applied separately to create a separate instrumental track mash-up of the instrumental tracks from songA and the at least one songB, as well as a separate vocal track mash-up of the vocal tracks from songA and the at least one songB. The two track mash-ups are then superposed to form the final mash-up. The term MUiMUv that is applied to this family of embodiments is derived from this approach and stands for “Mash-up Instrumental Mash-upVocal.”
In similar embodiments, track mash-ups can be created from tracks of differing types and then superposed. For example, elements of a vocal track of songB can be selected and used to replace selected elements of the instrumental track of songA.
In a second general aspect of the present invention, referred to herein as the muMap family of embodiments, after beat matching of the songs and, in embodiments, after alignment of the songs, the mash-up is created by introducing groups of parsed elements from component tracks of songA and groups of parsed elements from component tracks of songB into successive intervals or frames of a mash-up template in a staggered fashion. The combination of elements that is presented in each frame of the mash-up can be determined for example by (1) a fixed pattern of textures, (2) a desired pattern of textures, (3) a sequence of textures determined by an algorithm applied in conjunction with improved chaotic mapping, and (4) combinations thereof.
If, for example, each of the two songs includes two tracks, and if it is assumed that the parsed elements of the songs' tracks are to be maintained in relative alignment (i.e., without temporal shifting of the parsed elements) then there are a total of sixteen possible combinations of the four tracks, parses of which can be presented in each frame of the mash-up (no parses from tracks, parses from any one of four separate tracks, parses from any of six pairs of tracks, parses from any of four combinations of three tracks, and parses from all four tracks occurring simultaneously). The muMap approach can be combined with MUiMUv to produce even more complex mash-ups.
A repetition threshold can be established whereby some frames of the template repeat, while others are skipped. If songA is to be combined with a plurality of songB's to produce a plurality of mash-ups, then the order in which the combinations appear can be shifted or “rotated” so that greater variety is obtained.
It should be noted that temporal shifting, or rearrangement, or both temporal shifting and rearrangement, of the parses of songB relative to songA can be included in any of the mash-up approaches disclosed herein, not only for purposes of aligning the songs but also to provide for a wider range of possible mash-up variations.
It should be noted that, while much of the present disclosure is discussed in terms of mash-ups, the present invention is applicable to both remixes and mash-ups. Accordingly, the term “mash-up” is used herein generically to refer to both remixes and mash-ups, except where the specific context requires otherwise.
The present invention is mainly described in this application in terms of its application to musical symbols or notes, following the illustrations of my said prior patents. However, it should be noted that the present invention is not limited to mash-up variations of music, and that embodiments of the present invention are generically applicable to all types of symbols, characters, images, and such like.
A first general aspect of the present invention is a method practiced by a computing device for automatically creating an output, referred to herein as a mash-up, by combining elements derived from at least two inputs. The method includes accepting a first input comprising songA and a second input comprising songB; parsing songA into a series of consecutive songA elements; parsing each songB into a series of consecutive songB elements, wherein each songA element corresponds to a songB element; if each of the songA elements is not equal in length to its corresponding songB element, beat-matching songB with songA by adjusting lengths of at least one of the songA elements and the songB elements so that all songB elements are equal in length to their corresponding songA elements.
The method further includes combining songA with songB, in order to create a mash-up, said combining comprising application of at least one scheme selected from the group consisting of:
The method further includes presenting the mash-up to a user.
In embodiments, songA and songB are musical compositions or recordings.
In any of the above embodiments, after beat-matching, all of the songA elements and songB elements can have the same length.
Any of the above embodiments can further comprise modifying at least one of the replacement elements before the hole elements are replaced by the substitution elements.
In any of the above embodiments, songA can include a first plurality of song tracks and songB includes a second plurality of song tracks, so that each of the songA elements and songB elements comprises a plurality of song track elements, all of the song track elements within a given songA or songB element being equal in length. In some of these embodiments, applying improved chaotic mapping includes applying improved chaotic mapping separately to pairs of song tracks, each of the pairs comprising one song track from songA and one song track from songB, so that the mash-up includes at least one song track of songA in which elements thereof have been replaced by elements from a song track of songB.
In any of the above embodiments that further include modifying at least one of the replacement elements before the hole elements are replaced by the substitution elements, the song tracks of songA can include a song track of a first kind, referred to herein as an instrumental track, and a song track of a second kind, referred to herein as a vocal track, and wherein the song tracks of songB include an instrumental track and a vocal track. In some of these embodiments applying improved chaotic mapping to songA and songB includes applying improved chaotic mapping to the instrumental track of songA and the instrumental track of songB, and separately applying improved chaotic mapping to the vocal track of songA and the vocal track of songB. In other of these embodiments applying improved chaotic mapping to songA and songB includes applying improved chaotic mapping to the instrumental track of songA and the vocal track of songB, and separately applying improved chaotic mapping to the vocal track of songA and the instrumental track of songB.
Any of the above embodiments can include a plurality of songB's from which the replacement elements are selected.
Any of the above embodiments can further include aligning songB with songA by performing a null period process on a selected one of the inputs, the null period process comprising at least one of:
A second general aspect of the present invention is a method practiced by a computing device for automatically creating an output, referred to herein as a mash-up, by combining elements derived from a plurality of inputs. The method includes accepting a plurality of N inputs, the inputs being denoted as song(i) where i is an integer ranging from 1 to a total number N of the inputs, each of the song(i) comprising a plurality of song tracks; for each i, parsing song(i) into a series of consecutive song(i) elements; if all of the consecutive song(i) elements are not of equal length, adjusting the consecutive song(i) elements so that they are all of equal length, said equal length being denoted as L(i); beat-matching the inputs by adjusting at least one of the L(i) such that all of the L(i) are equal to the same value L; creating a mash-up template divided into consecutive mash-up frames of length k times L, where k is an integer, the mash-up template comprising a plurality of parallel mash-up tracks, each mash-up track being divided into a plurality of consecutive track frames of length k times L.
The method further includes creating the mash-up by sequentially introducing elements from the song tracks of the inputs into the track frames of the mash-up template, so that each successive template frame of the mash-up template is populated by a combination of corresponding elements derived from the song tracks of the inputs, where said combination of corresponding elements can be derived from any number of the song tracks from zero up to the combined total number of the song tracks of the inputs and presenting the mash-up to a user.
In embodiments of this general aspect, the inputs are musical compositions or recordings.
In any of the above embodiments of this general aspect, the number of mash-up tracks in the mash-up template can be less than or equal to the combined total number of song tracks in the inputs.
In any of the above embodiments of this general aspect, the mash-up frames can include a beginning group thereof that are successively populated, such that each of a first group of one or more mash-up frames in the beginning group contains at least one corresponding element from only one song track, said first group being followed by a second group of one or more mash-up frames in the beginning group, each containing two corresponding elements from two song tracks, and so forth until at least one mash-up frame in the beginning group contains a corresponding element from each of the song tracks of the inputs.
In any of the above embodiments of this general aspect, the combinations of corresponding elements that populate the track frames can vary from mash-up frame to mash-up frame according to a specified pattern. In some of these embodiments, the pattern is repeated after a specified number of frames.
In any of the above embodiments of this general aspect, the combinations of corresponding elements that populate the track frames can be determined using improved chaotic mapping. In some of these embodiments, the combinations of corresponding elements that populate the track frames are determined with reference to a Rotating State Option Implementation Table, according to a series of ‘Left-hand’ or ‘Right-hand’ path options.
In any of the above embodiments of this general aspect, the mash-up can be terminated by a Coda group of mash-up frames in which corresponding elements from the tracks of the inputs are successively eliminated until a mash-up frame in the Coda group includes only one corresponding element.
Any of the above embodiments of this general aspect can further comprise modifying at least one of the corresponding elements before introducing it into a track frame.
A third general aspect of the present invention is a method of creating a plurality of mash-ups. The method includes successively applying the method of any of the embodiments of the previous general aspects to a plurality of inputs, wherein the combinations of elements introduced into the mash-up template are repeated in an order that is rotated from one mash-up to the next.
The mash-up of any of the embodiments of any of the general aspects can be combined with a graphical element. The graphical element can be a graphical image, a video, or a part of a video, a film or a part of a film, a video game or a part of a video game, a greeting card or part of a greeting card, a presentation slide element or a presentation slide deck. The graphical element can be an element of a storyboard that describes a proposal for a musically accompanied graphical work, where the musically accompanied graphical work can be a musically accompanied video. The graphical element can be a slide presentation created by a presentation software application, where the presentation software can be configured to perform the steps of any of the above embodiments for creating a mashup.
Any embodiment of any of the general aspects that includes combining a mash-up with a graphical element can further include forwarding the combined mash-up and graphical element to at least one recipient.
In any of the above embodiments of any of the general aspects, the song inputs can be associated with a website, and the website can be configured, each time a user visits a designated page of the website, to repeat the steps according to any of the above embodiments to create a new mash-up of the song inputs; and play the new mash-up to the user.
In any of the above embodiments of any of the general aspects, the song inputs can be associated with software, the software being configured, each time a user activates the software, to repeat the steps recited in claim 1 or claim 12 to create a new mash-up of the inputs, and present the new mash-up to the user.
Any of the above embodiments of any of the general aspects can further comprise storing the mash-up in a digital device. The digital device can be a greeting card that is configured to present the mash-up to a user when the greeting card is opened.
In any of the above embodiments of any of the general aspects, a digital device can be configured to perform the steps of the embodiment to create the mash-up. In some of these embodiments, the digital device is included in a greeting card, a toy, an MP3 player, a cellular telephone, or a hand-held or wearable electronic device. And in other of these embodiments the digital device is configured to repeat the steps of the embodiment to create a new mash-up of the inputs each time the inputs are accessed, so that a new mash-up is presented each time the inputs are accessed.
In any of the above embodiments of any of the general aspects, the method can be practiced by an application running in hardware or software on a hand-held or wearable electronic device, by a computing device that is accessible via a network to a hand-held or wearable electronic device, or by a computing module included in a hand-held or wearable electronic device, which can be configurable to play a new mash-up of the inputs each time the input musical compositions are selected, or automatically from a plurality of input songs. The computing module can be included in a hand-held or wearable electronic device and can be configurable to play a new mash-up or variant of a mash-up of the input musical compositions each time the input musical compositions are selected.
And in any of the above embodiments of any of the general aspects, a plurality of tracks from input musical compositions can be made accessible to a hand-held or wearable electronic device, and the hand-held or wearable electronic device can be configured to enable a user to access subsets of the tracks as the inputs and to create therefrom the mash-up.
The features and advantages described herein are not all-inclusive and, in particular, many additional features and advantages will be apparent to one of ordinary skill in the art, in view of the drawings, specification, and claims.
Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes, and not to limit the scope of the inventive subject matter.
The current application builds on the earlier work described above, improving upon it by offering methods for generating mash-ups, which include remixes, by using mash-up algorithms disclosed herein to combine elements from a plurality of musical works or other inputs. In embodiments, the mash-up algorithms can include application of improved chaotic mapping. Furthermore, some of these new methods can make use of the by-products of the song production process—vocal and instrumental tracks—thus allowing artists and record companies additional revenue streams from what were formerly cast-off song component tracks.
It should be noted that, while much of the present disclosure is discussed in terms of mash-ups, the present invention is applicable to both remixes and mash-ups. Accordingly, the term “mash-up” is used herein generically to refer to both remixes and mash-ups, except where the specific context requires otherwise.
Similarly, much of the discussion presented herein is directed to the mash-up of two hypothetical musical works songA and songB. However, it will be understood that the method disclosed herein can be applied to any ordered sequences of inputs.
With reference to
Beat-Matching
After both songs have been parsed (200 in
Once songA and songB have been parsed 200 and beat matched 202, variation algorithms 204 are applied to each of them, and they are combined using a mash-up algorithm 206.
Mash-Up Algorithms: MUiMUv
With reference to
Then, improved chaotic mapping is applied [9]-[13] separately to the instrumental and vocal tracks of songA and songB to create variations of the tracks [14]-[17], after which improved chaotic mapping is used to combine elements from the two songs [18] by, in the case of MUiMUv, substituting parses from the instrumental track of songB in place of parses of the instrumental track of songA, then substituting parses from the vocal track of songB in place of parses of the vocal track of songA, and combining these together. Or in the case of muMap, by combining tracks 14-17 according to a “map” or template of possible textures, e.g., instrumental and vocal combinations, as discussed in more detail below.
In embodiments of MUiMUv, wherever a changed parse occurs due to improved chaotic mapping, e.g., in the jth parse of songA's instrumental track, the commensurate jth parse of songB's instrumental is inserted. Similarly, wherever a changed parse occurs in songA's vocal track, i.e., when j≠g(j) for any jth parse, the jth parse of songB's vocal track is inserted. In this way, an instrumental track that mixes the two songs' instrumental tracks and a vocal track that mixes the two songs' vocal tracks are created. Then these two tracks are combined to produce a mash-up of the two songs [19], which is presented to a user. For example, the mash-up can be directed to an amplifier [20], a mixer [21], and speakers [22] for audio presentation.
The name “MUiMUv” that is used to refer to this general aspect comes from how the algorithm works. Improved chaotic mapping mashes two different instrumental tracks to create an “Instrumental Mash-up” track “MUi”, and mashes two different vocal tracks to create a vocal mash-up track “MUv”, after which the two resulting tracks (MUi and MUv) are combined to produce the final mash-up (MUi+MUv).
In Blocks [6]-[8], the parse times found for songA are applied to the instrumental track of A (instA), and the new re-calculated parse times found for the beat-matched songB are applied to the instrumental track of B (instB). Similarly, the parse times found for songA are applied to the vocal track of A (vocA), and the new re-calculated parse times found for the beat-matched songB are applied to the vocal track of B (vocB).
Before improved chaotic mapping can be applied in Blocks [11] and [12], the two vocal tracks have to be aligned so that, for example, the vocals in songB enter where the vocals in songA would have entered, (or the vocals in songB can enter over instA at the moment in time vocB would have entered in songB.) Usually silence precedes any vocal audio on a vocal track because songs usually start with an instrumental introduction. To measure the duration of any silence preceding the vocal entry on vocA, a threshold can be set to identify in which parse the voice(s) starts as well as the duration of the preceding silence. Then silence can be either added to, or removed from, vocB, for example in quantities equal to the length of a quarter note, until the initial silence of vocB equals the same number of quarter note events as the initial silence of vocA, presuming that it is desired for vocB to align with vocA, (Block 9). Similarly, the first downbeat of instB can be aligned with the first downbeat of instA, (Block 10).
Once the two vocal tracks are aligned, parse-by-parse, improved chaotic mapping is applied to substitute parses from vocB in place of parses of vocA, (Block 11). Whenever j≠g(j) for a given set of initial conditions, the jth parse of vocA is replaced by the jth parse of vocB to create a new vocal track “MUv”.
Similarly, once the two instrumental tracks are aligned, parse-by-parse, improved chaotic mapping is applied to the parses of instA. Whenever j≠g(j), typically for a different set of initial conditions than those used for mashing the vocals, the jth parse of instA is replaced by the jth parse of instB to create a new instrumental track “MUi”, (Block 12).
Finally, the mashed instrumental track MUi is combined with the mashed vocal track MUv to produce the final mash-up, MUi+MUv (Block [13]), which is presented to a user, for example by directing the mash-up to an amplifier, mixer, and speakers, (Blocks [14]-[16]).
It will be understood that the scope of the present invention includes many other mash-up methods that are variations of the MUiMUv method. For instance, MUi could be produced by alternating parses of instA with parses from instB so that every other parse of MUi comes from instA and the intervening parses are derived from instB. MUv could be made in a similar manner. Then MUi and MUv could be superposed to give the mash-up comprising MUi+MUv.
As another example, “MUiavbMUibva” uses improved chaotic mapping, in conjunction with a set of parameters including initial conditions, to mash instA with vocB to produce a first mash-up MU1. Then, improved chaotic mapping is used to mash instB with vocA using improved chaotic mapping to produce a second mash-up MU2. Lastly, improved chaotic mapping is used to mix MU1 with MU2 to produce a final mash-up MU3. For MU1, the alignment can be adjusted so that vocB aligns with instA exactly where vocA would have entered in songA. For MU2, the alignment can be adjusted so that vocA aligns with instB exactly where vocB would have entered in songB.
In embodiments, the overall structure of songA is preserved by adjusting MUiMUv so that only the first verse (“verse1”) of songA mashes with versel of songB, verse2 mashes with verse2, and the hooks mash as well. Embodiments incorporate machine learning and signal processing methods that allow automated identification of the structures of songA and songB.
MUiMUv can create a mash-up that can be combined with a graphical element and forwarded to at least one recipient, where the graphical element can be, for example, any of a graphical image; a video or part of a video; a film or part of a film; a video game or part of a video game; a greeting card or part of a greeting card; a presentation slide element or presentation slide deck; a slide presentation created by a presentation software application, wherein the presentation software application is configured to perform the steps of any of the MUiMUv algorithms disclosed herein; an element of a storyboard that describes a proposal for a musically accompanied graphical work such as a musically accompanied video.
The MUiMUv method can also be applied to inputs associated with a website or software program so that each time a user visits a designated page of the website or the software program, the website or software program is configured to operate on the inputs to produce a mash-up and play/present it to the user.
MUiMUv can also be implemented in a digital device that is installed for example in a greeting card that is configured to produce and present a mash-up when the greeting card is opened. Such a digital device can also be included within a toy, an MP3 player, a cellular telephone, or a hand-held or wearable electronic device. The digital device can produce a new mash-up of the inputs each time the inputs are accessed, or automatically from a plurality of input songs.
MUiMUv can be embedded in an application or computing module running in hardware or software on a hand-held or wearable electronic device, or on a network to a hand-held or wearable electronic device, any of which are configurable to play a new mash-up or variant of a mash-up of input musical compositions each time input musical compositions are selected, from a plurality of inputs supplied by the application or other users, manually or automatically, including mash-ups successively created in a chain of users or machines. The MUiMUv method can also be practiced by a computing module included in a hand-held or wearable electronic device wherein a plurality of tracks from input musical compositions are accessible to a hand-held or wearable electronic device, and the hand-held or wearable electronic device is configured to enable the user to access subsets of the tracks as the inputs and to create therefrom a mash-up.
muMap
The MUiMUv algorithm and its variants as described above are particularly suited for creating mash-ups of songs that have the same or similar structures. Another general aspect of the present invention incorporates a mash-up algorithm referred to herein as muMap, which is highly effective for creating mash-ups of songs that have different structures, for example if songA proceeds from verse1 to a hook, whereas songB proceeds from verse1 to verse2 and then to a hook, so as to create a mash-up with an overall structure that can be readily discerned by a listener.
After parsing and beat-matching, the “muMap” algorithm uses the vocal and instrumental tracks of songA and songB to create a mash-up. In its simplest form, it offers a series of textures progressing from one track to two tracks, then three, and so forth. In embodiments, this continues until the textures include all of the tracks of all of the songs. For example, if a mash-up is created using two songs as inputs with two tracks per song, then in embodiments the applicable textures are one track, two tracks, three tracks, and finally four tracks. After all four tracks enter, the texture reduces to different combinations of tracks, chosen to provide contrast to what came before and what follows.
An example of an application of the muMap algorithm is presented in
According to the example of
After the Introduction, different combinations of tracks ensue, as shown in
As shown in
To ensure alignment after parsing and beat-matching, vocB can enter at an elapsed time determined by when it originally entered in songB. This gives three advantages:
Another example of the application of the muMap algorithm is presented in Table 1 below. According to this example, all blocks, except perhaps the 0th block (opening block), consist of parses equivalent to 8 quarter-note groups. Block 1 is defined as the block containing the parse where vocB enters. To identify the parse where vocB enters, a threshold value can be set. Once vocB crosses that threshold, the parse where the vocal(s) of vocB starts is considered Block 1.
After the Introduction section comprising Blocks 0-4, improved chaotic mapping can operate on the parses of songA, e.g., grouped in 16-quarter-note parses corresponding to the length of each different texture given by muMap, to determine when the current texture should move to the next textured ‘state.’ In embodiments whenever j≠g(j), the current state moves to the next state according to a pre-determined texture order, such as the order depicted in the muMap of
For instance, after the Introduction (Blocks 0-4) in the muMap of
Or, once all 4 tracks have entered (constituting the end of the Introduction), improved chaotic mapping can be used to shift from one state to another, not in any predetermined order, but according to a state implementation table, such as Table 2 below, which compares j vs. g(j) with respect to each of the three x, y, and z variables, where 0 signifies no change, i.e., when j=g(j), and 1 signifies change, i.e., when j≠g(j).
In this example, to create the State Implementation Table, improved chaotic mapping operates on a list of events, e.g., the 16-quarter-note parses of songA that occur after the Introduction, and calculates j vs. g(j) not only with respect to the x-variable (as occurs in
Improved chaotic mapping can also be used to vary individual tracks comprising the original songA and songB, as shown earlier in
But to ensure that the same instrumental variations do not occur in every mash-up that results from using songA as the first song, improved chaotic mapping can operate on the parses of songB in order to apply the resulting j vs. g(j) values to the parses of instA to determine which g(j)th parses of instA will substitute for the associated jth parses of instA. In this way, variations of instA will always differ according to which songB is being mashed with songA.
The varied individual tracks can also incorporate signal processing techniques, such as convolution, as well as effects sometimes used by professional “disk jockey” (“DJ”) music presenters, such as scratching, increasing tempo, decreasing tempo, filtering, and combinations thereof, according to improved chaotic mapping's j vs. g(j) values and an appropriate implementation table. Again, running improved chaotic mapping on the parses of songB to produce the j vs. g(j) values to be applied to the parses of songA will ensure distinct variations of songA tracks from mash-up to mash-up. In the Change Implementation Table (Table 3) below, which compares j vs. g(j) across all 3 variables of x, y, and z, where 0 signifies no change, i.e., when j=g(j), and 1 signifies change, i.e., when j≠g(j), the term “convolve” refers to the convolution of song tracks such as instA and instB, while “8×” refers to decreasing the tempo 8-fold for the first eighth-note of whichever instrumental track is being varied.
To create the Change Implementation Table of Table 3, improved chaotic mapping operates on a list of events, e.g., the 16-quarter-note parses of songB that start commensurately with the start time of Block 5 in
As noted earlier, 8 possible combinations of “0s” and “1s” exist for the 3 variables. Each of these can be associated with a “change” to instA. For example, for any given parse of 16 quarter notes that follows the Introduction in the mash-up song generated by muMap, suppose the j and g(j) values found from the mapping with respect to the x-variable are not equal (resulting in “1”), but the j and g(j) values found from improved chaotic mapping with respect to the y- and z-variables are equal (resulting in two “0s”), the implemented state will be Change No. 3 (100), i.e., convolve instA and instB for the entire parse+decrease tempo 8× for the first eighth of the parse. This change is then reflected in the instA component of muMap. Effects changes such as those determined by Table 3 can also be applied to inputs for other mash-up methods, including MUiMUv and its variants.
A Coda can be added to the mash-up as well and can include n parses of 8-quarters each, depending on the state of the mash-up in the parse immediately preceding the desired Coda.
Assuming an 8-quarter note parse size, if improved chaotic mapping returns a state having only two lines (tracks) for the mash-up parse occurring right before the start of the Coda, then those 2 lines can continue for 16 more quarter notes, after which the upper line is eliminated, and the lower track continues for 8 more quarters, then fades over the next 8 quarters. Here, the Coda includes four 8-quarter-note parses, i.e., n=4.
If improved chaotic mapping returns a state of three lines (tracks) for the mash-up parse occurring right before the start of the Coda, then those 3 lines can continue for 16 more quarter notes, provided two of the lines are not vocA and vocB, in which case instA or instB replaces one of the vocal tracks according to which instrumental track was not present in the previous state. Next, an instrumental track is eliminated, leaving the mash-up with one vocal and one instrumental for another 16 quarter notes. Finally, the vocal track can be eliminated such that the remaining instrumental track is left to play for another 8 quarter notes with a fade at the end, resulting in a Coda of five 8-quarter-note parses (n=5).
If improved chaotic mapping returns four lines (tracks) for the mash-up parse occurring right before the start of the Coda, then vocB can be eliminated for the first parse of the Coda. For each successive 8-quarter-note parse of the Coda, the following steps can be taken:
Yet another general aspect of the present invention incorporates a mash-up algorithm referred to herein as the “improved muMap.”
Trimming instA: It turns out that sometimes the length of instA, before vocB enters, is excessive. For example, the Michael Jackson song “Thriller” has an instrumental introduction that lasts about 1 minute and 24 seconds. To remedy this, the following steps can be applied: once the complete mash-up is made, vocB can be checked to see if it enters on, or after, a specified threshold time, such as the 34th eighth note. If so, the two instA parses, occurring before the parse where vocB enters, can be identified, i.e., instA Parse 1 and instA Parse 2 in
It should be noted that, while many of the examples presented above are directed to mash-ups that combine an A song with only one B song, in embodiments the disclosed method can be used to create mash-ups between an A song and a plurality of B songs.
Rotating Change Implementation Table
As described above, the instrumental track from songA can be varied in each mash-up by running improved chaotic mapping on the parses of songB and applying the j, g(j) values from songB to the parses of instA to identify those parses of instA that will acquire an effect (whenever j≠g(j)). Then improved chaotic mapping operates on parses of songA to determine which effect (or which change) will be applied. But if extreme initial conditions are used when running improved chaotic mapping on the parses of songA, e.g., [2, 2, 2], then virtually every parse of instA will have j≠g(j) for all 3 variables, resulting in 111 for every parse, where the three 1's signify j≠g(j) for each of the 3 variables x, y, and z. This means the effect shown in row 7 of Table 3 above (convolve instA with instB for the entire parse), will be applied to each parse of instA identified to acquire an effect. To avoid this redundancy, extreme initial conditions can be eschewed in favor of initial conditions closer to [1, 1, 1] for setting up the j vs. g(j) values across x, y, and z, that are used to select a change from the change implementation table.
However, another problem can arise. To assign an effect to a “changing” parse in instA, improved chaotic mapping operates on the parses of songA to generate the j vs. g(j) values, but that means that the same effects will occur in every instA as it appears in every mash-up of songA with any songB. So to ensure that instA substantively varies in each mash-up with a songB, an implementation table can be rotated for each AB1, AB2, AB3, . . . mash-up, up to some maximum such as 8 B songs, where ABk refers to a mash-up of songA with songBk, where songBk is one of a group of N songs, indexed by k=1 to N, with which songA is being combined to form mash-ups. Where the group of B songs is greater than a specified maximum, such as 8 B songs, a wraparound can be used to start the rotation over again.
For example, suppose the Rotating Change Implementation table of Table 4A is used for varying instA in the mash-up of songA with songB1, which is the first of a plurality of B songs, each of which is in the same key as songA, or is in a related key, to produce the new mash-up AB1.
Then, to vary instA for the second mash-up, AB2, the ‘Change Implemented’ column can be rotated (vertically shifted by one row) as shown in Table 4B, thereby guaranteeing that instA will not acquire the exact same set of effects in AB2 as it did in AB1.
The above strategies for trimming instA, rotating the change implementation table, and applying changes to instA dictated by j vs. g(j) outcomes based on songB, can be applied to a full realization of the improved muMap, whereby, for example, each track is parsed in groups of 8 quarter notes and arranged according to the improved muMap, shown in
Embodiments of this general aspect include, among others, the following variants on the improved muMap:
For variant (1) above, it is necessary to vary instA by applying improved chaotic mapping to parses of songB, for example 4-quarter-note length events, and then applying the j vs. g(j) values to instA. When j≠g(j) for songB parses, the g(j)th element of instA substitutes for the jth parse of instA to create jgjVarlnstA. Similarly, to vary instB, improved chaotic mapping operates on parses of songA and applies the j vs. g(j) values to instB, whereby if j≠g(j), the g(j)th element of instB substitutes for the jth element of instB, to create jgjVarlnstB. Then jgjVarlnstA is substituted for instA in the improved muMap and jgjVarlnstB is substituted for instB in the improved muMap.
For variant (2) above, improved chaotic mapping is applied to parses of songB, for example to 4-quarter-note length events, to determine which parses of instA will change, i.e., whenever j≠g(j) for the improved chaotic mapping applied with respect to one variable, the x-variable. How those instA parses will change is then determined by running improved chaotic mapping on the parses of songA producing a set of j≠g(j) comparisons with respect to each x-, y-, and z-variable associated with each parse of songA. For each jth parse of instA that is to change, the Rotating Change Implementation Table specifies a signal processing effect, according to the binary numbers corresponding to the j vs. g(j) results for each x-, y-, and z-variable associated with each parse of songA, as depicted in
Similarly, improved chaotic mapping is applied to parses of songA, for example 4-quarter-note length events, to determine which parses of instB will change, i.e., whenever j≠g(j). How those instB parses will change is then determined by running improved chaotic mapping on the parses of songB to find the j vs. g(j) values, not only with respect to the x-variable but also with respect to y- and z-variables. Then the values of j and g(j) returned by the mapping with respect to each variable are compared, whereby if j=g(j), a “0” is assigned to the jth event; if j≠g(j), “1” is assigned to the jth event. Since 8 possible combinations of 0s and 1s exist for the three variables, a Rotating Change Implementation Table similar to Table 4A and Table 4B can be used to determine which effect is assigned to each parse of instB that is to change.
For variant (3) above, the steps for (1) are performed first, i.e., instA is varied by applying improved chaotic mapping to the parses of songB and then applying the j vs. g(j) values to instA. When j≠g(j) for the parses of songB, the g(j)th element of instA substitutes for the jth element of instA, to create jgjVarlnstA. Similarly, to vary instB, improved chaotic mapping operates on the parses of the songA and applies the resulting j vs. g(j) values to instB: when j g(j) for songA parses, the g(j)th parse of instB substitutes for the jth parse of instB, to create jgjVarlnstB.
Then the steps for variant (2) are implemented, i.e., improved chaotic mapping is applied to parses of songB to determine which parses of jgjVarlnstA will change, i.e., whenever j≠g(j). How those jgjVarlnstA parses will change is then determined by running improved chaotic mapping on the parses of songA to find the j vs. g(j) values, not only with respect to the x-variable, but also with respect to y- and z-variables. Then the values of j and g(j) returned by the mapping with respect to each variable are compared, whereby if j=g(j), a “0” is assigned to the jth event; if j≠g(j), “1” is assigned to the jth event. Since 8 possible combinations of 0s and 1s exist for the three variables, a Rotating Change Implementation Table similar to Table 4A and Table 4B can be used to determine which effect is assigned to each jgjVarlnstA parse identified for change in the AB1 mash-up. Then the table can rotate for the AB2 mash-up so that a new effect is associated with each row combination of 0s and 1s in the table.
Similarly, improved chaotic mapping is applied to parses of songA to determine which parses of jgjVarlnstB will change, i.e., whenever j≠g(j). How those jgjVarlnstB parses will change is then determined by running improved chaotic mapping on the parses of songB to find the j vs. g(j) values, not only with respect to the x-variable, but also with respect to y- and z-variables. Then the values of j and g(j) returned by the mapping with respect to each variable are compared, whereby if j=g(j), a “0” is assigned to the jth event; if j≠g(j), “1” is assigned to the jth event. Since 8 possible combinations of 0s and 1s exist for the three variables, a Rotating Change Implementation Table similar to Table 4A and Table 4B can be used to determine which effect is assigned to each jgjVarlnstB parse identified for change in the AB1 mash-up. Then the table can rotate for the AB2 mash-up so that a new effect is associated with each row combination of 0s and 1s in the table.
Variants (4), (5), and (6) above each preserve the Introduction of the improved muMap without varying any of the vocal or instrumental tracks, while applying the variation possibilities offered by variant (1) (g(j) substitution), variant (2) (Rotating Change Implementation Table), and variant (3) (g(j) substitution plus Rotating Change Implementation Table), respectively, to the instrumental tracks.
Other improved muMap variants are included in the scope of the invention that further change the improved muMap of
Applying Improved Chaotic Mapping to Vary the Structure of muMap for Each Mash-Up: Left/Right muMap (LRmuMap)
In some cases, a user may wish to have more ‘paths’ through a structural mash-up other than those presented in
Four musical options for the ‘Introduction’ of a muMap mash-up according to the LRmuMap method are shown in
To determine the material following the Introduction (Blocks 0-4) of the mash-up, a Rotating (or shifted) Implementation Table can be used to determine a structure that will build the second (larger) section of the mash-up comprising Blocks 5-35 shown in
Specifically, a Rotating State Implementation Table can determine which state occupies each of Blocks 5-35, according to a series of ‘Left-hand’ or ‘Right-hand’ path options for each pair of blocks, excepting Block 29 which offers only one option (vocA, vocB, instA, instB), as shown in
In the embodiment of
To determine the flow of states from Block 5-Block 35, improved chaotic mapping can operate on a list of events, e.g., 16-quarter-note parses of songB, and return j vs. g(j) values not only with respect to the x-variable, but also with respect to y- and z-variables, for a given set of initial conditions and other parameters. Then the values of j and g(j) that are found from the mapping, with respect to each variable, are noted across all 3 variables of x, y, and z, where 0 is assigned when j=g(j), and 1 is assigned when j≠g(j). Though the parses of songB will likely number more than 13, the first 13 j vs. g(j) values across all 3 variables can be used to determine the left-hand or right-hand option for the 13 blocks in question. Or, one can simply run improved chaotic mapping on a hypothetical list of 13 elements, acquiring thirteen j vs. g(j) values to be applied to the 13 block pairs, thus determining whether the left-hand or right-hand option is implemented.
In some embodiments, improved chaotic mapping determines state options for the entire mash-up, including the Introduction, for example by running the mapping on 8-quarter-note parses of songB and finding j vs. g(j) values across all 3 variables x, y, and z. Assuming songB has M 8-quarter-note parses, those M parses can be divided by 15: M/15=S, where S represents a new sampling interval applied to the M parses. Then S is truncated, thus eliminating any decimals so that only an integer remains. The M parses are divided by 15 to account for the 13 block pairs following the Introduction plus 2 additional “quasi-events” used to decide which of Options 1A, 1B, 2A, or 2B will define the Introduction, as explained shortly.
The sampling interval S can then be used to select those parses indexed by j=1, S, 2S, 3S, . . . , 12S, 13S, 14S and their associated j vs. g(j) values across all 3 variables x, y, and z. The j and g(j) values can then be converted to 0s and 1s, as explained earlier, so they form rows in an implementation table that returns either a left-hand (LH) option or right-hand (RH) option for each of 8 possible combinations of 0s and 1s, given 3 variables x, y, and z.
The State Option Implementation Table can be constructed as shown in Table 5A:
The j vs. g(j) values for j=1 can determine the selection of a left-hand (option 1) or right-hand (option 2) track combinations for the mash-up Introduction, according to Table 5A. Similarly the j vs. g(j) values for j=S can further refine the selection to either A (left-hand option) or B (right-hand option), thus determining which of the four textural options presented by
For example, for j=1, suppose the j vs. g(j) values yield 011. From Table 5A, 011 indicates a right-hand option. Therefore Blocks 0-4 of
Finally, a Rotating State Option Implementation Table, such as Table 5B, can enable a different textural structure for each mash-up, i.e., a different path through
Many ways exist to create different textural paths for each mash-up. These include:
It should be noted that 1) a rotating or shifting implementation table can be implemented in many ways, e.g., using modular operations that are synonymous with rotation about a cylinder, and 2) the improved chaotic mapping with a designated variation procedure can alter any of a song's component tracks before a mash-up algorithm is applied. For example, improved chaotic mapping enables instA (instB) to vary by substituting the g(j) parse for the jth parse in each instrumental track, whenever j≠g(j) as determined by j vs. g(j) values resulting from songB (songA).
The muMap method and its various incarnations can create mash-ups that can be combined for example with a graphical element, which can be forwarded to at least one recipient, where the graphical element can be any of a graphical image; a video or part of a video; a film or part of a film; a video game or part of a video game; a greeting card or part of a greeting card; a presentation slide element or presentation slide deck; a slide presentation created by a presentation software application, wherein the presentation software application is configured to perform the steps of any of the mash-up methods disclosed herein; an element of a storyboard that describes a proposal for a musically accompanied graphical work, e.g., a musically accompanied video.
The muMap method can also be applied to inputs associated with a website or software program, so that each time a user visits a designated page of the website or accesses the software program, the website or software program is configured to operate on the inputs to produce a mash-up and play/present it to the user.
The muMap method can also be implemented in a digital device and installed for example in a greeting card that is configured to produce and present a mash-up when the greeting card is opened. Such a digital device can also be included within a toy, an MP3 player, a cellular telephone, or a hand-held or wearable electronic device. The digital device can produce a new mash-up of the inputs each time the inputs are accessed, or automatically from a plurality of input songs.
The muMap method can be embedded in an application or computing module running in hardware or software on a hand-held or wearable electronic device, or on a network to a hand-held or wearable electronic device, any of which are configurable to play a new mash-up or variant of a mash-up of input musical compositions each time input musical compositions are selected, from a plurality of inputs supplied by the application or other users, manually or automatically, including mash-ups successively created in a chain of users or machines. The muMap method can also be practiced by a computing module included in a hand-held or wearable electronic device wherein a plurality of tracks from input musical compositions are accessible to a hand-held or wearable electronic device, and the hand-held or wearable electronic device is configured to enable the user to access subsets of the tracks as the inputs and to create therefrom a mash-up.
Mash-up variations created using the present invention can be uploaded to websites, including social media sites and digital music services such as Pandora and Spotify. In embodiments, a website can be programmed to generate and play different mash-ups of popular songs each time the website is visited. Similarly, a mobile app can implement the present invention to produce mash-ups of specific songs on a playlist, e.g., to rejuvenate a play list or create playlists where mash-ups are made of songs such that the context of the songs changes from one hearing to the next.
The present invention can also allow users of digital music services to directly interact with songs and easily create variations of them (e.g., remixes and mashups), thus acting as a differentiator among digital music services which all offer essentially the same service. The invention can offer artists and producers new ways to market albums that allow fans to interact directly and easily with the album songs to make variations of them (e.g., remixes and mashups), thus acting as a differentiator for artists, producers, and their work. It can also foster collaborations among artists with regard to their past and current work, especially artists with diverse styles, e.g., Kanye West and the Beatles.
Mash-up variations can also be combined with graphical works such as photographs or videos. For example, with reference to
With reference to
The present invention can enhance social interactions in other ways. For example, with reference to
In other embodiments of the present invention, an e-card hosting website enables a sender to produce a unique and personal mash-up composition 1102 created using the present invention, to be played when the card is opened. The sender can further include a recording of his or her own voice, or of a synthesized voice, or of another acoustic or electronic input, to further personalize the message. For synthesized voices, the invention further enables the user to vary and make a mash-up of any of the synthesized voice tracks, thereby changing the context, text, pitch or speed of the voice(s), etc.
With reference to
Each of the mash-up outputs 1202, 1208, 1212, 1216 can be stored by the website 1218, so that a visitor to the website 1218 can enjoy listening to the succession of mash-ups, which may begin as small changes to the input compositions 1204, and then evolve to mash-up variations where the input compositions 1204 are hardly recognizable.
With reference to
Video games typically feature sound tracks that accompany the actions of a hero, heroine, or the user. For example, theme music is often associated with actions of the hero or even with the user. With reference to
In embodiments, the present invention is implemented in a small electronic chip that can be included in a larger item, such as a hand-held device (smart phone, iPod, iPad, tablet, etc.), an MP3 player, a greeting card, or wearable technology such as a smart watch, smartband, or wearable computer. In some of these embodiments, the item is thereby enabled to play mash-ups of music stored therein. For example, the chip can be instructed to create a new mash-up of a group of two or more compositions every time they are selected on an MP3 player. With reference to
The foregoing description of the embodiments of the invention has been presented for the purposes of illustration and description. Each and every page of this submission, and all contents thereon, however characterized, identified, or numbered, is considered a substantive part of this application for all purposes, irrespective of form or placement within the application. This specification is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of this disclosure.
Although the present application is shown in a limited number of forms, the scope of the invention is not limited to just these forms, but is amenable to various changes and modifications without departing from the spirit thereof. The disclosure presented herein does not explicitly disclose all possible combinations of features that fall within the scope of the invention. The features disclosed herein for the various embodiments can generally be interchanged and combined into any combinations that are not self-contradictory without departing from the scope of the invention. In particular, the limitations presented in dependent claims below can be combined with their corresponding independent claims in any number and in any order without departing from the scope of this disclosure, unless the dependent claims are logically incompatible with each other.
| Number | Name | Date | Kind |
|---|---|---|---|
| 5003860 | Minamitaka | Apr 1991 | A |
| 5281754 | Farrett et al. | Jan 1994 | A |
| 5331112 | Sato et al. | Jul 1994 | A |
| 5371854 | Kramer | Dec 1994 | A |
| 5418323 | Kohonen | May 1995 | A |
| 5606144 | Dabby | Feb 1997 | A |
| 6028262 | Minamitaka | Feb 2000 | A |
| 6137045 | Short et al. | Oct 2000 | A |
| 6177624 | Takahashi et al. | Jun 2001 | B1 |
| 7034217 | Pachet | Apr 2006 | B2 |
| 7135635 | Childs et al. | Nov 2006 | B2 |
| 7193148 | Cremer et al. | Mar 2007 | B2 |
| 7498504 | Bourgeios | Mar 2009 | B2 |
| 7560636 | Wolfram | Jul 2009 | B2 |
| 7629528 | Childs, Jr. et al. | Dec 2009 | B2 |
| 7840608 | Kurzweil et al. | Nov 2010 | B2 |
| 8035022 | Wolfram | Oct 2011 | B2 |
| 9286876 | Dabby | Mar 2016 | B1 |
| 9286877 | Dabby | Mar 2016 | B1 |
| 20140095978 | Chang | Apr 2014 | A1 |
| 20140355789 | Bohrarper | Dec 2014 | A1 |
| Entry |
|---|
| Lorenz, E.N., Deterministic nonperiodic flow, J. Atmos. Sci., 1963, vol. 20, pp. 130-141. |
| Dabby, D.S., Musical Variations from a Chaotic Mapping, 1995 MIT doctoral thesis, 162 pgs. |
| Dabby, D.S., Musical Variations from a Chaotic Mapping, Chaos, 1996, 13 pgs. |
| Dabby, D.S. Dabby, A chaotic mapping for music and image variations, Proc. Fourth Int'l. Chaos Conference, 1998, 12 pgs. |
| Dabby, D.S. Dabby, Creating Musical Variation, Science, 2008, 2 pgs. |
| Carabias-Orti, J. J. et al., Music Scene-Adaptive Harmonic Dictionary for Unsupervised Note-Event Detection, IEEE Transactions on Audio, Speech, and Language Processing 18 (3), 2010, pp. 473-486. |
| Number | Date | Country | |
|---|---|---|---|
| 62563669 | Sep 2017 | US |
