Patent Grant
9412351
Embodiments of the invention generally relate to software configured for editing MIDI-based musical performances. More specifically, aspects of proportional quantization in a musical performance are described herein.
Music editing applications allow music composers, media artists, and other users to create and edit a musical performance stored as Musical Instrument Digital Interface (MIDI) data. Users can import MIDI data files or compose musical pieces stored as MIDI data and use tools provided by the music editing application to edit the sequences of notes in the MIDI data. For example, a graphical user interface (GUI) of such a music editing application can allow users to modify one or more characteristics of the MIDI data, such as the pitch, position, timing, duration, and velocity (or “loudness”) of the sequences of recorded notes in the MIDI files. Music editing can be performed on a digital audio workstation (“DAW”), GarageBand™ by Apple Inc., or any other suitable music editing applications that exist in hardware, software, firmware, or any combination thereof.
Music editing applications typically provide a musical grid and reference track (e.g., metronome click) that the performing musician will play to. Occasionally, the recorded sequence of MIDI note events will not match the grid position precisely. Often, this is caused by an intended groove, but sometimes the timing is simply inaccurate and the user may want to correct this imprecision without having to repeat the performance perfectly. However, some deviation from the grid might be desirable in certain cases to maintain the feel and expression of the original recording. For instance, a composer may wish to establish a strong (“tight”) relationship between a kick drum sound (i.e., MIDI event) and quarter-note markers in the musical performance. Thus, quantization would be appropriate in this case to ensure that the position of any imprecise kick drum MIDI event is corrected accordingly. On the other hand, certain musical techniques or performances include subtle dynamics, flurries, or other musical characteristics (e.g., quickly arpeggiated piano chords, drum flams, or drum rolls), resulting in a succession of notes that can be positioned very close together. Thus, typical quantization algorithms would be detrimental to these performances as many of the notes would be pushed to overlapping positions or the relationship of the notes' placement with respect to one another would be significantly changed, resulting in unintended and often unpleasant musical arrangements.
In summary, a significant downside of common quantization algorithms is that they do not properly maintain musically intended deviations from the grid, but force note positions exactly onto the grid, thus destroying any musical feel or playing detail beyond the grid resolution. While reducing quantize strength to values below 100% may keep some deviations intact to some degree, things like flams, quick arpeggios or grace notes, etc., are still compressed in time and lose their intended effect. Thus, there is a need for improved quantization algorithms to address these problems.
Certain embodiments of the invention relate to a method of proportional quantization that can to correct imperfections in the timing of MIDI events in an arrangement, while still maintaining the relationship of MIDI events with respect to one another. In a simplified embodiment, the method includes identifying a number of target positions within a grid. The target position identifies a point that adjacent MIDI events (e.g., MIDI notes) will be moved toward when quantization is applied. Each target position has a defined search range that defines an area around a target position. In some cases, a “center of gravity,” can be an average position of the MIDI events contained within each search range, which can be referred to as a “reference point.” The average position can be based on a number of MIDI characteristics for each MIDI event including, but not limited to, a position and velocity (i.e., “loudness”). The reference point is then quantized or moved toward the corresponding target position by a predetermined amount and the associated MIDI events in that particular search range are moved accordingly. Thus, the reference point can be quantized to precise positions in a grid, while maintaining the relationship of the corresponding MIDI events associated therewith. MIDI events outside of the search ranges can also be affected by adjacent quantization events, as further discussed below.
According to certain embodiments, a computer-implemented method includes receiving input data including MIDI events arranged in a timeline, determining a target grid position from among the plurality of grid positions, determining a search range around the target grid position, and identifying a set of MIDI events within the search range around the target grid position. The method can further include determining a reference point for the set of MIDI events based on a function of the set of MIDI events, adjusting a position of the reference point toward the target grid position, determining a proportional movement for each MIDI event on the timeline based on its location relative to the adjusted reference point, and adjusting each MIDI event based on the determined proportional movement. The function of the set of MIDI events can be a weighted average based on one or more MIDI characteristics of the set of MIDI events.
In some implementations, the MIDI characteristics include one or more of a MIDI event velocity and a MIDI event position relative to its corresponding reference point. The reference point may be adjusted toward the target grid position based on a quantization strength. In some cases, the method can further include receiving quantization data indicating a desired quantization strength, and setting the quantization strength based on the quantization data. Certain embodiments may further comprise determining a second target grid position from among the plurality of grid positions, determining a second search range around the second target grid position, identifying a second set of MIDI events within the search range around the second target grid position, and determining a second reference point for the second set of MIDI events, where the second reference point corresponds to a function of the second set of MIDI events. The method may further include adjusting a position of the second reference point toward the second target grid position, where determining a proportional movement for each MIDI event on the timeline is further based on its location relative to the second reference point. Events can be moved to any location on the timeline, as dictated by the quantization algorithm. For instance, some events may be moved to a grid line, between grid lines, or overlap grid lines, as would be appreciated by one of ordinary skill in the art.
The disclosure will be readily understood by the following detailed description in conjunction with the accompanying drawings. It should be understood that while the following figures and associated detailed description provide a number of illustrative embodiments to explain various aspects of the present invention, it should be understood that the specific examples are not limiting and concepts described in one embodiment of the invention can be modified and/or applied to any other embodiment or concept described herein. The accompanying drawings include:
Embodiments of the invention generally relate to software configured for editing MIDI-based musical performances. More specifically, aspects of proportional quantization in a musical performance are described herein.
Certain embodiments of the invention relate to a method of proportional quantization that can to correct imperfections in the timing of MIDI events in an arrangement, while still maintaining the relationship of MIDI events with respect to one another. In a simplified embodiment, the method includes identifying a number of target positions within a grid. The target position identifies a point that adjacent MIDI events (e.g., MIDI notes) will be moved toward when quantization is applied. Each target position has a defined search range that defines an area around a target position. In some cases, a “center of gravity,” can be an average position of the MIDI events contained within each search range, which can be referred to as a “reference point.” The average position can be based on a number of MIDI characteristics for each MIDI event including, but not limited to, a position and velocity (i.e., “loudness”). The reference point is then quantized or moved toward the corresponding target position by a predetermined amount and the associated MIDI events in that particular search range are moved accordingly. Thus, the reference point can be quantized to precise positions in a grid, while maintaining the relationship of the corresponding MIDI events associated therewith. MIDI events outside of the search ranges can also be affected by adjacent quantization events, as further discussed below.
MIDI Capture and Editing
Music editing applications allow music composers, media artists, and other users to create and edit a musical performance stored as Musical Instrument Digital Interface (MIDI) data. Users can import MIDI data files or compose musical pieces stored as MIDI data and use tools provided by the music editing application to edit the sequences of notes (i.e., MIDI events) in the MIDI data. For example, a graphical user interface (GUI) of such a music editing application can allow users to modify one or more characteristics of the MIDI events, such as the note number, pitch, position, timing, duration, and velocity (or “loudness”) of the sequences of recorded notes in the MIDI files. In many cases, the recording device/software will provide some kind of musical grid and metronome click to which the performing musician will play. Typically, the recorded MIDI note events will not match the grid position precisely. Often, this is caused by an intended groove, but sometimes the timing is simply inaccurate and the user may want to correct this imprecision without having to repeat the performance perfectly. However, some deviation from the grid might be desirable in certain cases to maintain the feel and expression of the original recording.
To illustrate, a composer may wish to establish a strong (“tight”) relationship between a kick drum sound (i.e., MIDI event) and quarter-note markers in the musical performance. Thus, quantization would be appropriate in this case to ensure that the position of any imprecise kick drum MIDI event is corrected accordingly. On the other hand, certain musical techniques or performances include subtle dynamics, flurries, or other musical characteristics (e.g., quickly arpeggiated piano chords, drum flams, or drum rolls), resulting in a succession of MIDI events that can be positioned very close together. Thus, typical quantization algorithms would be detrimental to these performances as many of the MIDI events would be pushed to overlapping positions or the relationship of the MIDI events' placement with respect to one another would be significantly changed, resulting in unintended and often unpleasant musical arrangements.
Some typical parameters of quantization functionality can include grid resolution, quantization strength, quantization range, and quantization swing. Grid resolution can refer to how the grid is divided with respect to timing. For example, a grid can be divided into 1/16 notes, 1/32 notes, 1/64 notes, or other suitable resolution. Quantization strength typically defines what percentage of timing correction (i.e., quantization) is applied. Quantization range typically defines the catch range around a grid position in which events will be affected by the quantization. For example, a MIDI event (e.g., kick drum) close to a grid position may be moved or “snapped” to that grid position, while other MIDI events farther from the grid position may remain unchanged. Finally, quantization swing can include modifying grid positions on offbeats to achieve a swing feeling.
MIDI Characteristics
MIDI characteristics can include any number of performance characteristics that define how a musical element is played (or not played). For example, performance characteristics can include velocity data, note data (e.g., note type, rest type, note ties, etc.), rhythmic order data, timing data, pitch data, or any type of data that can characterize aspects of the performance, as would be appreciated by one of ordinary skill in the art with the benefit of this disclosure. In some cases, MIDI characteristics can be referred to as MIDI performance data.
Velocity is one type of musical performance data that can be described as the speed or force with which the key is being played. In a MIDI input device (e.g., keyboard), the harder a key is played, the higher the velocity value is registered. Similarly, the softer the key is played, the lower the velocity value. In MIDI, velocity is typically measured on a scale from 0 to 127, with 127 being the highest value that can be registered. It would be understood by one of ordinary skill in the art that any range of values, MIDI or otherwise, can be used to represent key velocity values.
MIDI events, in this example, are represented by a series of horizontal rectangles or blocks aligned on matrix grid 210 including horizontal (240) and vertical (230) lines representing time parameters and musical pitch parameters, respectively. The horizontal placement of note events indicates their temporal (e.g., bar, beat, and sub-beat) positioning within the region. The length of the rectangle in matrix grid 210 is directly proportional to the MIDI event length. The vertical position of MIDI events indicates their respective pitch, with those nearer the top of matrix grid 210 being higher in pitch. Chords are displayed as a vertical stack of MIDI event rectangles.
In this example, matrix editor 210 also includes a vertical keyboard 220 on the left side of matrix grid 210 that indicates MIDI event pitches. As shown, horizontal black lines run across matrix grid 210 to enable the user to easily transpose notes by dragging them up or down the horizontal rows 240. Although a piano-type matrix editor is shown in this example, a user can select a different type of matrix editor for different types of instruments, such as a string-type or guitar-type matrix editor. In such embodiments, instead of vertical keyboard 220, the matrix editor can display a vertical guitar/string fret board, a string neck, a guitar neck, or other suitable style of user interface.
Matrix editor 200 also includes tools to facilitate the editing process, including moving MIDI events, adding events, deleting events, changing MIDI characteristics of MIDI events (e.g., changing pitch (vertical movement), changing time position (horizontal movement), changing duration, changing velocity, etc.), and the like. Those of ordinary skill in the art with the benefit of this disclosure would appreciate the editing capabilities provided by matrix editor 200.
In this particular example, each of the MIDI events are kick drum samples. In a musical performance, the kick drum may provide a reference beat that other instruments in an arrangement are synchronized to. Thus, it may be important for a kick drum to have consistent timing to ensure that all of the instruments are in synch. Original performance 302 includes a series of kick drums (MIDI events 322, 332, 342, 352) that are poorly aligned with their corresponding target grid positions. This may occur, for example, as a result of a live-capture performance by an inexperienced musician. MIDI events 322, 332, and 352 are triggered after their corresponding target grid position by varying amounts, while MIDI event 342 is triggered prior to its corresponding target grid position. Quantizing the MIDI events can be advantageous, as shown in this case, to ensure that the reference beat (i.e., kick drums) is accurate, consistent, and reliable. An example of a resultant quantized performance 304 is shown directly below the original performance.
Referring to
In this example, a kick drum sound (MIDI event 412) and a number of snare drum sounds (MIDI events 422, 424, 426, 432, 434, 436) are placed on time line 401. A first set of snare samples includes a snare triplet comprising events 422, 424, and 426. A second set of snare samples includes a snare triplet comprising events 432, 434, and 436. A 100% quantization strength is applied to the kick drum sample (event 412) and the first set of snare drum samples (events 422, 424, 426). A 50% quantization strength is applied to the second set of snare drum samples (events 432, 434, 436). The intended outcome of the musical performance is to play the kickdrum on a quarter note marker (i.e., target position 410) and the first note of each snare triplet on their corresponding marker (i.e., target positions 420, 430), while still maintaining the positional relationship of each event in the snare drum triplet. As shown in original performance 402, the kick drum and snare samples are poorly aligned with their corresponding target grid positions. In this example, the result of the application of a typical quantization algorithm results in unintended effects, as illustrated in the resultant musical arrangement of quantized performance 404.
Referring to
Each event of the first snare triplet (events 422, 424, and 426) is triggered after target position 420. This example illustrates how applying a conventional quantization process can lead to unintended results. Each event 422, 424, 426 has a different velocity, which is represented as an amplitude (i.e., height of bar). Each event 422, 424, 426 is quantized at 100% quantization strength and is aligned with target position 420. Thus, each event 422, 424, 426 loses its spacing relative to one another such that the snare triplet becomes a single event having three overlapping velocities. That is, the positional relationship between each event in the triplet is lost due to quantization.
In this example, events 412 and 422-426 are quantized at 100%, however the actual range of a quantization algorithm can vary. For example, 100% quantization in one MIDI editor may span a first range or threshold, while a different MIDI editor may have a range different from the first range. The range can be defined by a unit of time (e.g., milliseconds), beat denominations (e.g., 1/16 notes), or other suitable unit of measurement, as would be appreciated by one of ordinary skill in the art. For the sake of simplicity, that actual threshold from target position 420 at 100% quantization is not specified in
Each event of the second snare triplet (events 432, 434, 436) is triggered after target position 430. This example illustrates how applying a conventional quantization process, even with a reduced quantization strength, may still lead to unintended results. Each event 432, 434, 436 in original performance 402 has a different velocity. Since the quantization strength is 50%, each event 432, 434, 436 is quantized by a smaller amount then the first snare triplet (at 100% quantization strength). Although the events do not shifted as much, the relationship between events 432, 434, 436 is changed, as shown in 404. Thus, quantization can be a useful tool for improving the timing of a musical performance, but conventional methods can distort the positional relationship between proximate events.
Proportional Quantization
The following figures (
Each of the MIDI events has a corresponding velocity. In
A quantization range (“range”) is a set bounded area that a quantization algorithm is applied to, according to certain embodiments of the invention. The bounded area, or “width” of each range can be any suitable amount. Wider ranges will quantize MIDI events that span a wider area on timeline 501 than narrower ranges, and vice versa.
Referring to
In some embodiments, a reference point represents a “weighted average” of any MIDI events positioned within a quantization range. Referring to
The reference point, or weighted average, can be determined in a number of ways. For example, the weighted average may be determined based on the relative positions of the MIDI events, the relative velocity (or other suitable characteristic) of each MIDI event, or any combination thereof, as would be appreciated by one of ordinary skill in the art with the benefit of this disclosure.
Referring to
Referring to target position 510, one relatively low velocity MIDI event 508 is positioned prior to target position 510 in quantization range 650, while two relatively high velocity MIDI events 512, 514 are positioned after target position 510. As a result, reference point 655 is positioned between the location of MIDI events 512, 514 because they have a significantly higher velocity than MIDI event 508 and therefore reference point 655 is skewed to the right of target position 510.
Referring to target position 530, only one MIDI event 528 is located within the corresponding quantization range 670. Thus, the resulting reference point 675 is positioned in the same location as MIDI event 528 since the calculation for the average position and velocity does not factor in any other MIDI events. The weighted averaging principle is applied to the remaining quantization ranges shown in
In a second embodiment, the group of MIDI events within each corresponding quantization range is quantized, but the MIDI events maintain their positional relationship with respect to one another. That is, the spacing between each MIDI event located within their corresponding quantization range is maintained, while the group of MIDI events as a whole are shifted towards the corresponding target position. More specifically, the reference point, which represents the average of the MIDI events within their corresponding quantization range, is moved toward the target position. For the sake of clarity, one could imagine the group of MIDI events fixed on a slide rule, where the group of MIDI events are shifted toward the target position based on the position of the reference point and the quantization strength.
The amount that the reference point is moved corresponds to the quantization strength. In this particular example, the quantization strength is set to 100%, but any suitable quantization can be used. Thus, in each embodiment, the articulation and timing of the individual MIDI events are maintained (e.g., a rapid fire snare roll), while still receiving the benefit of the timing correction by aligning the group of MIDI events as a whole with their corresponding target position. Consequently, MIDI events are quantized based on their weighted average and, as a result, the MIDI events sound “in time” or “in the pocket,” yet their relative placement with respect to the other MIDI events remains intact. This resolves the problems associated with conventional quantization methods and provides a very powerful musical correction tool with varied uses. It should be noted that although the figures described herein include MIDI events, the quantization concepts can be applied to other applications, as would be appreciated by one of ordinary skill in the art.
In each case, the reference points are quantized and aligned with their respective target positions. Each of the MIDI events associated with a reference point (or reference points) is also quantized and moved accordingly, but their resulting position may or may not be aligned with the target position. In some implementations, a compression algorithm can be applied, which can change the relative position of each MIDI event with respect to one another and with respect to the reference point. This is further discussed below with respect to
Compression Algorithms
Timeline A is a reproduction of the resulting proportional quantization process shown in
Referring to
In some embodiments, MIDI events positioned outside of any quantization range are not affected by the compression scheme. For example, MIDI event 916 is in the same position as MIDI event 716. Also, compression schemes may be applied to only certain quantization ranges, groups of MIDI events, or the like. For example, compression may be applied to one section of a musical performance, but not another section. The examples provided herein illustrate a compression scheme applied after a proportional quantization process is performed. Compression can be performed prior to quantization as well, which can have a different musical outcome, as would be appreciated by one of ordinary skill in the art with the benefit of this disclosure.
Referring to
Point B is closer to reference 1020 than 1010, and point C is very closely positioned near reference 1020 and very far from reference 1010. Therefore, the quantization of reference 1020 will have a proportionally higher influence on B and C than reference 1010. Note that since C is closer to reference 1020 than B, it is more heavily influenced by it. As such, C to C′ has a slightly larger displacement than B to B′. Neither movement can be equal to the displacement of reference 1020 because of the opposite influence of reference 1010.
In an effort to clarify this proportional quantization process, one can imagine that the timeline shown in
With respect to actual MIDI events, the proportional movement can depend not only to their proximity to the closest reference point, but also the relative “strength” of the reference points. For example, of one reference point is associated with a high number of high velocity MIDI events, and the second reference point is associated with only one low velocity MIDI event, then a MIDI event positioned equally between the two reference points would be more influenced by the first reference point with many high velocity MIDI events than the second reference points, and would move proportionally toward the first reference accordingly. Any number of factors including MIDI event location, MIDI characteristics, the magnitude of reference point movements, and any other suitable metric, can be used to determine how the proportional movement of MIDI events outside of quantization regions is affected and would be appreciated by one of ordinary skill in the art with the benefit of this disclosure.
At 1110, method 1100 begins with receiving input data including MIDI events arranged in a grid, where the grid can have a plurality of grid positions, and the MIDI events have one or more MIDI characteristics. For example, MIDI events can be arranged in any suitable order (see, e.g.,
At 1120, method 1100 includes determining a target grid position from among the plurality of grid positions. A target grid position is shown and described in
At 1140, method 1100 includes identifying a set of MIDI events within the search range around the target grid position. That is, all of the MIDI events within a search range are identified. For example, MIDI events 508, 512, and 514 are located within search range 650 of
At 1150, method 1100 includes determining a reference point for the set of MIDI events. The reference point can correspond to a weighted average of the set of MIDI events within the search range, where the weighted average is based on one or more MIDI characteristics of the set of MIDI events within the search range. For example, reference point 655 represents the weighted average of MIDI events 508, 512, and 514. The weighted average may factor in one or more of a MIDI event's velocity, positional location, or other suitable metric, as would be appreciated by one of ordinary skill in the art with the benefit of this disclosure.
At 1160, method 1100 includes adjusting a position of the reference point toward the target grid position using a quantization strength. As described above, a quantization strength can control how much a reference point is moved during a quantization process.
At 1170, method 1100 includes determining a movement for the set of MIDI events using the reference point. At 1180, method 1100 includes proportionally adjusting a position of each of the MIDI events on the timeline, where each MIDI event in the set generally maintains a positional relationship with respect to each other MIDI event in the set as the position of the set is adjusted. In some embodiments, as the reference point is moved toward the target grid position, each of the MIDI events in the corresponding set of MIDI events are proportionally moved based on the movement of the reference point. That is, the positional relationship of the MIDI events with respect to one another remains intact (i.e., generally the same distance between adjacent MIDI events) while the group of MIDI events, as a whole, is shifted toward the target grid position, resulting in a quantized arrangement that maintains the musicality of the original performance. An example of this is shown process is shown in the transition between
It should be appreciated that the specific steps illustrated in
As discussed above, method 1100 can be modified in any suitable manner, as would be appreciated by one of ordinary skill in the art with the benefit of this disclosure. For example, method 1100 can further include receiving quantization data indicating a desired quantization strength, and setting the quantization strength based on the quantization data. Method 1100 can further include determining a second target grid position, an associated search range, and a second set of MIDI events associated therein, determining a second reference point based on a weighted average of the second set of MIDI events, and adjusting a position of the second reference point and associated MIDI events in a similar fashion as discussed above in method step 1110-1170.
In some embodiments, method 1100 can include identifying an outlying MIDI event located outside of and between the first and second search ranges, determining a position of the outlying MIDI event, and adjusting the position of the outlying MIDI event using the first and second reference points. In some implementations, adjusting the position of the outlying MIDI event includes determining a distance of the outlying MIDI event with respect to the first reference point and second reference point, determining a strength of the weighted average for the first and second reference points, and adjusting the position of the outlying MIDI event based on its distance to the first and second reference points and the strength of the weighted average for the first and second reference points. The strength of the weighted average for each reference point can correspond to one or more of a number of MIDI events within the corresponding search range, and a velocity of the MIDI events within the corresponding search range. Other methods of determining the strength of a weighted average can be applied, as would be appreciated by one of ordinary skill in the art.
In some embodiments, method 1100 can further include receiving compression input data indicating an amount of compression to apply to the set of MIDI events within the search range, and compressing the set of MIDI events within the search range, where the MIDI events are moved relative to one another based on the sign and magnitude of the amount of compression specified in the compression input data.
System Architecture
It should be appreciated that system 1200 as shown in
In some embodiments, display subsystem 1205 can provide an interface that allows a user to interact with system 1200. The display subsystem 1205 may be a cathode ray tube (CRT), a flat-panel device such as a liquid crystal display (LCD), a projection device, a touch screen, or the like. In general, use of the term “output device” is intended to include all possible types of devices and mechanisms for outputting information from system 1200. For example, a software keyboard may be displayed using a flat-panel screen. In some embodiments, the display subsystem 1205 can be a touch interface, where the display provides both an interface for outputting information to a user of the device and also as an interface for receiving inputs. In other embodiments, there may be separate input and output subsystems. Through the display subsystem 1205, the user can view and interact with a GUI (Graphical User Interface) 1220 of a system 1200. In some embodiments, display subsystem 1205 can include a touch-sensitive interface (also sometimes referred to as a touch screen) that can both display information to the user and receive inputs from the user. Processing unit(s) 1210 can include one or more processors, each having one or more cores. In some embodiments, processing unit(s) 1210 can execute instructions stored in storage subsystem 1240. System 1200 can further include an audio system to play music (e.g., accompaniments, musical performances, etc.) through one or more audio speakers (not shown).
A communications system (not shown) can be implemented and in electronic communication with processing units 120. The communication system can include various hardware, firmware, and software components to enable electronic communication between multiple computing devices. A communications system or components thereof can communicate with other devices via Wi-Fi, Bluetooth, infra-red, or any other suitable communications protocol that can provide sufficiently fast and reliable data rates to support the real-time jam session functionality described herein. In some embodiments, the communications system can be integrated with another system level blocks, as would be appreciated by one of ordinary skill in the art.
Storage subsystem 1240 can include various memory units such as a system memory, a read only memory (ROM), and a non-volatile storage device (each not shown). The system memory can be a read and write memory device or a volatile read and write memory, such as dynamic random access memory. System memory can store some or all of the instructions and data that the processor(s) or processing unit(s) need at runtime. ROM can store static data and instructions that are used by processing unit(s) 1210 and other modules of system 1200. Non-volatile storage devices can be a read and write capable memory device. Embodiments of the invention can use a mass storage device (such as a magnetic or optical disk or flash memory) as a permanent storage device. Other embodiments can use a removable storage device (e.g., a floppy disk, a flash drive) as a non-volatile (e.g., permanent) storage device.
Storage subsystem 1240 can store MIDI (Musical Instrument Digital Interface) data relating to notes played on a virtual instrument of system 1200 in MIDI database 1230. The MIDI database 1230 can store performance data including velocity data, MIDI event data, rhythmic data, input data, compression data, quantization region data, or any other data, as previously described. Further detail regarding system architecture and the auxiliary components thereof (e.g., input/output controllers, memory controllers, etc.) are not discussed in detail so as not to obfuscate the focus on the invention and would be understood by those of ordinary skill in the art.
Processing unit(s) 1305 can include a single processor, which can have one or more cores, or multiple processors. In some embodiments, processing unit(s) 1305 can include a general purpose primary processor as well as one or more special purpose co-processors such as graphics processors, digital signal processors, or the like. In some embodiments, some or all processing units 1305 can be implemented using customized circuits, such as application specific integrated circuits (ASICs) or field programmable gate arrays (FPGAs). In some embodiments, such integrated circuits execute instructions that are stored on the circuit itself. In other embodiments, processing unit(s) 1305 can execute instructions stored in storage subsystem 1310.
Storage subsystem 1310 can include various memory units such as a system memory, a read-only memory (ROM), and a permanent storage device. The ROM can store static data and instructions that are needed by processing unit(s) 1305 and other modules of electronic device 1300. The permanent storage device can be a read-and-write memory device. This permanent storage device can be a non-volatile memory unit that stores instructions and data even when computer system 1300 is powered down. Some embodiments of the invention can use a mass-storage device (such as a magnetic or optical disk or flash memory) as a permanent storage device. Other embodiments can use a removable storage device (e.g., a floppy disk, a flash drive) as a permanent storage device. The system memory can be a read-and-write memory device or a volatile read-and-write memory, such as dynamic random access memory. The system memory can store some or all of the instructions and data that the processor needs at runtime.
Storage subsystem 1310 can include any combination of computer readable storage media including semiconductor memory chips of various types (DRAM, SRAM, SDRAM, flash memory, programmable read-only memory) and so on. Magnetic and/or optical disks can also be used. In some embodiments, storage subsystem 1310 can include removable storage media that can be readable and/or writeable; examples of such media include compact disc (CD), read-only digital versatile disc (e.g., DVD-ROM, dual-layer DVD-ROM), read-only and recordable Blue-Ray® disks, ultra density optical disks, flash memory cards (e.g., SD cards, mini-SD cards, micro-SD cards, etc.), magnetic “floppy” disks, and so on. The computer readable storage media do not include carrier waves and transitory electronic signals passing wirelessly or over wired connections.
In some embodiments, storage subsystem 1310 can store one or more software programs to be executed by processing unit(s) 1305, such as a user interface 1315. As mentioned, “software” can refer to sequences of instructions that, when executed by processing unit(s) 1305 cause computer system 1300 to perform various operations, thus defining one or more specific machine implementations that execute and perform the operations of the software programs. The instructions can be stored as firmware residing in read-only memory and/or applications stored in magnetic storage that can be read into memory for processing by a processor. Software can be implemented as a single program or a collection of separate programs or program modules that interact as desired. Programs and/or data can be stored in non-volatile storage and copied in whole or in part to volatile working memory during program execution. From storage subsystem 1310, processing unit(s) 1305 can retrieve program instructions to execute and data to process in order to execute various operations described herein.
A user interface can be provided by one or more user input devices 1320, display device 1325, and/or and one or more other user output devices (not shown). Input devices 1320 can include any device via which a user can provide signals to computing system 1300; computing system 1300 can interpret the signals as indicative of particular user requests or information. In various embodiments, input devices 1320 can include any or all of a keyboard touch pad, touch screen, mouse or other pointing device, scroll wheel, click wheel, dial, button, switch, keypad, microphone, and so on.
Output devices 1325 can display images generated by electronic device 1300. Output devices 1325 can include various image generation technologies, e.g., a cathode ray tube (CRT), liquid crystal display (LCD), light-emitting diode (LED) including organic light-emitting diodes (OLED), projection system, or the like, together with supporting electronics (e.g., digital-to-analog or analog-to-digital converters, signal processors, or the like), indicator lights, speakers, tactile “display” devices, headphone jacks, printers, and so on. Some embodiments can include a device such as a touchscreen that function as both input and output device.
In some embodiments, output device 1325 can provide a graphical user interface, in which visible image elements in certain areas of output device 1325 are defined as active elements or control elements that the user selects using user input devices 1320. For example, the user can manipulate a user input device to position an on-screen cursor or pointer over the control element, then click a button to indicate the selection. Alternatively, the user can touch the control element (e.g., with a finger or stylus) on a touchscreen device. In some embodiments, the user can speak one or more words associated with the control element (the word can be, e.g., a label on the element or a function associated with the element). In some embodiments, user gestures on a touch-sensitive device can be recognized and interpreted as input commands; these gestures can be but need not be associated with any particular array in output device 1325. Other user interfaces can also be implemented.
Network interface 1335 can provide voice and/or data communication capability for electronic device 1300. In some embodiments, network interface 1335 can include radio frequency (RF) transceiver components for accessing wireless voice and/or data networks (e.g., using cellular telephone technology, advanced data network technology such as 3G, 4G or EDGE, WiFi (IEEE 802.11 family standards, or other mobile communication technologies, or any combination thereof), GPS receiver components, and/or other components. In some embodiments, network interface 1335 can provide wired network connectivity (e.g., Ethernet) in addition to or instead of a wireless interface. Network interface 1335 can be implemented using a combination of hardware (e.g., antennas, modulators/demodulators, encoders/decoders, and other analog and/or digital signal processing circuits) and software components.
Bus 1340 can include various system, peripheral, and chipset buses that communicatively connect the numerous internal devices of electronic device 1300. For example, bus 1340 can communicatively couple processing unit(s) 1305 with storage subsystem 1310. Bus 1340 also connects to input devices 1320 and display 1325. Bus 1340 also couples electronic device 1300 to a network through network interface 1335. In this manner, electronic device 1300 can be a part of a network of multiple computer systems (e.g., a local area network (LAN), a wide area network (WAN), an Intranet, or a network of networks, such as the Internet. Any or all components of electronic device 1300 can be used in conjunction with the invention.
Some embodiments include electronic components, such as microprocessors, storage and memory that store computer program instructions in a computer readable storage medium. Many of the features described in this specification can be implemented as processes that are specified as a set of program instructions encoded on a computer readable storage medium. When these program instructions are executed by one or more processing units, they cause the processing unit(s) to perform various operation indicated in the program instructions. Examples of program instructions or computer code include machine code, such as is produced by a compiler, and files including higher-level code that are executed by a computer, an electronic component, or a microprocessor using an interpreter.
It will be appreciated that computer system 1300 is illustrative and that variations and modifications are possible. Computer system 1300 can have other capabilities not specifically described here (e.g., mobile phone, global positioning system (GPS), power management, one or more cameras, various connection ports for connecting external devices or accessories, etc.). Further, while computer system 1300 is described with reference to particular blocks, it is to be understood that these blocks are defined for convenience of description and are not intended to imply a particular physical arrangement of component parts. Further, the blocks need not correspond to physically distinct components. Blocks can be configured to perform various operations, e.g., by programming a processor or providing appropriate control circuitry, and various blocks might or might not be reconfigurable depending on how the initial configuration is obtained. Embodiments of the present invention can be realized in a variety of apparatus including electronic devices implemented using any combination of circuitry and software.
Network 1406 may include one or more communication networks, which could be the Internet, a local area network (LAN), a wide area network (WAN), a wireless or wired network, an Intranet, a private network, a public network, a switched network, or any other suitable communication network. Network 1406 may include many interconnected systems and communication links including but not restricted to hardwire links, optical links, satellite or other wireless communications links, wave propagation links, or any other ways for communication of information. Various communication protocols may be used to facilitate communication of information via network 1406, including but not restricted to TCP/IP, HTTP protocols, extensible markup language (XML), wireless application protocol (WAP), protocols under development by industry standard organizations, vendor-specific protocols, customized protocols, and others. In the configuration depicted in
In the configuration depicted in
It should be appreciated that various different distributed system configurations are possible, which may be different from distributed system 1400 depicted in
While the invention has been described with respect to specific embodiments, one skilled in the art will recognize that numerous modifications are possible. Thus, although the invention has been described with respect to specific embodiments, it will be appreciated that the invention is intended to cover all modifications and equivalents within the scope of the following claims.
The above disclosure provides examples and aspects relating to various embodiments within the scope of claims, appended hereto or later added in accordance with applicable law. However, these examples are not limiting as to how any disclosed aspect may be implemented,
All the features disclosed in this specification (including any accompanying claims, abstract, and drawings) can be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise. Thus, unless expressly stated otherwise, each feature disclosed is one example only of a generic series of equivalent or similar features.
Any element in a claim that does not explicitly state “means for” performing a specified function, or “step for” performing a specific function, is not to be interpreted as a “means” or “step” clause as specified in 35 U.S.C. §112, sixth paragraph. In particular, the use of “step of” in the claims herein is not intended to invoke the provisions of 35 U.S.C. §112, sixth paragraph.
| Number | Name | Date | Kind |
|---|---|---|---|
| 6051771 | Iizuka | Apr 2000 | A |
| 8153882 | Adam | Apr 2012 | B2 |
| 8618404 | O'Dwyer | Dec 2013 | B2 |
| 9105260 | Helms | Aug 2015 | B1 |
| 20030066692 | Devige | Apr 2003 | A1 |
| 20060075887 | Shotwell | Apr 2006 | A1 |
| 20060180007 | McClinsey | Aug 2006 | A1 |
| 20080072744 | Ito | Mar 2008 | A1 |
| 20100132536 | O'Dwyer | Jun 2010 | A1 |
| 20110011246 | Buskies | Jan 2011 | A1 |
| 20120014673 | O'Dwyer | Jan 2012 | A1 |
| 20130233155 | Little | Sep 2013 | A1 |
| 20130340594 | Uemura | Dec 2013 | A1 |
| 20140053710 | Serletic, II | Feb 2014 | A1 |
| 20140140536 | Serletic, II | May 2014 | A1 |
| 20150013527 | Buskies | Jan 2015 | A1 |
| 20150013528 | Buskies | Jan 2015 | A1 |
| 20150013532 | Adam | Jan 2015 | A1 |
| 20150013533 | Buskies | Jan 2015 | A1 |
| Number | Date | Country | |
|---|---|---|---|
| 20160093277 A1 | Mar 2016 | US |
