The present invention relates to electronic musical instruments.
The present invention provides a system and methods for an electronic musical instrument. Through a novel combination of sensor inputs, it allows simulation of real world instruments including but not limited to a Trombone, Trumpet and Saxophone.
The device itself includes a series of sensor inputs configured to act as a user interface, and a speaker to output sound. Various sensors can be employed, including a touch screen, microphone, accelerometer, and camera or light sensor.
Sensor inputs are processed through a set of sub-processors to determine events and respond accordingly with parameters and actions for manipulating sound. Attributes that can be varied include tone, pitch, attack/accent (also known as velocity), volume, and special modes such as vibrato, growl or tonguing. Parameters and commands are sent to a playback processor which responds to the input parameters and commands by processing stored digital representations of sounds and sends them to an output buffer for playback.
Generated sounds are stored digitally as either data, or algorithms/equations. They are contained within a Tone data object which comprises a set of representations which may provide different phases and/or qualities.
Sensor inputs can be configured to trigger playback of sound and control its various attributes either alone, or in combination. For example, Tone and pitch may be determined exclusively by location of touches on a display, or by a combination of device rotation and touch location. These methods are illustrated by a variety of embodiments including a simulated Trombone, Trumpet, and Saxophone.
Further objects, advantages, and features of the invention will become apparent from a consideration of the drawings and ensuing description.
Presently preferred embodiments of the invention are described below in conjunction with the appended drawing figures, wherein like reference numerals refer to the like elements in the various figures, and wherein:
The system of the present invention comprises an electronic device with sensor inputs configured to act as a user interface and speaker output to produce sound responsive to the inputs.
It has a speaker 150 for outputting sound, one or more digital sound representations, a memory 160 for storing them, and a processor 170 for executing software capable of receiving configuration parameters, maintaining state, receiving sensor input data, processing the input data, and responding. The response is done in accordance with the configuration parameters, system state, and the input events. It involves controlling playback of audio through the speaker; sounds may be started and stopped and attributes such as tone, pitch, accent, nuance, volume, and vibrato may be varied. A power source powers the device 180, and display may be attached to the touch screen or separate 115.
Sound Representation
Audio to be output is represented digitally within a data object called a Tone. As shown in
One or more representations of the Tone which offer different musical nuance with the same inherent pitch may be contained within the Tone. For example, the Tone may consist of a set of attack, loop and decay files which have a strong accent and vibrato, and another set of which have a soft accent and a steady sustain. Parameters for selecting one set versus another are also stored within the Tone model and associated with each set. An example of such a parameter would be, “Volume>0.5”, which would indicate that the particular representation by played if the volume output is above 0.5.
In some embodiments, sound waveforms may also be generated by algorithmic and/or mathematical models, or some combination thereof. In this case, the algorithm or model is associated with the Tone. If no stored representations are used, the pitch may be set directly.
Event Processing and Output
As shown in
As shown in
The audio output sub-processor is responsible for receiving and executing instructions on sound playback.
The process of
Methods of Triggering Sound and Setting Attributes
Sounds are triggered and their attributes set by the inputs, alone, or in combination. Inputs may require varying degrees of processing, for example accelerometer input can be filtered to determine angle change or vibration; mic input can be processed to determine level or pitch. Derivative methods may also be employed, for example, in the case of using touch as a trigger, duration between touch events may be used to determine whether a fast attack or a slow attack should be played. (Attack is often referred to as, or linked to note velocity).
Table 1 summarizes various methods by which sounds are triggered and attributes set.
Several of these methods are illustrated by embodiments representing real instruments including a Trombone, a Trumpet, and a Saxophone.
Trombone
By tightening lips (embouchure) and “buzzing” at a higher frequency, users can increase the pitch to a higher partial in the overtone series. Quality, nuance and volume are determined largely by the embouchure, and air speed and direction.
As embodied by the present invention. The device has a touch display 600, a mic 610, and speaker 620, with additional sensors and processor electronics contained within the case.
The display is partitioned into 8 overtone partials 630 on the Y-axis, and 7 slide positions 640 along the X-axis, and shows a corresponding image. Sound is triggered when a user either blows into the mic, or touches the display. Pitch is determined by the location of the touch on the display. Volume is determined by mic level, force of touch (or area of touch) on the display, or angle of the device as determined by an accelerometer. Attack type, note quality and other nuance are determined by shaking the device, or may be linked directly to volume or duration of notes.
In determining the Tone and pitch, the partial is first determined from the location along the Y-axis. A base Tone (
Thus, for example, with a display 320 pixels high and 8 partials assigned, a touch at Y-position of 310 pixels would fall within the 8th partial, and correspond to a base Tone of Bb4.
A pitch adjustment of the base Tone is then determined. First, the number of semitones variation due to slide extension is calculated from the X-axis touch location according to the following equation (we assume the slide is equal to the entire display width):
Slide semitones=X position pixels*(6 semitones/Display width pixels)
This value is then added to a pre-configured number of adjustment semitones for the previously determined Tone. Sample adjustment semitone values are shown in Table 2.
Total semitones=Adjustment semitones+Slide semitones
The total semitones are then used to calculate the pitch adjustment by the following formula:
Pitch adjustment=2^(Total semitones/12)
Therefore, in this particular example, assuming display dimensions of 480 pixels wide by 320 pixels high, if the user touches location (200 pixels, 310 pixels), the touch falls within the 8th partial which corresponds to the base Tone of Bb4 and has two Adjustment semitones. The final pitch adjustment is calculated as follows:
Slide semitones=200 pixels*(6 semitones/480 pixels)=2.5 semitones
Total semitones=2+2.5=4.5 semitones
Pitch adjustment=2^(4.5/12)=1.3
With the Tone selected, a sound type, if available may also be selected 710. For example, if the volume, force (or touch area), and/or shake is above a certain threshold, a different attack type may be selected. Table 3 shows sample activation parameters for selecting different attack and loop types. Note that the volume may be determined from force (or area) of touch or from one of the additional sensor inputs, such as mic level, or accelerometer angle. In this case, a delay may be added to ensure that the external event is determined and flag set prior to determining the type. Attack type may also be determined from the duration between successive touches; if short, then a faster attack is used, whereas if long, a slower attack is used. In order to calculate the duration between successive touches the time of last touch must be stored and then later subtracted from the time of current touch.
With qualities of the note determined, the Tone, its type, and pitch adjustment are sent 712 to the playback processor. If 714 configured to trigger sound by touch, the playback command is sent 716 to the playback processor.
If 704 a touch is determined to have moved, a similar process is followed. The Tone and pitch adjustment are determined 718, as previously described; however, if the partial has changed from the previous partial, such as if a player was moving from a Bb up one partial to a D, a “slur” can be assumed, and the playback processor is sent 720 a slur request with the new Tone and pitch adjustment. Otherwise, if the movement has occurred within a partial, the new pitch is requested 720 of the playback processor such that it can continue to use the same base Tone but adjust the pitch.
Finally, if 706 a touch is determined to have ended, and the system is configured to trigger by touch 722, a stop is requested 724 of the playback processor. A decay phase may also be employed. In this case, the playback processor will playback a decay segment before ramping down and stopping playback. In a modified embodiment, the type of decay phase may first be determined (for example, fast vs. slow), and then sent to the playback processor along with the request for stop.
If 904 a shake event is detected, a flag that the event occurred and the time at which it occurred is set 910, such that any of the event processors responsible for starting playback may refer to it to determine attack type. In a modified embodiment, the shake could be configured to start and stop the sound playback, as well. In yet another embodiment, the shake could be configured to request a special playback mode of the playback processor, such as a rapid fire tonguing mode where the notes are started and stopped rapidly rather than sustained.
Trumpet
The valves are numbered 1 through 3, starting with the valve closest to the mouthpiece. The first valve decreases the pitch by 2 semitones, the second by a semitone, and the third by 3 semitones. Simultaneously, by tightening lips (embouchure) and “buzzing” at a higher frequency, users can increase the pitch to a higher partial in the overtone series. Quality, nuance and volume are determined largely by the embouchure, and air speed and direction.
As embodied by the present invention. The device has a touch display 1100, a mic 1110, and speaker 1120, with additional sensors and processor electronics contained within the case.
Various embodiments are presented. One set of embodiments determines Tone and pitch by touch exclusively, whereas another set of embodiments determines Tone and pitch by a combination of touch location and device rotation.
In
In a variant of
In
In each of the embodiments, the sound may be triggered by various methods including, but not limited to touch, and mic levels. If mic levels are used, the open valve area is not required for embodiments of
Display sensor information is received 1500 periodically, and processed to determine whether a touch as begun 1502, moved 1504, or ended 1506. If a touch has begun, the Tone and pitch adjustment are determined 1508 through one of several methods depending on embodiment
In embodiments of
The semitone adjustment due to the valve presses is then determined. 1st valve closed, 2nd valve closed, and 3rd valve closed cause 2, 1, and 3 semitone decreases, respectively. The semitone decrease is additive, such that if 1st and 2nd valves are closed, there is a 3 semitone decrease; likewise, if 1st and 3rd valves are closed, there is a 5 semitone decrease.
With the valve semitones determined, the total semitone adjustment from base Tone pitch can be determined.
Total semitones=Adjustment semitones+Valve semitones
The total semitones are then used to calculate the pitch adjustment by the following formula:
Pitch adjustment=2^(Total semitones/12)
A similar procedure is followed for the embodiments of
When the touch event is received, the device angle is determined from the accelerometer data, and matched to find the associated partial, base Tone, and adjustment semitones. Table 5 shows an example of the association.
Determination of the pitch adjustment proceeds as described for the other embodiments. In order to ensure that the angle is determined prior to partial being determined, a slight delay may be inserted.
With Tone and pitch determined, the type of attack or other quality of Tone is found 1510 as described in the Trombone embodiment. Finally, with Tone, pitch adjustment, and other Tone quality determined, the parameters are sent 1512 to the playback processor, and if 1514 set to trigger playback by touch, playback is requested 1516.
A similar process is followed if a touch moved event is received 1504. A new Tone, pitch adjustment, and note quality are determined 1518. If the Tone or partial changes a slur may be signaled 1520 to the playback processor along with the other Tone parameters.
Finally, if a touch end event is received, and 1522 the system is configured to trigger playback by touch, a playback stop is requested 1524 of the playback processor.
As in the previously described Trombone embodiment,
If 1704 the angle change occurs about an axis configured to correspond to volume, the volume can be determined 1714 as previously described in accordance with FIG for the Trombone embodiment. With volume determined, it is sent 1716 to the playback processor.
If 1706 a shake event is detected, a flag that the event occurred and the time at which it occurred is set 1718, such that any of the event processors responsible for starting playback may refer to it to determine attack type. In a modified embodiment, the shake could be configured to start and stop the sound playback, as well.
Saxophone
By changing the oral cavity users can “lip up” to higher partials to play altissimo notes. However, they can reach many notes by the standard keys, which include the octave key. Quality, nuance and volume are determined largely by the shape of the oral cavity, lip position, wind speed and direction.
As embodied by the present invention. The device has a touch display 1800, a mic 1810, and speaker 1820, with additional sensors and processor electronics contained within the case.
Areas for each key are defined on the display. There are the left hand main keys (B, A/C, G, front F, and Bb), palm keys (D, Eb, F), and little finger keys (G#, Low C#, Low B, Low Bb). There are also right hand main keys (F, E, D, F#), side keys (E, C, Bb, High F#), and little finger keys (Low Eb, Low C). A thumb key for changing octave may also be located on the display, or an alternate input may be used, such as the camera 1840 located on the back of the device. If sound is to be triggered by touch, an open key area is also defined to indicate that no keys are pressed, but sound is to be played. Base Tone and pitch are determined by location of touches in these regions. As with other embodiments, volume is determined by mic level, force (or area) of touch on the display, or angle of the device as determined by an accelerometer. Attack type, note quality and other nuance are determined by shaking the device, or may be linked directly to volume, or duration of notes.
Similarly to the other previously described embodiments, partial or level is first determined, followed by adjustment due to key presses. The Saxophone differs from the Trumpet embodiments in that there is less reliance on partial shift, and more on key press shift. With the standard key arrangement (including thumb octave key) the instrument is capable of two and a half octaves. Altissimo registers can also be reached extending the range to 3 or even 4 octaves.
Partial, or octave shift, can be set through various methods. In one embodiment (
Locations of the touches are then used to determine key presses. As with the other embodiments, the semitone shift due to key presses is then added to the base Tone adjustment semitones to determine the final pitch shift of the base Tone.
Attack type and other qualities of the note is then determined 2010. With Tone, pitch adjustment, note quality and any other parameters determined, they are sent 1512 to the playback processor. If 2014 configured to trigger playback by touch, playback is also requested 2016.
A similar process is followed if 2004 a touch moved event is received. A new Tone, pitch adjustment, and note quality are determined 2018. If the note changes a slur may be signaled 2020 to the playback processor along with the other Tone parameters.
Finally, if 2006 a touch end event is received and 2022 playback is configured to be triggered by touch, a playback stop is requested 2024 of the playback processor.
The invention has now been described with reference to the preferred embodiments. Alternatives and substitutions will now be apparent to persons of skill in the art.
The present invention claims priority to provisional U.S. patent application Ser. No. 61/153,584 filed Feb. 18, 2009.
Number | Name | Date | Kind |
---|---|---|---|
5763804 | Rigopulos et al. | Jun 1998 | A |
6011212 | Rigopulos et al. | Jan 2000 | A |
7161079 | Nishitani et al. | Jan 2007 | B2 |
7394012 | Schultz | Jul 2008 | B2 |
7772476 | Lemons | Aug 2010 | B2 |
7842877 | Charles | Nov 2010 | B2 |
20020026866 | Nishitani et al. | Mar 2002 | A1 |
20070044638 | Egan | Mar 2007 | A1 |
20100206156 | Scharfeld | Aug 2010 | A1 |
Number | Date | Country | |
---|---|---|---|
20100206156 A1 | Aug 2010 | US |
Number | Date | Country | |
---|---|---|---|
61153584 | Feb 2009 | US |