FIELD OF THE INVENTION
The present invention relates to the field of digital musical instruments, and in particular the field of expressive control of virtual instruments.
BACKGROUND OF THE INVENTION
MIDI keyboard controllers are a flexible tool for keyboardists. They allow a keyboardist to use their existing knowledge of keyboard layout to play a multitude of different instrument sounds. However, not all instruments are sufficiently expressively controlled by a keyboard. A MIDI keyboard can come close to simulating the expressivity and feel of an acoustic piano, but it is difficult for a keyboardist using a MIDI keyboard to reproduce the nuances of a strummed guitar or percussion instrument.
There have been musical controllers that allow for virtual strumming and control of virtual percussion instruments, but none sufficiently capture the full nuance and rhythmic expressiveness of a real acoustic instruments while utilizing a keyboardists existing knowledge of the keyboard.
What is still very much needed is a musical controller that has the expressive potential to capture the nuances of a guitar or acoustic drum.
OBJECTS OF THE INVENTION
It is an object of this invention to create a more expressive controller for musicians.
Other objects and advantages of the invention will be apparent from the specification and drawings.
SUMMARY
In order to offer musicians a more nuanced way to interact with virtual instruments, one embodiment is a device comprised of acoustic elements with embedded vibration sensors that the user actuates for example by plucking, strumming, or striking the elements. A vibration sensor attached to each acoustic element is used to generate an excitation signal for musical instrument models.
Another embodiment is a method for actuating an acoustic element and a note input apparatus such that a signal from the acoustic element is sent to a musical instrument model as the excitation signal, and the note data from the note input apparatus is sent to the instrument model as pitch information.
Another embodiment is a device comprised of acoustic elements with embedded vibration sensors that the user actuates for example by plucking, strumming, or striking the elements. A vibration sensor attached to each acoustic element is used to generate an excitation signal for musical instrument models, and a note input device is included so that note input such as MIDI can be used to control the musical pitch of the musical instrument models.
BRIEF DESCRIPTION OF THE DRAWINGS
For a more complete understanding of the invention, reference is made to the following description and accompanying drawings, in which:
FIG. 1 shows an exemplary embodiment of the invention;
FIG. 2 shows different tine shapes and orientations;
FIG. 3 shows a schematic for the signal topology of some embodiments;
FIG. 4 shows a schematic for single-channel-in topology;
FIG. 5 shows a block diagram of a typical Karplus Strong string physical model implementation [PRIOR ART];
FIG. 6 shows a block diagram of the invention synthesis;
FIG. 7 shows an embodiment for percussion synthesis;
FIG. 8 shows an embodiment of the “magic pick”;
FIG. 9 shows the signal topology of a “magic pick” embodiment;
FIG. 10 shows an embodiment with an audio input and with a form factor of a floor pedal;
FIG. 11 shows an embodiment with an onboard keyboard;
FIG. 12 shows an embodiment with the form factor of a traditional keyboard;
FIG. 13 shows the system utilizing a mobile device; and
FIG. 14 shows the instrument as an audio plugin for a digital audio workstation.
DETAILED DESCRIPTION OF EMBODIMENTS
The arrangement in FIG. 1 shows an exemplary embodiment of the invention. The embodiment has a body [1] that holds one or more tines [2] that protrude outward from the body and are plucked by the user's hands. Each tine has a vibration sensor [ 3] made of piezoelectric film attached to it which converts the tine vibrations into a electric signals. The signals from the vibration sensors are digitized by an analog-to-digital converter and fed as an excitation signal into a digital physical model of a string instrument. In other embodiments, the digital physical model of a string can be replaced by any musical instrument model, analog or digital, for example an analog model of a marimba, a digital physical model of timpani, etc., and the acoustic signal from the vibration sensor is used by the musical instrument model as the excitation signal. The embodiment can also have a MIDI input port [5] that is configured to receive note data from a MIDI device [4]. In other embodiments the MIDI port can be replaced by some other note input device such as an Open Sound Control (OSC) port, USB MIDI port, onboard keyboard, capactive touch grid, touchscreen or tactile keyboard, or any other apparatus capable of receiving note or chord data. Other embodiments have a musical chord input device, and still others have no note interface at all. Notes played on the MIDI keyboard determine the musical pitch of the musical instrument model, as well as other parameters for example decay time, brightness, etc. In this embodiment, audio is outputted from a output jack [6]. Physical parameters of the digital physical string model can be adjusted by means of sensors [7] such as rotary dials, linear faders, force-sensitive sensors, capactitive touch sensors, foot pedals, or other means of tactile control.
The tines [2] in this embodiment are made of polycarbonate film. In other embodiments they can be made of plastic, wood, metal, or any stiff material with elastic properties. There are no strict restrictions on the number of tines, anywhere from 1 to 24 can be useful depending on the context. For example, one tine is useful for imitating rhythm guitar or solo guitar, whereas 24 strings are useful for recreating harp parts. The tines can range from 1/1000″-¼″ thick, with a recommended thickness of between 0.005″-0.03″. The length can range from ¼″ to 20″ long, and 0.1″-5″ wide depending on the context. The spacing between the tines can range from between 1/16″-3″ apart, with a recommended spacing of ⅜″-1″ apart, so long as they remain individually pluckable by hand. They can also take on different shapes and orientations, as shown in FIG. 2. For example, the tines can be rectangular and straight and parallel [8], or they can be rounded [9] for acoustical reasons, they can be placed at an various angles [10] to each other, and they can be different lengths [11]. In some embodiments the tines can be replaced by lengths of strings or wires stretched across a playing surface like a guitar or harp.
In this embodiment the vibration sensors are made of piezoelectric film that are glued to the tines. In other embodiments the vibration sensor can be another sensor such as a magnetic coil pickup, acoustic microphone, optical microphone, force sensing resistor, bend sensor, or some other sensor capable of sensing the tines. In some embodiments they can be attached to the tines by an adhesive, or with a laminate covering the tine and vibration sensor together, or with a press fit, or in some other configuration allowing the vibration sensors to sense broadband vibrations in the tines. In the case of acoustic or optical microphones, the vibration sensors can be decoupled from the tine structure and attached elsewhere on the body, so long as they are able to sense broadband vibrations in the tines. The vibration sensor can output signal energy in a broadband frequency range from 20 hz to 20 khz. Other embodiments can have a low frequency cutoff of the sensor signal between 0 hz-400 Hz and a high frequency cutoff between 8 kHz-96 kHz, with a recommended frequency range of 20 hz to 20 kHz. The analog-to-digital converter in this embodiment samples the signal at sample rate of 44.1 kHz, other embodiments can have analog-to-digital converter settings performed at audio rates, between 4 khz to 192 khz samples per second, with a recommended sample rate being above 16 kHz.
There are no strict restrictions on body size, so long as they can securely support the tines, anywhere from having a 2″ major dimension to an 8′ major dimension, and the shape can vary by context. For example, some embodiments can be designed to fit on the user's lap, as shown in FIG. 11, or suspended by a strap like a guitar, or placed on the floor, or attached to a keyboard stand or microphone stand, and the size would vary in these cases.
FIG. 3 shows the signal topology of the embodiment in FIG. 1. When the user plucks the tines, the signal generated by the vibration sensor [14] is routed into an analog to digital converter [15] and the digitized signal is fed into a separate digital physical model of string instruments [22] by means of a tine-signal routing matrix [16]. As specified before, the digital physical model of string instruments [22] can be replaced by other musical instrument models. MIDI note data [18] from a MIDI keyboard input [17] is sent into a note logical assignment block [19] which distributes the note pitch information as well as sustain and dampening information to the digital physical model of string instruments [22], for example in ascending musical pitch order. The tine-signal routing matrix [16] and note logical assignment block [19] work together to determine which tine signal is routed to which string model, based on the MIDI note data [18]. There are many different ways this can work depending on the playing context. For example, if there are two MIDI notes depressed on the MIDI keyboard input [17], the note logical assignment block [19] might fully dampen the physical characteristic of one digital physical string model, and then set the musical pitch data of the remaining physical string models so that the signals received from the vibration sensors are assigned to the digital physical models in ascending note order. Alternatively, the tine-signal routing matrix [16] can be simplified to a simple signal pass through for other playing contexts. The digital output of each model are summed together in a summer [23] block and outputted to a digital-to-analog converter [24]. The special case where the tine-signal routing matrix [16] is configured to route one signal to every digital physical model of string instruments [22] is called single-in topology. A simplified view of single-in topology is shown in FIG. 4. This topology can be useful in contexts where there is only one tine, or where it is desired for a single tine to sound with multiple notes as in a musical chord.
A schematic of an embodiment of a musical instrument model is shown in FIG. 6. A generalized schematic of the Karplus-Strong method of physical modeling of string instruments is shown in FIG. 5. In Karplus-Strong synthesis, shown in FIG. 5, there is an excitation signal [37], which is fed into a delay line [38] and fed back into the delay line through a low-pass filter block with gain [39], approximating the impulse response of a vibrating string. In Karplus-Strong synthesis, the excitation signal [37] is a finite-duration waveform that is preset or parameterized such as a filtered noise burst, a single impulse, or a short recording of a pluck attack. Playback of a Karplus-Strong model consists of triggering playback of the excitation signal, often in response to MIDI note input [36]. In the present embodiment (FIG. 6), the excitation signal [37] is replaced by a continuous audio signal [44] from the sensor [42] on the tine. In this embodiment, incoming MIDI note data does not trigger the onset of any sounds but instead simply assigns the musical pitch to the modified Karplus-Strong physical model of a string. In some embodiments other parameters of the musical instrument model can be controlled by the note data as well, for example, a MIDI note-on message can be used to open a gate and increase the decay time for the audio signal [44] to come through, and a MIDI note-off can be used to decrease the decay time of the musical instrument model without closing the gate allowing the audio signal [44] signals to continue through resulting in a dampened string sound.
Another embodiment is the percussion embodiment, shown in FIG. 7, where the tines are replaced by percussion pads [46] which are struck by the user by means of a mallot [51], drumstick, or by hand or foot. The percussion pads [46] each have a sensor attached to them which convert the pad vibrations into a electric signals, and just as in FIG. 3 the signal from the vibration sensor is used as the excitation signal for a musical instrument model, and note information from a MIDI input port [48] is used in this embodiment to determine the pitch of the musical instrument model. The percussion pads [46] are made of ⅛″ wood strips in this embodiment, but in other embodiments they can be made of any substance that produces a sound when struck, for example, wood, plastic, rubber, or metal. The percussion pads [46] can vary greatly in thickness, size, and shape for acoustic reasons, from 0.1″ to 3″ thick, 0.5″ to 15″ long and wide. The pads in this embodiment are mounted on the surface of the body [45], but in other embodiments the pads can be mounted in different ways, for example being suspended outward from the body to allow for striking on both sides, or protruding at different angles from the body. Varying the mounting type will result in the different acoustic properties and also can be changed according to the desired sound. The vibration sensor can be a peizoelectric element, magnetic coil pickup, acoustic microphone, optical microphone, force sensing resistor, or some other form of vibration sensor. The vibration sensors are glued to the bottom of the percussion pads [46], but in other embodiments the vibration sensors can be attached by example by a different adhesive, or attached by a clamp or screw, or it can be press fit into place by the body structure. Or, in the case of acoustic microphones, they can be decoupled from the tine structure and rely on airborne acoustic transmission, or in the case of optical microphones, attached elsewhere on the body.
Another embodiment is exemplified in figure FIG. 8, which we refer to as the “magic pick” embodiment. This embodiment has a plectrum [52] with a sensor attached that sends a signal either by a audio cable [53] or wirelessly to a processor [56]. The embodiment has a MIDI input [57] and an audio output [55]. In other embodiments, the MIDI input is a OSC input, usb, onboard keyboard, chord buttons, or some other note input device. This allows for strumming on any jagged surface with only a plectrum and a single analog-to-digital converter.
The topology of the “magic pick” is shown in FIG. 9. Signals generated from the vibration sensor on the plectrum [58] are digitized by an analog-to-digital converter [61] and fed into a signal pluck separator [62] that identifies individual plucks and separates the individual pluck signals into different channels and sends them into a stroke detection block [63] to determine whether the direction of the strum is up-stroke or down-stroke. The individual pluck signals are then sent as excitation signals to a musical instrument model [64], distributing according to whether the signal was generated by an up-stroke or a down stroke. As with FIG. 3, musical pitch information from the MIDI input [68] is used to control the pitches of the physical models. In another embodiment of the “magic pick,” the signal from the vibration sensor is converted into MIDI output data as a strum controller for any MIDI synthesizer or software.
The “magic pick” is generally the size of a guitar plectrum but can vary, from ½″ long to 4″ long depending on the context. Like the tines in FIG. 1, they can be made of plastic, wood, or metal, or other appropriate material, and range from 0.01″ thick to ¼″ thick. Some embodiments of the “magic pick” include a handle made from wood or plastic, from ½″ to 6″ long, often with sensors on the handle for extra control. The sensor on the “magic pick” is made of piezoelectric film, though in other embodiments it can be a magnetic coil pickup, acoustic microphone, optical microphone, force sensing resistor, bend sensor, or other sensor capable of sensing the plectrum.
An embodiment related to the “magic pick” is the “magic washboard,” which uses the same topology as the “magic pick” (FIG. 9), but instead the sensor is attached to the washboard or other corrugated surface instead of on the plectrum itself.
All embodiments can use audio input jacks to receive excitation data for the musical instrument models. An example of this is shown in FIG. 10. In this embodiment, only one input jack is used, though more inputs are possible, and the body [76] of the embodiment is in the form of a floor pedal. In this embodiment a contact pickup [70] attached to the drum, and vibrations generated from the drum or percussion instrument [71] are picked up by the contact pickup [70] and plugged into the audio input jack [74], where the inputs are digitized and fed into a physical instrument model as in FIG. 4, replacing the tines and vibration sensors with the audio from the input jack. Onboard foot control pedals [77] can be used to determine the pitch, damping, and parameters of the musical instrument model. There can also be a MIDI input [75] to receive note data from a MIDI keyboard. The output [78] of the instrument is then fed into an amplifier [79].
All embodiments can have an onboard musical keyboard as the note input device. For example, FIG. 11 and FIG. 12 show example embodiments with onboard musical keyboards. FIG. 12 shows an embodiment taking the form factor of a traditional musical keyboard with tines [81] protruding from the keyboard body. FIG. 11 shows an alternate form factor where the body sits on the user's lap and has a keyboard [80] attached in a way that resembles an the form factor of an accordion. Other embodiments can have keyboards that function in alternate ways, for example by capacitive touch, or by touchscreen. Alternate note layouts, for example isomorphic keyboard layouts, are also possible.
All embodiments can do the digital processing and user input using a mobile device, as shown in FIG. 13. In these embodiment, the audio signals from the instrument body [83] are fed into the mobile device [82], either as digital or analog signals. The keyboard entry takes place on the mobile device screen, and the digital processing takes place as an app or software on the mobile device. The audio is then outputted from the mobile device, or from an audio interface. In a related embodiment, the embodiment also functions as an audio interface for the mobile device.
All embodiments can do the digital processing and MIDI input using a software plugin for a digital audio workstation (DAW) such as a VST, as in FIG. 14. In this embodiment, the audio signals from the tines are fed into a computer, either through an audio interface or where the embodiment itself functions as an audio interface. The raw pluck signals are saved to a track in the DAW, and an accompanying VST plugin converts the plucked sounds into musical instrument models.
All embodiments can also include an acoustic speaker or transducer on the body.
All embodiments can also include one or more external foot pedals for additional control.
All embodiments can also include effects processors in the signal chain, such as reverb, delays, loopers, distortion, or other musical audio effects to enhance the output audio. Advanced effects include an audio looper to loop the raw pluck signals from anywhere to 0.5 seconds to 5 minutes in length, with recommended lengths of 1 to 5 seconds, to allow looped rhythmic playback while the user plays different notes or chords on the MIDI keyboard.
All embodiments can be played by more than one player or modified to do so, with one person actuating the acoustic elements while another person plays the keyboard input.
It will thus be seen that the objects set forth above, among those made apparent from the preceding description, are efficiently attained and, because certain changes may be made in carrying out the above method and in the construction(s) set forth without departing from the spirit and scope of the invention, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative and not in a limiting sense.
The invention has been described with respect to various embodiments having various features. A person of ordinary skill in the art will appreciate that the scope of the invention described herein also includes embodiments incorporating one or more of these features in combinations other than those expressly described herein.
The various devices, methods, procedures, and techniques described above provide a number of ways to carry out the invention. Of course, it is to be understood that not necessarily all objectives or advantages described may be achieved in accordance with any particular embodiment described herein. Also, although the invention has been disclosed in the context of certain embodiments and examples, it will be understood by those skilled in the art that the invention extends beyond the specifically disclosed embodiments to other alternative embodiments and/or uses and obvious modifications and equivalents thereof. Accordingly, the invention is not intended to be limited by the specific disclosures of preferred embodiments herein.
Patent Grant
10360887
