Patent Grant
9773480
The present invention relates to percussion controllers.
A musical instrument that produces sound as a result of one object striking another is known as a “percussion” instrument. The striking object can be a person's hands/fingers, such as when one plays bongos or a piano. Or the striking object can be something held by a musician, such as a drum stick, mallet, or beater, for striking a drum or triangle, for example.
A percussion “controller” is an electronic device that senses impacts and pressures associated with performing musical rhythms using virtual music software and sound synthesis in conjunction with either computers or electronic musical instruments, such as synthesizers. The performer typically uses the controller to accompany other performers who are using other instruments, for example, trumpets, pianos, guitars, etc. In other words, an electronic drum set has both a percussion controller and a drum synthesizer. Triggered by the performer, the percussion controller sends messages, which contain information about pitch, intensity, volume level, tempo, etc., to devices that actually create the percussive sounds. Percussion controllers are available in a variety of different forms and vary widely in capabilities.
Basic percussion controllers typically include a set of resilient (e.g., rubber or rubber-like, etc.) pads that can be played with either drum sticks or the musician's hands and fingers. In some cases, these controllers are integrated with a synthesizer. In such cases, the synthesizer generates rhythm “signals,” which produce rhythm sounds after transmission to and playback over an audio system. The percussion controllers and synthesizer are sometimes federated (i.e., separate devices), which enables buyers to select a best controller and a best synthesizer from different manufacturers.
Percussion controllers may also be capable of receiving the triggering rhythm patterns on conventional percussion instruments, such as acoustic drum sets, cymbals, and hand drums. To do so, the acoustic instrument is typically equipped with electronic triggers.
Drummers can also choose to retrofit a traditional acoustic drum kit with a controller and drum/cymbal triggers. This enables the drummer to add his own acoustic accompaniment to the sounds generated by the controller, thereby creating rhythmic effects that would otherwise be impossible using traditional percussion instruments alone. Many drummers today are combining their acoustic drums with additional percussion controllers. This enables them to achieve the dynamics and responsive feeling attainable only from actual drums and cymbals, while also realizing the benefits of compactness and electronic convenience of triggered percussion sounds, like cow bells and ago-go bells, wood blocks, conga drums, gongs, tympani, and the like.
Although quite useful for expanding the sound-generating capabilities of a musician, currently-available percussion controllers are not without their limitations and drawbacks.
First, conventional percussion controllers sense the dynamics of impacts in a predefined physical impact zone that is instrumented with pressure- or force-detecting sensors. The controllers then process the sensor signals. This technique of electronic sensing captures only a limited part of the dynamic range of the percussions.
Also, to the extent that the percussion requires more sensors, such additional sensors can interfere with one another. Increased processing is required to remove this “cross-talk,” which further reduces the dynamic range available. In fact, the signal processing exhibits combinatorial growth for each additional sensor. This approach to sensing thus limits the ability of the controller to accurately capture a percussionist's performance, limits the number of impact zones available to the percussionist, and drives up the cost of the percussion controller itself.
The performer notices these limitations as occasional false notes and a general lack of realism responding to the thrown forces. A design that reduces the occurrence of false notes results in a reduction in dynamic responsiveness. Furthermore, the performer also notices a lack of tonal dynamic response to strike placement as compared with the way that acoustic percussion instruments naturally respond. Consider that a snare drum exhibits a continuum of tones depending on where the strike is placed. Typical percussion controllers offer one or two positional sound variations. Although rather impractical, it would take hundreds of sensors across a fourteen-inch-diameter surface to recreate the tonal location sensitivity of a single snare drum batter. The same locational sensitivity occurs for a ride cymbal (about 20 inches in diameter), for a hi-hat (about 14 inches in diameter), and perhaps to a lesser extent for crash cymbals and tom-toms. As a consequence, a trap-set percussion controller with realistic locational sensitivity would require many thousands of sensors.
Second, percussionists use many different techniques; for example finger throwing, finger muting, stick throwing, mallet throwing, etc. Conventional percussion controllers are custom designed for one or another of these techniques.
Further consideration of stick throwing reveals different striking techniques, such as by using the stick's tip, shank, or butt. Striking an acoustic percussion instrument using these different techniques results in different sounds. Conventional percussion controllers are unable to detect and respond differently to these different percussive techniques.
Also, percussion instruments exhibit a wide variation of physical arrangements (e.g., a trap set, a snare drum, a triangle, maracas, a tympani, a xylophone, a piano, etc.). So, notwithstanding the flexibility potentially provided by an electronic implementation of an instrument, an electronic multi-percussionist will nevertheless be forced to purchase many different custom-designed percussion controllers (e.g., an electronic xylophone, an electronic trap-set, and an electronic hand-drum, etc.).
Third, a percussionist' ability to place a strike improves with training and practice. This improved ability enables a percussionist to direct a strike to increasingly specific (i.e., smaller) regions of an instrument with increasing accuracy. Unfortunately, existing custom-designed percussion controllers do not possess an ability to decrease the spacing between striking zones, which would enable the creation of additional striking zones. As a consequence, with improvement, the percussionist either compromises their abilities with the more basic controller or buys, at significant expense, a new controller more suitable to their improved abilities. A far more desirable alternative would be for the percussion controller to have the ability to adapt to the improving percussionist.
Discussion of Conventional Percussion Controllers
Roland Corporation HandSonic 15.
This device is an electronic hand percussion multi-pad that, according to the manufacturer, permits a hand percussionist to play up to 600 acoustic and electronic percussion sounds, and up to 15 such sounds simultaneously.
The mat absorbs some of the impact from the hand/fingers and creates a rebound or bounce to provide a more natural feel to the performer. Below the mat, and under each physical-impact zone, is an individual pressure sensor. A structural base is disposed beneath the sensors. There may be stiff shock-isolating devices integrated between the base and the sensor. A small processor samples all the sensors, and processes each sensor signal to adjust the sensor's sensitivity, remove noise, and most significantly remove the structure-borne cross-talk that occurs when the physical impact on one sensor is acoustically transferred through the sensor to the base and subsequently into adjacent sensors.
Alternate Mode Inc. trapKat.
The HandSonic 15 includes a sound synthesizer, which is integrated with the sensor-signal processor. Some controllers, such as the trapKat electronic percussion system, do not integrate the synthesizer or provide the synthesizer as an option. In such products, the processor must send control signals to the synthesizer. In either case, when either an impact or a pressure is detected in a zone, the measured strength of the impact/pressure is mapped to a musical event message (typically in accordance with the MIDI protocol) that is sent to the synthesizer.
The trapKat, which is depicted in
The HandSonic 15 by Roland Corporation and the trapKat by Alternate Mode Inc. are similar in the sense that they both: (1) have a single structural base, (2) have sensors beneath an impact surface that is arranged into predefined zones, (3) process the array of sensor signals to remove noise and crosstalk, (4) detect zone impacts or pressures, and (5) map the zone impacts/pressures into events for synthesis.
The trapKat is designed to accommodate thrown (drum) sticks, which changes the arrangement and dimensions of the physical-impact zones. Although the trapKat can be configured to be played using hand or finger-throwing techniques, and it can map its zones to hand-percussion sounds, it is not as well suited to hand percussion as the HandSonic 15. Since neither the trapKat nor the HandSonic 15 is well suited to accommodate both stick and hand techniques, a multi-percussionist using these techniques would require both of these percussion controllers.
Roland Corporation's TD-9KX2-S V-Tour Series Drum Set.
A different approach to the trap-set percussion controller is illustrated by the TD-9KX2-S V-Tour Series drum set, depicted in
This federated sensor device approach features the ability for the percussionist to physically arrange and customize the layout of the physical-impact zones along structural rails. But the railing still couples structure-borne cross-talk from one impacted sensor to other sensors.
All the prior-art approaches to percussion controllers suffer certain common problems. In particular, a percussionist playing an acoustic percussion instrument performs with a very wide dynamic range, sometimes exceeding 120 dB, ranging from the barely audible “triple pianissimo” to the explosively loud “triple forte.” Sensors with such extreme dynamic range are very expensive. As a consequence, most percussion controllers use relatively less expensive sensors that disadvantageously cannot recreate such a broad dynamic range.
In summary, the drawbacks of existing percussion controllers include:
The present invention provides a percussion controller that is capable of exhibiting at least one and preferably more of the following characteristics/capabilities, among others:
The present inventor recognized that a percussion controller having the desired capabilities can be realized by decoupling the sensing of impact intensity (i.e., force of impact) from the impacted surface. That is, to the extent a percussionist strikes a sensor-enabled surface, information related to the strike is not used to determine the force of impact of the strike. Rather, the information related to the strike is being used to determine the location of impact of the strike.
The present inventor recognized that even further advantages accrue by decoupling both the sensing of impact intensity and the sensing of impact location from the impacted surface. That is, the sensor-enabled surface is not used to determine either the force of the strike or the location of the strike.
To decouple the force and location measurements from the impacted surface, information pertaining to the kinetics of the striker (e.g., a drumstick, mallet, hand, etc.), as the striker is “thrown” by the percussionist, is obtained before the striker impacts the surface. That information is then processed using inertial navigation (“IN”) techniques. This enables the force/pressure of the strike and location of the strike to be determined; that is, to be predicted, before the strike actually occurs.
It will be appreciated that if sensors are not being relied on for routine force and/or location determination, limitations arising from “cross-talk” become moot or of significantly reduced consequence. That results in improved dynamics, decreased cross-talk-induced triggering of false notes, no noise-related limitations on the size or configuration of “impact” zones, a reduction in processing-related time lags, and greatly increased utility since the surface can be freely reconfigured, among other benefits.
In the accordance with the illustrative embodiment, a percussion controller capable of achieving at least some of these objects comprises: (i) one or more instrumented strikers, (ii) a sensor-enabled striking surface, and (iii) a data processing system executing appropriate specialized software.
In the illustrative embodiment, the instrumented strikers include inertial sensing devices, which are capable of taking measurements related to the kinetics of the moving strikers. The sensor-enabled striking surface includes a mesh of contact (force/pressure) sensors that underlie a resilient striking surface.
In operation, a performer uses the instrumented striker(s) in the manner in which its non-instrumented analog is used. That is, the performer uses instrumented drum sticks in the same fashion as conventional drum sticks, etc. In the illustrative embodiment, readings from the inertial sensing device are transmitted from the instrumented strikers to the data processing system. In a significant departure from the prior art, the data processing system uses Inertial Navigation techniques to process the received data, predicting the force and, in some embodiments, the location of each impact before it actually occurs.
To relate the (predicted) location of a strike to a musical event (e.g., hitting a snare drum, etc.), the sensor-enabled surface is “virtually” segregated into a plurality of impact zones via the data processing system. Each such impact zone typically represents a different musical event. Prior to a first performance, the percussion controller is typically programmed to define and store a variety of impact zone arrangements. A desired arrangement is recalled by the performer before a performance. In some embodiments, the data processing system activates indicator lights that are associated with the sensor-enabled striking surface, thereby displaying the boundaries of the impact zones for the performer.
In the illustrative embodiment, with impact zones established and having predicted, via IN techniques, the force and location of the impact, the processor maps the predicted location into the appropriate predefined impact zone. This provides some information about a musical event (e.g., hitting a drum, etc.). The force prediction is used to provide additional information about the musical event; that is, how hard the drum is hit. In this fashion, the predicted force and location of the strike are mapped into musical events.
The percussion controller then generates musical event messages (e.g., via the MIDI protocol) for transmission to a synthesizer. The musical event messages control the synthesizer, causing it generate music signals that correspond to the received musical event messages. When amplified and delivered to a speaker, the musical signals result in desired sounds; that is, the musical performance.
Regardless of how information pertaining to the kinetics of the striker(s) is obtained (e.g., inertial measurements, EM interrogation, etc.) it must be transmitted to the data processing system without interfering with percussion performance techniques. To that end, in the illustrative embodiment, the data processing system and the measurement/sensing devices that obtain striker kinetics information are separated and communicate wirelessly with one another.
The sensor-enabled striking surface of the present percussion controller provides the following four functions, among any other others: (i) striker rebound; (ii) initialization; (iii) navigation error correction; and (iv) verification of IN predictions. These functions are discussed briefly below.
The presence of a resilient striking surface is very desirable. When a striker impacts a resilient striking surface, it rebounds, so as to more closely mimic an impact on an actual acoustic percussive instrument (e.g., drum heads, etc.).
IN needs to be initialized before it is used and requires ongoing error corrections. In accordance with the illustrative embodiment of the present invention, initialization and navigation error correction are accomplished by simply striking the sensor-enabled striking surface.
In some embodiments, the sensor-enabled striking surface is used to verify the predicted impact location. The force and/or location predictions will be issued a few milliseconds before actual impact on the striking surface. As a consequence, prediction accuracy will be very high, but there remains the possibility of extremely infrequent prediction errors. In such cases, at the time of impact, the data processing system might determine that there was a prediction error. Depending on the nature of the error, the data processing system may or may not take corrective action.
In some alternative embodiments, the striking surface is not sensor-enabled; it is simply a resilient striking pad. In such embodiments, an auxiliary instrumented pad is used to provide the initialization and updating functions. Since the percussionist would have to occasionally strike the auxiliary instrumented pad during a performance, such embodiments are less desirable than the illustrative embodiment in which the striking surface is instrumented. Furthermore, in such embodiments, the percussion controller will not be able to correct prediction errors.
It will be appreciated that by virtue of the techniques disclosed herein, musical event messages (e.g., a MIDI note-on, etc.) can be formatted and transmitted at predetermined intervals before an actual impact with the sensor-enabled striking surface. The performance is therefore enhanced since sensor-processing delay, event-mapping delay, event-message-formatting delay and queuing delay are eliminated.
In some embodiments, compensation is provided for the remaining “delays”: including transmission delay, sound-generation-processing delay, and buffering delay. A specialized application running in the data processing system has parameters for predefined external delays that are stored and recalled by the performer to account for a wide variety of synthesis modules and transmission technologies that are available.
In some embodiments, the percussion controller includes “virtual” impact zones. These virtual impact zones are not on the sensor-enabled striking surface; rather, they are in “space” near the performer. The virtual impact zones effectively expand the area of the sensor-enabled striking surface. They can be used, for example, to “place” virtual instruments (e.g., splash and crash cymbals, etc.) in the locations they would reside in an actual drum set. The virtual impact zone boundaries are programmable and can be stored and recalled by the performer. The data processing system, applying information from the instrumented striker to IN as previously discussed, predicts the striker's impact with the virtual impact zones. The subsequent mapping of impact zones and impact force into musical events for the synthesizer is performed in known fashion.
In some further embodiments, striker motion is tracked (using IN techniques) and then that motion is correlated against predefined motion patterns. The subsequent mapping of matched motion patterns into musical events for the synthesizer is an adaptation of a conventional method. In other words, in such embodiments, predefined “non-throwing” motions of a striker are interpreted as musical commands.
In some embodiments, the percussion controller is capable of serving several percussionists by appropriately adapting the linked-layer protocol for the (wireless) striker communications, thereby eliminating any potential radio interference problems that might otherwise occur.
In some additional embodiments, throwing positions and forces used by the percussionist are monitored for the purpose of improving technique. More particularly, the processor accesses position-matching and force-matching algorithms (in addition to IN). This enables a student's throwing technique to be measured with high accuracy and then compared to a prerecorded reference performance, such as that of a teacher, expert, etc. This is expected to rapidly improve a student's throwing technique.
In yet some further embodiments, position-matching and force-matching algorithms are used in conjunction with IN to provide a background process that gathers statistics related to various good and bad throwing techniques exhibited by the percussionist during a musical performance. The information can aid the percussionist in correcting bad habits.
In some embodiments, the sensor-enabled striking surface with which the musician primarily interacts to “play” a virtual instrument, is supplemented by one or more “instrumented mats.” The instrumented mat(s), which can be placed wherever convenient (e.g., on the floor at the musician's feet, etc.), can be used to control the operation of sensor-enabled striking surface. For example, the additional mat can be programmed so that:
In some embodiments, the instrumented mats employ the same type of IN processing as the sensor-enabled striking surface, such that use of the mats require an instrumented “striker;” that is, for example, an instrumented slipper. In some other embodiments, IN processing is not used. Rather, the sensors in the mat are actuated by actual contact. This non-IN approach may be preferred in embodiments in which the mat is used simply to control the sensor-enabled striking surface since far fewer “zones” are likely to be required than when the mat is used as an actual instrument.
In summary, the illustrative embodiment of the present invention will incorporate one or more of the following features/characteristics/capabilities:
The advantages realized by the inventive approach include, without limitation:
Although presented in the specific context of a percussion controller, the teachings of the present invention can be adapted to other applications, for example, and without limitation, to other human/computer interfaces such as touch panels, plasma panels, switch panels, computer keyboards, control panels, sound-mixing controls, or stage-lighting controls.
The terms appearing below are defined for use in this disclosure and the appended claims as follows:
In the illustrative embodiment, information about the kinetics of the instrumented striker 402 is obtained via inertial sensing from on-striker devices. That information is wirelessly transmitted, via wireless communications link 401, to data processing system 406. Applying Inertial Navigation techniques, the data processing system uses the inertial measurements to predict the force with which instrumented striker 402 will impact sensor-enabled striking surface 404. In some embodiments, such information is also used to predict the location that instrumented striker 402 will impact sensor-enabled striking surface 404. Instrumented striker 402 is described in more detail in conjunction with
After mapping the predictions to virtual impact zones of sensor-enabled striking surface 404, data processing system 406 generates musical event messages, which are conveyed by signals 413 to music synthesizer 420. The musical event messages control synthesizer 420 in known fashion, causing it generate music signals 415 that are transmitted to amplifier 422 for amplification. The amplified music signals 417 are then transmitted to speakers 424, to actually generate the desired sounds; that is, the musical performance.
Instrumented strikers 402 that are not in use (“cold”) reside in charging cradle 408. The cradle is operable to recharge a rechargeable energy source within each cold instrumented striker 402. In the illustrative embodiments, charging is performed inductively. In some embodiments, charging cradle 408 includes plural indicators 410, as shown in
To facilitate recharge, charging cradle 408 senses, via appropriate circuitry/sensors, the presence of an instrumented striker 402 before charging. The cradle transmits signals to data processing system 406 over communications link 405. The signals convey information pertaining to the presence and state of charge of any instrumented strikers within charging cradle 408. In the illustrative embodiment, communications link 405 is wired; in some other embodiments, this link is wireless. As discussed later in conjunction with
Indicator panel 414 includes indicators 416 (e.g., lights, etc.) that provide an indication of the state of charge of the instrumented strikers that are currently in use (“hot”) by the performer. The state of charge of hot instrumented strikers is tracked by data processing system 406. The state of charge can be estimated by time-in-use or hot instrumented strikers can transmit the state of charge to data processing system 406. The data processing system transmits, via communications link 409, a signal to indicator panel 414 that conveys the status of the hot instrumented strikers. Indicator panel 414 can also provide an indication of the status of other elements of percussion controller 400.
Optional instrumented pad 412 is used, in some embodiments, to supplement the capability of sensor-enabled striking surface 404. Instrumented pad 412 is a simply a smaller version of the sensor-enabled striking surface. Instrumented pad 412 communicates with data processing system 406 over wired communications link 407.
In the illustrative embodiment, percussion controller 400 includes one more foot switch(es) 418b that control some aspects of the operation of sensor-enabled striking surface 404 and/or instrumented pad 412. For example, foot switch 418b can be used to change the layout of a particular instrument being simulated by sensor-enabled striking surface 404 (e.g., change the location of drums, etc. within a “virtual” trap set, etc.) by simply choosing from among several pre-programmed arrangements. For example, a first “click” on the switch provides a first layout and the second “click” on the switch provides a second layout. Or foot switch 418b can be use to change the instrument being simulated by the sensor-enabled striking surface. Again, it is simply a matter of “clicking” between pre-programmed selections. Foot switch 418b communicates with data processing system 406 over wired communications link 411b.
Additional capability can be provided to the system via external pedal(s) 418a. Such pedals, which are conventional for electronic percussion systems, can, for example, actuate a virtual bass drum, etc. Pedal(s) 418a communicates with data processing system 406 over wireless communications link 411a. After reading the present disclosure, those skilled in the art will know how to integrate and use external pedal(s) 418a and foot switch(es) 418b with percussion controller 400.
Instrumented Striker 402.
Referring now to
In the illustrative embodiment, instrumented striker 402 is about the same size as a conventional striker. For example, a 5B standard drum stick is 16 inches in length and 7/16 inches in diameter. The location of the center-of-gravity should be about the same for both instrumented striker 402 and a conventional striker.
In the illustrative embodiment, instrumented striker 402 comprises three sections: tip/taper section 530, shank 532, and butt 534. The diameter of each section near the interface to the adjacent section is appropriate for sliding one into the other and then bonding the adjacent sections together. As depicted in
It will be appreciated that sections 530, 532, and 534 must be hollow or include hollowed-out regions to receive the various components. If any of the sections are hollow, after the components are positioned therein, fill is provided to prevent components from moving and to achieve the proper weight and weight distribution for striker.
For inertial measurements, instrumented striker 402 includes at least one 3-axis accelerometer and at least one angular acceleration sensor (“AAS”). Accelerometer 538 measures acceleration of the striker's reference frame along each of three orthogonal axes: up/down, left/right, forward/back.
Accelerometers do not resolve all the forces present on the three axes (i.e., throwing force, gravity, and angular acceleration [centripetal] forces). Another measurement device, such as an AAS, is required so that angular acceleration forces acting on the striker can be resolved, leaving gravity and the throwing forces combined. Using the fixed rotation, measured at initialization, between the Earth's magnetic field and the gravity field, local gravity can be accurately resolved, such that the throwing forces on instrumented striker 402 can be isolated. In the illustrative embodiment, the AAS is 3-axis digital compass 542.
3-axis digital compass 542 measures the attitude of the instrumented striker frame with respect to the Earth's magnetic field. This information is used, in the illustrative embodiment, to provide angular accelerations for roll, yaw, and pitch about the instrumented striker's frame axes and provides a reference to accurately calculate the direction of Earth's gravity field. As an alternative to digital compass 542, a 3-axis gyroscope can be used. Due to the concerns as to the affect of repeated forceful impacts of instrumented striker 402 on sensor-enabled striking surface 404, digital compasses are currently preferred over gyroscopes.
A second 3-axis accelerometer 548 is used to decrease measurement errors, thereby improving the accuracy of calculations based on the measurements obtained from these devices. Alternatively, a second AAS device (e.g., 3-axis digital compass) could be used.
In some alternative, but less preferred embodiments, the kinetics of the striker is determined by interrogating the striker with electromagnetic energy (“EM”). For example, in some embodiments, a high speed camera is used to track the movements of the strikers during a performance. The images from the camera are then processed and, using IN, the force and/or location of a strike is predicted. In additional embodiments, very high frequency (e.g., Ku band, etc.) radio can be used to interrogate the strikers. The energy is projected at the striker's tip and butt locations and, for example, the Doppler shift is measured at multiple sensors (a minimum of three) and processed in known fashion (e.g., triangulation, etc.) to obtain striker velocities and derive the striker positions, etc., either augmenting or replacing the IN processing. The location of the EM emitters is important so that the percussionist does not obstruct the emissions. In conjunction with the present disclosure, those skilled in the art will be able to make and use such alternative embodiments of the invention.
Information pertaining to the kinetics of instrumented strikers 402 must be transmitted to the data processing system without interfering with percussion performance techniques. To that end, in the illustrative embodiment, instrumented striker 402 includes wireless transmitter/logic circuits 546 and compact antenna 540 for transmitting the measurements obtained by accelerometers 538 and 548 and digital compass 542 to data processing system 406. The logic circuits implement link-layer logic and the conventional wireless physical link.
Power is required to operate transmitter and logic circuits 546. To that end, instrumented striker 402 includes rechargeable energy source 544. In the illustrative embodiment, the rechargeable energy source is a capacitor (e.g., super capacitor, etc).
Rechargeable energy source 544 must be routinely recharged. In the illustrative embodiment, metal coil 536 is disposed within the tip of instrumented striker 402 to facilitate inductive charging of rechargeable energy source 544 in charging cradle 408. Coil 536 is electrically coupled (not depicted) to rechargeable energy source 544.
In some other embodiments, instrumented striker 402 includes an energy-harvester, such as a piezoelectric crystal, etc., which charges the rechargeable energy source. The energy harvester captures energy, such as the energy released as the instrumented striker impacts sensor-enabled striking surface 404 and uses that energy to power the on-striker electronics. In such embodiments, the resiliency/elasticity of the resilient surface of sensor-enabled striking surface 404 is appropriately tailored so that a desired amount of the energy available from the strike is absorbed by deflection of the mat leaving a suitable amount of energy available for harvesting.
Although not depicted, some embodiments of percussion controller 400 include an instrumented glove (e.g., to be worn on the hands for hand percussion, etc.). The instrumented glove includes: (i) two or six accelerometers (one for each finger and one redundant); (ii) one or five 3-axis digital compasses (one for each finger); (iii) a replaceable energy source (e.g., a battery); (iv) a low-power transmitter and matched compact antenna; and (v) circuits to implement a link-layer logic and the conventional wireless physical link.
The Sensor-Enabled Striking Surface 404.
Referring again to
Resilient striking surface 650 provides a “rebound” upon striker impact, thereby mimicking the rebound response of an actual acoustic percussive instrument (e.g., drum heads, etc.).
Mesh of individually-addressable contact (force/pressure) sensors 652 underlies resilient striking surface 650. The contact sensors can be strain gauges, load cells, or the like, such as commercially available from Tekscan, Inc. of Boston, Mass. Sensor mesh spacing is typically less than about 2 centimeters, and more preferably less than about 1 centimeter. The smaller the spacing between sensors, the greater number of zones can be established on the striking surface.
Mesh of individually-addressable lights 654 underlies sensor mesh 652. The lights are positioned in the space between adjacent sensors. The use of the lights is discussed later in conjunction with
Although not directly used for force and/or location determination of a strike, sensor-enabled striking surface 404 provides certain important functionality. In particular, sensor mesh 652 is used for at least the following purposes:
As will be appreciated by those skilled in the art, IN needs to be initialized before it is used and requires ongoing error corrections. In accordance with the illustrative embodiment of the present invention, initialization and navigation error correction are accomplished by striking sensor-enabled striking surface 404. Data processing system 406 keeps track of each striker's state of initialization and the estimated error, and every strike or touch on the sensor-enabled striking surface can be used to fix the navigation solution.
As discussed further below, to relate the (predicted) location of a strike of instrumented striker 402 to a musical event, sensor-enabled striking surface 404 is “virtually” segregated into a plurality of impact zones via data processing system 406. More particularly, the data processing system “virtually” segregates sensor mesh 652 into impact zones. Each such impact zone typically represents a different musical event. Prior to a first performance, a user programs, in conjunction with data processing system 406, a variety of impact zone arrangements. The arrangements are stored in data processing system 406. A desired arrangement is recalled by the performer before a performance.
In the illustrative embodiment, data processing system 406 selectively activates lights within the mesh thereof to display the boundaries of the impact zones for the performer.
Sensor-enabled striking surface 404 will typically have dimensions of 14 inches×32.5 inches, 25 inches×32 inches, or 25 inches×39 inches, although other sizes are acceptable. A master percussionist can reliably strike within a square region that is about 1% on a side. With a sensor-enabled striking surface 404 having dimensions of 25 inches×32 inches, 252 impact zones can be created.
The location and force predictions of the “strike” will be issued a few milliseconds before actual impact on sensor-enabled striking surface 404. As a consequence, prediction accuracy will be very high, but there remains the possibility of extremely infrequent prediction errors. In such cases, at the time of impact, data processing system 406 might determine that there was a prediction error wherein:
(1) Synthesizer 420 begins to generate the wrong note; or
(2) Synthesizer 420 begins to generate the right note but with the incorrect force.
The solution to scenario “2” is to do nothing. “MIDI” velocity is used to convey “force” (at 127 different energy levels) and most force errors will be very small and barely noticeable in the generated sound. Scenario “1” represents the more significant error. The “note” error must be corrected; an uncorrected note will detract from the musical performance. The processor will issue a “note-off” command to the synthesizer for the wrong note. This is followed by a “note-on” command for the correct note. The result of this will be a barely perceptible, several-millisecond “click” sound (due to the incorrect note) followed by the sounding of the correct note.
It is notable that IN error reduction is well established; many conventional techniques are known and applicable to achieve one-in-a-million occurrences of error. Two textbooks that are particularly useful to an understanding of the IN algorithms, causes of IN error and rates of occurrence, and IN error correction techniques are: Britting, Kenneth R. “Inertial Navigation Systems Analysis” (ISBN-13 978-1-60807-078-7) and Bekir, Esmat “Introduction to Modern Navigation Systems” (ISBN-13 978-981-270-765-9).
The dependence of the predictive aspects of the present invention on making very accurate IN predictions is the reason why it is preferable to use two accelerometers, rather than one, in a stick/mallet/beater and up to six accelerometers, rather than five (one for each finger) in a glove. The extra accelerometer provides information critical to reducing errors.
In some alternative embodiments, the striking surface is not sensor-enabled; it is simply a resilient striking pad. In such embodiments, an auxiliary instrumented pad is used to provide the initialization and updating functions. Since the percussionist would have to occasionally strike the auxiliary instrumented pad during a performance, such embodiments are less desirable than the illustrative embodiment in which the striking surface is instrumented. Furthermore, in such embodiments, the percussion controller will not be able to correct prediction errors.
Data Processing System 406.
Transceiver 856A is a wireless transceiver (including antenna, not depicted) and transceiver 856B is a wireline transceiver. These transceivers enable data processing system 406 to (i) transmit information-conveying signals to other elements of percussion controller 400 and (ii) to receive information-conveying signals from such other elements. For example, in the illustrative embodiment depicted in
In the illustrative embodiment, processor 858 is a general-purpose processor that is capable of, among other tasks, running Operating System 862, executing Specialized Applications 864, and populating, updating, using, and managing Reference Data and Intermediate Results 866 in processor-accessible storage 860. In some alternative embodiments of the present invention, processor 858 is a special-purpose processor. It will be clear to those skilled in the art how to make and use processor 858.
Processor-accessible storage 860 is a non-volatile, non-transitory memory technology (e.g., hard drive(s), flash drive(s), etc.) that stores Operating System 862, Specialized Applications 864, and Reference Database and Intermediate Results 866. It will be clear to those skilled in the art how to make and use alternative embodiments that comprise more than one memory, or comprise subdivided segments of memory, or comprise a plurality of memory technologies that collectively store Operating System 862, Specialized Applications 864, and Reference Database and Intermediate Results 866.
It is to be understood that
Furthermore, in some embodiments, the various elements of data processing system 406 are co-located with one another. In some other embodiments, one or more of the elements is not co-located with the remaining elements. For example, in some embodiments, processor-accessible storage 860 is not co-located with processor 858.
The software routines stored in Specialized Applications 864 include the following:
Task 1104 recites determining a location of impact of the striker on the striking surface. As previously discussed, in some embodiments, this task involves obtaining kinetics information about instrumented striker 402 and applying inertial navigation techniques thereto. In some other embodiments, the location of impact is measured on sensor-enabled striking surface 404; that is, only the force of impact is predicted.
Task 1106 recites relating the location of impact with a musical event. As previously disclosed, this task involves determining the impact zone on the sensor-enabled striking surface in which impact is predicted to occur, and determining the musical event that corresponds to an impact at that zone.
Task 1108 recites generating a signal that conveys information pertaining to the musical event. As previously discussed, this can be done in conventional fashion via MIDI protocol.
Task 1110 recites transmitting the signal to a device that generates a signal that can be converted to sound that is related to the musical event.
Additional considerations and details about some of the methods and routines disclosed herein are presented in conjunction with
In the Striking Surface Initialize state, just after power is applied, instrumented strikers 402 in charging cradle 408 will begin receiving power, processor 858 (see, e.g.,
Sensors of the sensor-enabled striking surface 404 take initial readings and set system parameters used during performance. The direction and strength of the gravity field to the Striking Surface frame is read via an included 3-axis accelerometer (not depicted in sensor-enabled striking surface). Alternatively, readings from the 3-axis accelerometer 538 (see, e.g.,
In the Striker Initialization state, as instrumented strikers 402 individually energize, they respond to the discovery requests, and processor 858 registers them in a Striker Protocol Table. Gradually, processor 858 reduces the rate of issuing discovery request messages and increases the rate of polling instrumented strikers 402 for data from their sensors. When instrumented strikers 402 report that they are fully energized, indicator panel 414 requests that the operator performs a Striker Initialization. For this process, each instrumented striker 402 is first placed motionless on sensor-enabled striking surface 404, and then rolled across the striking surface. After each instrumented striker is initialized, the system proceeds to the Performance state.
The Performance mode is a real-time loop of process execution control. Instrumented strikers 402 and sensor-enabled striking surface 404 must be sampled and processed at consistent rates of approximately 1000 Hz; that is, once per millisecond, in order to the achieve psychoacoustic performance criteria required by professional musicians.
The Performance mode processing loop (
With continued reference to
If an immediate surface impact predicted, then the prediction for where the striker would impact on sensor-enabled striking surface 404 is used to create an impact scan list of the sensors surrounding the predicted point of impact. Process control is then passed to the normal surface scan process, after triggering an immediate interrupt to scan the predicted impact area. The interrupt causes a process to scan the predicted impact area using the impact scan list, recording the time of the scan and the impact location if an impact is discovered.
If no impact is detected, a delay is triggered of approximately 100 microseconds to repeat interrupt to scan the predicted impact area. If an impact is detected, processing begins for that instrumented striker's impact to: calculate the error corrections (as necessary), recording the striker's Navigation error offsets to be used in future striker inertial navigation updates, and returning to the normal processing from the interrupt. To avoid an infinite interrupt loop, a time-out control is used to conditionally trigger the delayed interrupt.
Continuing with
The instrumented striker sequence is depicted in
In the SF, the calculated predicted locations of the instrumented striker trace points can be easily checked for a negative elevation (i.e., below the axes in the plane of the sensor-enabled striking surface). Both the elevation of the last striker trace point prior to impact (i.e., position “5” in
With continuing reference to
The Inertial Navigation computations, as taught for example by Britting, address the centripetal and gravity force implications, but instructional value can also be derived from applications that assess these forces. For example, a rapid decrease in centripetal force can indicate the instrumented striker is slipping the grip, which could be detected by instructional applications. As another example, rolling the striker during a throw is inefficient and this could be detected by instructional applications. Also, immediately prior to impact there should be a release of the throwing force on the instrumented striker, which could be detected by instructional applications. Finally, the pivot of throw should remain stable in both the Striker Frame and the Striking Surface Frame which could be detected by instructional applications. Instructional applications would also be concerned with the accuracy of impact placement and timing that could make use of information from the surface impact scans. Parameters inside the Inertial Navigation computations or the surface scan procedures are made available to the instructional applications. The software architecture of the system provides, at minimum, Application Program Interfaces (API) for subscribing to the striker Inertial Navigation parameters or surface scan parameters.
To automate a throwing technique assessment for an instructional application, the primary rotational axis for each accelerometer is computed at every striker sample from a multitude of past samples. Then, calculating the short term weighted average of approximately 3 to 12 samples across both accelerometers, positional tracking algorithms are used to detect the nearness of the pivot to the Wrist Axis. This should be near the stick Butt, and of much shorter radius than an Elbow Axis. Additional calculations then utilize inertial navigation parameter streams to detect the pitching force about the wrist pivot and detect throwing-axis stability. These are recorded and can be displayed externally in real-time to the instructor and student.
The magnetic and gravitational fields should be constant in the combined striker volume. For the AAS approach to sensing motion of instrumented striker 402, this means that magnets and ferrous materials must not influence the uniformity of the magnetic field in the combined striker volume. Structural supports and stands should be made of non-ferrous material such as aluminum or carbon fiber composites. Loudspeakers will need to be kept approximately a few meters away from the combined striker volume. The performance location should not occur near structural steel beams or near metal walls because these might focus the Earth's magnetic field and distort AAS readings. One compensation that is possible for magnetic field distortion is to make measurements of the magnetic field across the combined volume during surface initialization, such as by using a conventional magnetometer device (not depicted). A mapping of the magnetic field in the combined volume is then created that is used during performance to correct the AAS readings based on the IN computed positions.
Dynamically varying magnetic fields nearby or inside the combined striker volume are not compatible with the AAS sensing approach; these fields from devices such as lapel microphones, headsets, earphones, or vocal microphones will distort the AAS measurements in a way that is very difficult to compensate. Thus, when instrumented strikers include an AAS device, a close microphone on the percussionist's voice should be avoided. Rather, a distant, highly directional microphone is preferred.
Referring now to
In
Britting teaches sensor axis alignment and platform alignment error corrections in Chapter 8; alignments are applied to magnetic attitude and the accelerometer measurements. A DCM is computed for aligning the AAS sensor, and another DCM is computed for each of the 3-axis accelerometers during the striker initialization, when the performer first places the instrumented striker on the sensor-enabled striking surface motionless, and then rolls it on the surface. Following Brittings teachings, measurements taking by the sensors in the instrumented striker at known times and positions (sensed by the sensor-enabled surface on the Surface Frame) are then converted into the AAS alignment DCM and the alignment DCM for each accelerometer.
It is to be understood that the disclosure teaches just one example of the illustrative embodiment and that many variations of the invention can easily be devised by those skilled in the art after reading this disclosure and that the scope of the present invention is to be determined by the following claims.
This case is a continuation of co-pending U.S. patent application Ser. No. 13/716,083, filed Dec. 14, 2012, which claims priority of U.S. Provisional Patent Application Ser. No. 61/570,621, filed Dec. 14, 2011, each of which is incorporated by reference herein.
| Number | Name | Date | Kind |
|---|---|---|---|
| 7030305 | Cupid | Apr 2006 | B1 |
| 7060887 | Pangrle | Jun 2006 | B2 |
| 7405353 | Kiyono et al. | Jul 2008 | B2 |
| 7446254 | Lee | Nov 2008 | B2 |
| 8003873 | Rasker | Aug 2011 | B2 |
| 8063296 | Copeland et al. | Nov 2011 | B2 |
| 8525006 | Yamanouchi | Sep 2013 | B2 |
| 8690670 | Kay et al. | Apr 2014 | B2 |
| 8801521 | Haga et al. | Aug 2014 | B2 |
| 20010015123 | Nishitani et al. | Aug 2001 | A1 |
| 20020026866 | Nishitani et al. | Mar 2002 | A1 |
| 20020088335 | Nishitani et al. | Jul 2002 | A1 |
| 20050017454 | Endo et al. | Jan 2005 | A1 |
| 20060174756 | Pangrle | Aug 2006 | A1 |
| 20070107587 | Lee | May 2007 | A1 |
| 20070265104 | Haga et al. | Nov 2007 | A1 |
| 20070270217 | Rabin | Nov 2007 | A1 |
| 20090320672 | Rasker | Dec 2009 | A1 |
| 20100261513 | Izen | Oct 2010 | A1 |
| 20100263518 | Nishitani et al. | Oct 2010 | A1 |
| 20110290097 | Takahashi | Dec 2011 | A1 |
| 20110303076 | Harada | Dec 2011 | A1 |
| 20120024128 | Takahashi | Feb 2012 | A1 |
| 20120090448 | Yamanouchi | Apr 2012 | A1 |
| 20120103168 | Yamanouchi | May 2012 | A1 |
| 20120111179 | Yamanouchi | May 2012 | A1 |
| 20120152087 | Sakazaki | Jun 2012 | A1 |
| 20120216667 | Sakazaki | Aug 2012 | A1 |
| 20130047823 | Tabata | Feb 2013 | A1 |
| 20130152768 | Rapp | Jun 2013 | A1 |
| 20130228062 | Tabata | Sep 2013 | A1 |
| 20130239780 | Tabata | Sep 2013 | A1 |
| 20130239783 | Tabata | Sep 2013 | A1 |
| 20130239784 | Tabata | Sep 2013 | A1 |
| 20130239785 | Tabata | Sep 2013 | A1 |
| 20130255476 | Sakurai | Oct 2013 | A1 |
| 20130262024 | Sakurai et al. | Oct 2013 | A1 |
| 20130340598 | Marquez | Dec 2013 | A1 |
| 20140260916 | Oppel | Sep 2014 | A1 |
| Entry |
|---|
| “Non-Final Office Action”, dated Sep. 11, 2014, issued in related U.S. Appl. No. 13/716,083. |
| “Notice of Allowance”, dated Jan. 21, 2015, issued in related U.S. Appl. No. 13/716,083. |
| Number | Date | Country | |
|---|---|---|---|
| 20150287395 A1 | Oct 2015 | US |
| Number | Date | Country | |
|---|---|---|---|
| 61570621 | Dec 2011 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 13716083 | Dec 2012 | US |
| Child | 14680591 | US |
