This disclosure relates to processing an input sound signal.
Modern technology allows musicians to reach large audiences through recordings and live sound amplification systems. Musicians often use microphones for live performance or recording. Microphones can offer good sound quality but may be prohibitively expensive and may be prone to acoustic feedback. Further, microphones are sensitive to variations in distance between the source and the microphone, which may limit the mobility of the performers on stage. Acoustic pickups give acoustic musicians an alternative to microphones. Pickups may consist of one or more transducers, attached directly to the instrument, which convert mechanical vibrations into electrical signals. These signals may be sent to an amplification system through wires or wirelessly. Acoustic pickups may be less prone to feedback, but may not faithfully re-create the sounds of the instrument. One type of acoustic pickups make use of piezoelectric materials to convert mechanical vibrations into electrical current. Often mounted under the instrument bridge of an acoustic instrument, piezoelectric pickups have been cited as sounding “thin”, “tinny”, “sharp”, and “metallic”. Other pickup designs have made use of electromagnetic induction and optical transduction techniques. Acoustic instruments with pickups installed, especially acoustic guitars, are sometimes referred to as “acoustic-electric”.
Sound reinforcement for acoustic instruments may be complicated by audio or acoustic feedback. Feedback occurs when sound from an amplification system is picked up by a microphone or instrument pickup and re-amplified. When feedback is especially severe, feedback loops can occur wherein a “howling” or “screeching” sound occurs as a sound is amplified over and over in a continuous loop. Acoustic instruments are, by design, well-tuned resonators, making instrument bodies and strings susceptible to such audio feedback. Acoustic instruments may be forced into sympathetic vibration by amplification systems, changing the instrument's behavior, and complicating live sound amplification solutions.
Like reference numbers and designations in the various drawings indicate like elements.
The several embodiments described herein are provided solely for the purpose of illustration. Embodiments may include any currently or hereafter-known versions of the elements described. Therefore, persons in the art will recognize from this description that other embodiments may practice various modifications and alterations.
In embodiments, given an input signal x[n] to a linear time-invariant system, and output signal y[n], a transfer function H(s), may be determined and used to estimate y[n] given x[n]. First, a frequency domain representation of x and y may be determined using a Z-transform:
X(s)=Z{x[n]}, Y(z)=Z{y[n]} (1)
The transfer function H(s), is then given by:
A discrete time linear filter may then be built to approximate the frequency-domain transfer function H(z) by fitting parameter vectors a and b:
The corresponding discrete time implementation is then:
Equation (4) may then be used to generate an estimate of y[n], ŷ[n] given x[n].
An example embodiment includes a process for processing one or more pickup signals from an acoustic instrument through the incorporation of an algorithm. The process algorithm can be performed using various mathematical techniques to create a high quality sound from low quality sensor inputs, and can be designed to emulate high-quality microphone signals. The application of the process algorithm is broken into distinct “training” and “implementation” phases. Training phases are described in
The sensors may be connected via individual wires to AV jack 120. The cable 125 may be connected to AV jack 120 and carries each sensor signal along one or more separate wires. One or more microphones 130 may be placed in reasonable proximity of the guitar 105 to record audio signals from guitar 105. Alternatively, the microphones 130 can be positioned by an expert (e.g., recording engineer, or other expert in the field), to optimally capture the sound of the instrument. Optimal positioning may include placing one microphone 6-12″ from the 12th fret of guitar 105, and a second microphone 12″-18″ from the instrument soundboard between audio-video (AV) jack 120 and bridge 110, angled towards the sound hole, or other microphone placements deemed optimal by an expert.
The acoustic environment may be controlled when capturing audio signals. This controlling may be accomplished by working inside a recording studio environment or anechoic chamber. Microphones 130 and cable 125 are connected to processing hardware 135. Example processing hardware 135 can include a digital computer with attached analog to digital converter (ADC) and pre-amplifiers, or dedicated hardware including a pre-amplification stage, ADCs, processing in the form of digital signal processing (DSP) chip and/or field programmable gate array (FPGA), system-on-module (SOM), or microcontroller and memory, or mobile device such as a tablet or smartphone with pre-amplification means, or other hardware capable of pre-amplifying, digitizing, processing multiple signals, and storing results.
The pre-amplifiers 140 may boost individual gain for each sensor and microphone and provide the needed input impedance for each sensor and microphone. Additional, pre-amplifiers 140 may be included to provide any necessary power to microphones or sensors. ADCs 150 convert each microphone and sensor signal into the digital domain. Example ADCs 150 may include Wolfson Microelectronics™ WMB737LGEFL. The ADCs may employ sampling rates that do not create undesirable aliasing effects for audio, for example 44.1 KHz or higher. An example processor 155, may include a central processing unit (CPU) of a digital computer, or a DSP chip and/or microcontroller capable of performing numeric calculations and interfacing with memory. Example memory may include random access memory (RAM), or more permanent types of computer memory. Processor 155 may calculate a variety of algorithm coefficients. A means for moving the contents (not shown) from the memory 160 to other devices may also be included.
Coefficients may be automatically determined using filter design tools available, for example, in MATLAB™, Octave, Python, or other software package. The filtered sensor signals may then be interleaved in step 310 for example with equation (5). Given signal vectors:
a single interleaved vector is then determined by:
S
Interleaved
=[S
n
1
S
n
2
S
n
3
S
n-1
1
S
n-2
2
. . . S
n-k
3]. (6)
Signal vectors shown here may be interpreted as digitized voltage values.
A design matrix may be constructed in step 315 from one or more of the interleaved sensor signals from Step 310. In the Step 315 matrix, each interleaved sensor signal may correspond to a single column of the design matrix shown in equation (7).
All of the filtered microphone signals may be combined in step 320. The signals may be combined by summing all microphone signals together into “target” vector b as shown in step 230. The filtered microphone signals can include signal vectors M1, M2, . . . , Mm,
b=M
1
+M
2
+ . . . +M
m (8)
Alternatively, the signals can be combined using the expert knowledge of a recording engineer as described above, for example through equalization, delay, phase shifting, and carefully selected signal gains. Alternatively, signals may be mixed in specific proportions to achieve a desired tonality.
The signals from design matrix A in step 315 and target vector b in step 320 may be then used in step 325 to solve an overdetermined system by least squares, resulting in {circumflex over (x)}.
{circumflex over (x)}=(ATA)−1ATb (9)
{circumflex over (x)} may be the vector of computed algorithm coefficients used in the configuration described in
Algorithm coefficients are shown as x in step 330. In alternative embodiments, design matrix A and target vector b can be used as part of other techniques, for example weighted least squares, nonlinear system identification, training of artificial neural networks, adaptive filtering approaches, deterministic modeling, Gaussian process modeling, non-linear least squares or treed models. Algorithm coefficients may then be stored in memory 160 to be used later by processing hardware 135, or to be transferred to other processing hardware, such as processing hardware 205.
In general, inputs taken from sensor signals, such as design matrix A and outputs taken from microphone signals, such as target vector b, may be used as the inputs and outputs of a learning model. A learning model may be understood as a computational model design to recognize patterns or learn from data. Many learning models are composed of predefined numerical operations and parameters, which may be numerical values that are determined as the learning model is trained on example data. Learning models may be supervised or unsupervised. Supervised models rely on label input and output data. A supervised learning model may be given an input signal and output signal and trained to reproduce the output from the input.
An artificial neural network is an example of a supervised learning approach. Artificial Neural Networks modify their connector strengths (through changing parameters) between neurons to adapt to training data. Neural networks may consist of many layers, this may be referred to as a deep belief or deep neural network. Neural Networks may further make use of circular recurrent connections (wherein the output of a neuron is connected to the input of another neuron earlier in the chain). Training data provided to neural networks may first be normalized by, for example, dividing by the standard deviation and subtracting the mean. Neural networks may be trained by, for example, a backpropagation algorithm that back propagates errors through the network to determine ideal parameters. Backpropagation may rely on the minimization of a cost function. Cost functions may be minimized by a number of optimization techniques, such as batch or stochastic gradient decent. An example cost function may be the mean square error of the output of the model, compared to the correct output, as shown in equation 10.
Where C in equation 10 is the cost associated with a single training example. The overall cost of a specific model may be determined by summing the cost across a set of examples. Further, a regularization term may be added to the overall cost function, that increases the cost function for large model parameters, reducing the potential complexity of the model. Reducing the complexity of the model may decrease the potential for the model to overfit the training data. Overfitting occurs when a model is fit to the noise in a data set, rather than the underlying structure. An overfit model can perform well on the training data, but may not generalize well, meaning the model may not perform well on data that the model was not trained on.
Alternatively, unsupervised learning models may rely on only input data. For example, sensor data alone may be used to learn about the structure of the data itself. Alternatively, professional or commercial recording of acoustic instruments may be used to learn model parameters that represent the structure of the underlying data. Algorithms such as k-means clustering may be used to group similar windows of input data together. Further, it may be useful to cluster a frequency representation of the input data, such as the Fourier transform, rather than the input data itself. It may also improve algorithm performance to first normalize input data by, for example, subtracting the mean and dividing by the standard deviation. Once similar regions in the input data have been identified, separate sub-models may be trained on each region. These sub-models may offer improved performance over a single model applied to all data. The blending of various sub-models may be accomplished by, for example, determining the Euclidean distance between window of input data and the centroid of each cluster earlier determined by k-means. The Euclidean distance may be then used to choose, prefer, or provide more weight to the model that corresponds to the centroid that is closest, or has the shortest distance to the current input data window. Alternatively, a weighted distance metric may be used rather than Euclidean distance.
In
The interleaved vector may then be processed in step 420 using similar processing hardware and processing methods presented above. The signal may then be post filtered in step 335 with a FIR or IIR digital filter distinct from the pre filters presented above. The post filter may be determined from the transfer function between the processed interleaved signal 415 and an ideal microphone signal, in order to emulate the frequency response recorded by an external microphone.
The post-filtered signal 430 may be gain adjusted in step 435 to ensure the appropriate output amplitude. The gain adjusted signal 435 may be then converted to the analog domain in DAC 235 and output in step 240.
The interleaved signal 415 may be fed into a plurality of methods indicated in step 510, for example, method 300. Alternatively, numerous example approaches can be implemented such as: weighted least squares, nonlinear system identification, training of neural networks, adaptive filtering approaches, deterministic modeling, Gaussian Process (GP) modeling, non-linear least squares or treed models. In step 515 the output of each method in step 510 may be gain adjusted according to the output of step 505. The signals produced in step 515 may be summed in step 520. The signal from step 520 may be filtered with a digital FIR or IIR filter 430. The filtered signal 430 may be gain adjusted in step 435 and output as discussed earlier.
In an alternative example embodiment, training is conducted (
In an alternative embodiment, training can be conducted on a separate instrument from method 400 or 500 in order to re-create sounds of vintage or otherwise desirable acoustic instruments. By training the processing algorithm on a vintage guitar, the results may be interpreted as “training” the desirable acoustic characteristics of the vintage instrument into the algorithm. This algorithm may then be applied in method 400 or 500 to other instruments, allowing lower quality instruments to take on the characteristics of vintage or higher quality instruments when amplified.
In an alternative embodiment, training may be implemented in conjunction with method 400 or a similar method as means to build a processing algorithm uniquely tailored to a specific player. By applying the training methods shown here or similar method to data collected from a single player, the algorithm may be interpreted as “trained” to the playing style of that musician.
The output signal 240 shown in
An example embodiment includes a musical instrument equipped with one or more interior or exterior microphones used to capture and reject external sounds, leaving only the sound created by the musical instrument for live sound applications.
Example processing hardware 625 may include a digital computer with attached analog to digital converter (ADC) and pre-amplifiers, or dedicated hardware including a pre-amplification stage, ADCs, processing in the form of digital signal processing (DSP) chip and/or field programmable gate array (FPGA), system-on-module (SOM), or microcontroller and memory, or mobile device such as a tablet or smartphone with pre-amplification means, or other hardware capable of pre-amplifying, digitizing, processing multiple signals, and storing results. Pre-amplifiers 630 may individually boost gain for each sensor and microphone and provide the needed input impedance for each sensor and microphone. Additionally, pre-amplifiers 630 may provide power to microphones or sensors. ADCs 635 may convert each microphone and sensor signal into the digital domain. Example ADCs 635 can include a Wolfson Microelectronics™ WMB737LGEFL, or other type. The ADCs discussed may employ sampling rates that do not create undesirable aliasing effects for audio, for example 44.1 KHz or higher.
Processor 640 may be the central processing unit (CPU) of a digital computer, or a DSP chip and/or microcontroller capable of performing numeric calculations and interfacing with memory. Example memory 645 may be random access memory (RAM), or more permanent types of computer memory. Digital to analog converter (DAC) 650 can convert the digital output signal into the analog domain. An example DAC 650 may be a Texas Instruments™ PCM2706CPJT. The output 655 from the DAC 650 may be sent to amplification system 620. The output of DAC 650 may be processed further, but is ultimately intended to be connected to a monitoring system, or amplification system such as amplification system 620.
Sensor signals are processed in step 715, for example, by the process described above. Anti-feedback microphone signals may be convolved with a model of acoustic path between anti-feedback microphones and sensors, F in step 720. Model F may be determined through the following steps.
A musical instrument including a pickup system is connected to an amplification system in a performance space, in one embodiment the system is set up in a performance space in preparation for a later performance.
One or more anti-feedback microphones are placed and connected to a digital computer.
A reference sound, such as a test noise (white noise, pink noise, or others), or a musical recording is played through the amplification system.
Both the anti-feedback microphone(s) and acoustic instrument pickup(s) signals are recorded. Instrument is either placed on a stand on stage, or held by a musician at one or more locations in the performance space.
Microphone and pick-up signals are then used to estimate their transfer function, H(s), in the frequency domain. This process is detailed above in the background section.
Equation 4 is then used in real time to estimate the effect of the sound leaving the amplification system on the pickup system from the microphone signal(s).
Signals from step 720 may be negated (i.e., numeric values are multiplied by 1) and added to processed sensor signal from step 715 in summing junction 725 (i.e., effectively removing the noise or feedback sensed by the sensors mounted to the instrument). The summed signal in 725 may be post filtered in step 730 with a FIR or IIR digital filter. The post filter may be determined from the transfer function between the processed signal 725 and an ideal microphone signal in order to emulate the frequency response recorded by an external microphone.
The post-filtered signal from step 730 may be gain adjusted in step 735 to ensure the appropriate output amplitude. The gain adjusted signal from step 735 may then converted to the analog domain in DAC 650 and output in step 655.
The output signal 655 may be useful in live sound amplifications, especially in high-volume (loud) environments, where pickups or microphones may be susceptible to feedback and external noise. The method presented above may be useful in removing external noise and feedback from instrument pickups and internal microphones, allowing these systems to perform well in noisy environments.
In an alternative embodiment, anti-feedback microphone 615 may be used to measure the noise level outside the instrument, and decrease the amplification level of any microphones inside the instrument when outside noise levels are high, effectively decreasing the level of external noise picked up by internal microphones.
In an alternative embodiment, anti-feedback microphone 615 may be used to measure external sounds, and search for correlation between external sounds and pickups signals. If correlation above a predetermined threshold is identified, internal microphone amplification can be decreased, potentially reducing or eliminating acoustic feedback.
This application is a non-provisional application claiming the benefit of U.S. Provisional Application Ser. No. 61/782,273, entitled “Improved Pickup for Acoustic Musical Instruments,” which was filed on Mar. 14, 2013, and is incorporated herein by reference in its entirety.
Number | Date | Country | |
---|---|---|---|
61782273 | Mar 2013 | US |