Patent Grant
8345887
The invention relates to audio signal processing. More specifically, the invention relates to efficient methods to produce a de-correlated multi-channel synthetic reverberation.
Humans detect and process information arriving through a number of different channels. After light signals (vision), sound and hearing may contribute most heavily to one's perception of one's environment. The human auditory system is remarkably discriminating, and though it often fares poorly in comparisons with lower animals, people can detect subtle cues in an audio signal and use them to make inferences about their surroundings, even when those surroundings cannot be seen. The detection and inference occur largely subconsciously, so a carefully-prepared audio program can provide an extremely compelling and visceral experience for a listener.
Virtual reality and game applications can be greatly enhanced by an accurate audio rendering of a simulated environment. Unfortunately, producing a high-resolution, multi-channel audio stream that models the interaction of sounds from various sources with surfaces, spaces and objects in the simulated environment can be as computationally expensive as producing a sequence of high-resolution visual images of the same environment. Techniques for producing convincing audio effects with fewer calculations may be of value in this area.
The individual delay lines produce discrete echoes of the input signal, each echo delayed by the length of the corresponding delay line. The feedback matrix mixes the variously-delayed output signals; matrix coefficients and scaling values are chosen so that the output signal 260 includes exponentially-decaying colored noise to simulate secondary, tertiary and subsequent echoes from objects in the environment that do not produce a specific, identifiable primary echo.
The feedback-delay network shown in
An alternate arrangement that can produce multi-channel synthetic reverberation effects with fewer calculations may be of value.
Embodiments of the invention are illustrated by way of example and not by way of limitation in the figures of the accompanying drawings in which like references indicate similar elements. It should be noted that references to “an” or “one” embodiment in this disclosure are not necessarily to the same embodiment, and such references mean “at least one.”
Embodiments of the invention produce a plurality of synthetic reverberation signals from an input signal by performing a matrix transform on a larger plurality of delayed, scaled versions of the input signal. Outputs of the matrix transformation are mixed back with the inputs of the delay lines to produce a decaying noise tail that simulates diffuse audio reflections.
The adjusted, delayed signals exiting delay lines 130-138 are treated as a vector and processed by an n×n matrix transform 140, producing a corresponding n-element output vector. Elements of this vector are fed back and mixed with the adjusted input signals entering delay lines 130-138. A subset of the output vector elements are also passed out of the FDN for further processing as independent synthetic reverberation channels (150, 160).
The feedback-delay network shown in
The n×n matrix transform indicated as element 140 of
HHT=nIn Eq. 1
where In is the n×n identity matrix. Hadamard matrices must be of order 1, 2 or a multiple of 4. Common embodiments of the invention use a Hadamard matrix of order 16, shown in
Experiments show that not all outputs of a transform with a Hadamard matrix are useable as a synthetic reverberation channel (although all outputs are fed back to the inputs of the delay lines). Only about half
for an order M matrix) of the outputs are useable. However, using M=16 gives 7 useable channels, which fits conveniently with a common “surround sound” format that uses seven primary audio channels and one low-frequency channel (“7.1-channel audio”). Furthermore, since M=16=2k for k=4, the Fast Walsh Hadamard Transform (“FWHT”) may be used in place of a conventional matrix multiplication to further improve computation efficiency.
Useable outputs can be selected by auditioning them directly, but it has been observed that the best-sounding rows are those that have the most “sign interleaving.” For example, row 510 has alternating ones and negative ones, so it is likely to produce a useable reverberation channel, while row 520 has eight successive ones, followed by eight successive negative ones, so it is likely to produce a signal that is not a convincing reverberation simulation. Mathematically, this is believed to be related to the effectiveness of pole/zero cancellation achieved by the FDN: better pole/zero alignment provides better signal characteristics. (Note that perfect pole/zero alignment is difficult to achieve in practice, and is undesirable in any case because it would eliminate the desired synthetic reverberation effect.)
In addition to the effect of sign interleaving discussed above, delay line lengths also contribute to the perceived quality of the reverberation channels. Delay line lengths are relatively prime, for reasons discussed earlier, but lengths that are closer produce signals that are closer in phase for the frequencies that are perceptually important (approximately 20-100 Hz). Delay line length differences on the order of 100 samples (at a sampling rate of 48 KHz) produce delayed signals that are only about to ¼ to 1/20 period out of phase, so matrix rows with more sign interleaving provide better pole/zero cancellation in the important frequency range. To reduce network configuration and operation complexity, it is preferred to arrange delay lines in order of increasing (or decreasing) delay lengths, instead of arranging them randomly. For signals thus arranged, nearby matrix rows with more sign interleaving provide better pole/zero cancellation than lines that are farther apart and/or that have less sign interleaving.
Sound signals processed by audio rendering subsystem 780 may include digitized samples of car and motorcycle engines, sirens, airplanes and construction equipment; gunshots, explosions, waves and splashes; and synthesized, imaginary sounds such as laser shots and animated character pratfalls. Live audio can also be inserted into the world model from a microphone input and processed according to the auditory environment formed by the objects in the world model.
It is appreciated that embodiments of the invention can be used to add synthetic reverberation effects to any audio signal, not only the audio signals associated with a virtual or gaming environment. For example, an embodiment can be incorporated within an “effects pedal,” which is a device used to alter the sound of a musical instrument such as a guitar or keyboard. Embodiments can also add reverberation effects to live or recorded voice, drum, or other audio sources.
An embodiment of the invention may be a machine-readable medium having stored thereon instructions which cause a programmable processor to perform operations as described above. Embodiments can also be implemented as programs or configurations for special purpose digital signal processor (“DSP”) engines. In other embodiments, the operations might be performed by specific hardware components that perform amplification, delay, mixing and matrix transformation functions. Those operations might alternatively be performed by any combination of programmed computer components and custom hardware components.
A machine-readable medium may include any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer), including but not limited to Compact Disc Read-Only Memory (CD-ROM), Read-Only Memory (ROM), Random Access Memory (RAM), and Erasable Programmable Read-Only Memory (EPROM).
The applications of the present invention have been described largely by reference to specific examples and in terms of particular allocations of functionality to certain hardware and/or software components. However, those of skill in the art will recognize that synthetic reverberation effects based on a Fourier or Hadamard matrix transformation can also be produced by software and hardware that distribute the functions of embodiments of this invention differently than herein described. Such variations and implementations are understood to be captured according to the following claims.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4446530 | Tsuboka | May 1984 | A |
| 4991218 | Kramer | Feb 1991 | A |
| 5851187 | Thomas et al. | Dec 1998 | A |
| 20040213416 | Dahl et al. | Oct 2004 | A1 |
