Virtual auditory displays (including computer games, virtual reality systems or computer music workstations) create virtual worlds in which a virtual listener can hear sounds generated from sound sources within these worlds. In addition to reproducing sound as generated by the source, the computer also processes the source signal to simulate the effects of the virtual environment on the sound emitted by the source. In a first-person computer game, the player hears the sound that he/she would hear if he/she were located in the position of the virtual listener in the virtual world. One important environmental factor is reverberation, which refers to the reflections of the generated sound which bounce off objects in the environment. Reverberation can be characterized by measurable criteria, such as the reverberation time, which is a measure of the time it takes for the reflections to become imperceptible. Computer generated sounds without reverberation sound dead or dry.
Artificial reverberation algorithms are well known in the art and are described e.g. in Stautner, J., and Puckette, M. (1982). Designing Multi-Channel Reverberators. Computer Music Journal, Vol. 6, no. 1, Dattorro, J. (1997). Effect Design (Part 1: Reverberator and Other Filters; Part 2: Delay-Line Modulation and Chorus). Journal of the Audio Engineering Society, Vol. 45, no. 9–10, Jot, J.-M. (1997). Efficient Models for Reverberation and Distance Rendering in Computer Music and Virtual Audio Reality. Proceedings of the 1997 International Computer Music Conference. The implementation of these algorithms on digital signal processors is based on a network of digital delay lines which are connected together and to the input and output points of the algorithm by feed-forward or feedback connections. Rooms of different sizes and acoustical properties can be simulated by modifying the topology of the network (the number of delay lines and the connections between them), by varying the duration of the delays, or by adjusting the amplification or attenuation coefficients of multipliers and filters inserted on the feed-forward or feedback connections.
As depicted in
Reverberation processors of the type described above are commonly used for the production of music and soundtracks in recording studios. In these applications, it is not common to produce drastic changes in reverberation characteristics while the sound is playing. Noticeable drop-offs and other artifacts in the output signal of the processor will occur, for instance, when the user loads a different reverberation “program” or adjusts the room size parameter (which may involve changing the network structure or modifying delay lengths). However, such artifacts are not acceptable in interactive audio applications. In immersive 3D games or simulation systems, for instance, different reverberation settings may be associated with different rooms or environments composing a virtual 3D world in which the virtual listener is allowed to travel. Consequently, in these systems, the reverberation processor must be able to change settings while creating a minimum of disruptive or distracting audible artifacts.
Artifacts due to dynamic changes in reverberation settings can be avoided by using two reverberation processors set to simulate different room acoustics and cross-fading from one processor to the other (at their input or at their output). However, it is generally more advantageous to use a single reverberation processor and modify its parameters in order to produce the desired change in room acoustics. The coefficients of multipliers and filters comprising a reverberation processor are easily changed, without noticeable artifacts, by ramping their values to new values over a short time. This avoids the introduction of sudden discontinuities in the audio signal waveform, audible as pops or clicks. For the same reason, it is necessary to avoid sudden changes in the duration of the delay lines.
Methods for implementing continuously variable delays are well known in the art and are described e.g. in Laakso, T. I. et al. (1996). Splitting the Unit Delay—Tools for Fractional Delay Filter Design. IEEE Signal Processing Magazine, Vol. 13, no. 1. However, these methods involve digital audio interpolators, adding significant computational complexity to each delay line. Furthermore, when an interpolator is used, a large variation in a delay length can produce a noticeable change in the pitch of the delayed signal, which may result in an audible artifact. Another technique for implementing variable delays is described in Van Duyne, S. A. (1997). A Lossless, Click-Free, Pitchbend-able Delay Line Loop Interpolation Scheme. Proceedings of the 1997 International Computer Music Conference and illustrated in
In order to provide a wider range of variation in simulated room acoustics, it is common to allow control for the echo density of the reverberation decay, suggesting more or less diffusing room walls. For instance, the reverberation algorithm described in Dattorro, supra allows to control diffusion by adjusting the feedback coefficients of a set of all-pass filters in the reverberation network. Another kind of common musical effect, also described e.g. in Dattorro supra, is the echo effect, which can be obtained simply by a single delay line with feedback. A cyclic echo can sometimes be obtained with existing reverberation algorithms as a side effect—usually unwanted—for particular settings of a reverberation processor's parameters. However, reverberators such as described in the above references do not provide parameters for controlling explicitly and intuitively aspects of an echo effect embedded in the reverberation decay. Such control would be useful to simulate larger rooms or semi-open environments such as a courtyard.
According to one aspect of the invention, audio artifacts are minimized when changing reverberation settings by causing the amplitude of a signal from a delay line having its read pointers changed to ramp down prior to moving the read pointer. The amplitude of the signal is then ramped up after the read pointer has been moved.
According to another aspect of the invention, a set of delay lines, whose output signals are combined to produce the output of a reverberation processor, are updated in sequence so that there is no audible drop-off in the processor's output signal.
According to another aspect of the invention, a reverberation processor provides continuous control over the salience of a periodically repeating echo in the late reverberation decay.
According to another aspect of the invention, a reverberation processor simultaneously provides continuous control over the salience of a periodically repeating echo in the late reverberation decay, and over the duration between successive repetitions of this echo.
According to another aspect of the invention, a reverberation processor simultaneously provides continuous control over the salience of a periodically repeating echo in the late reverberation decay, and over the “diffusion” (or echo density) of the reverberation decay. Furthermore, these two controls combine so that reducing the amount of diffusion has the effect of prolonging the audibility of the repeating echo along the reverberation decay.
Other features and advantages of the invention will be apparent in view of the following detailed description and appended drawings.
a is a block diagram of a reverberation processor, made of an early reflections module and a late reverberation module;
b is a block diagram of an early reflections module;
c is a block diagram of a late reverberation module;
d is a block diagram of a variable delay line using a cross-fading technique;
a depicts a standard reverberation processor. The reverberation processor 10 has one input and four outputs: Left, Right, Right Surround, and Left Surround. It is has two primary components: the Early Reflections module 12, and the Late Reverb module 14. The input signal is low-pass filtered and then passes into a delay line 15 (roomDelay). The delay line 15 has two taps, reverbRead 15a and reflRead 15b, which feed the early reflections and reverberation module 12 and 14. Each of the four outputs of the Early Reflections module 12 is added to one of the four outputs of the Late Reverb module 14, and these signals are the reverberation outputs.
In a preferred embodiment of the present invention, the reverberation processor can be controlled by the following set of parameters (several of which would be affected by a simulated change of the room size):
The problem of switching reverberation settings while creating minimal disruptive or disturbing artifacts is remedied in an embodiment of the present invention by the following mechanism. Any signals coming from delay line reads whose location might be changed are multiplied by variable coefficients. If the reverberation algorithm would not normally call for a multiply to be performed on that signal, it is multiplied by 1. When the delay line read pointer must be moved, the multiply coefficient is ramped towards zero. A short time later, when the coefficient has reached a sufficiently low value, the read pointer is moved, and the coefficient is then ramped back up to its correct value. If multiple delay line read pointers must be moved, they are moved sequentially so that the audible impact of dips in the delayed signals is minimized at any moment.
This method of updating delay lengths in a reverberation processor is particularly efficient because:
With this scheme, quick drops in audio parts of the reverberation are sometimes audible (the entire reverberation signal is never muted at once, but aspects of the sound may be heard to dip). Echoes are sometimes created when delay line read pointers are moved to much longer values. However, the overall effect is usually subtle and much less distracting than clicks and pops.
For example, the delay lines depicted in
Early Reflections Module
After ReflLevel the signal enters the delay line EarlyDiff 122. EarlyDiff 122 has 4 taps 122a, 122b (earlyDiffReadLS), 112c (earlyDiffReadR), and 122d (earlyDiffReadRS) distributed across its length, which feed the four all-pass filters 120. The first tap, 122a, is at a fixed delay length of 0. The range of the other three tap delays changes proportionally with the amplitude of Reverb Delay.
The three signals which are read from the EarlyDiff delay line 122 are multiplied by first level setting coefficients 128b–c, set initially to 1, as indicated in
The four all-pass filters 120a–d are identical except for the lengths of their all-pass delay lines 121a–d. The read pointers 132a (earlyAPreadRS), 132b (earlyAPreadR), 132c (earlyAPreadLS), and 132d (earlyAPreadL) to the delay lines 121 in the four all-passes 120 (and hence the effective length of the all-pass delays) are distributed and scaled proportionally to the amplitude of Reverb Delay. The signals which are read from the all-pass delay lines 121 are multiplied by second level setting coefficients 130a–d set initially to 1. These second level setting coefficients 130 are ramped down and then up when the all-pass delay lengths are changed, to avoid artifacts. The all-pass coefficient used in all four filters, named earlyAPcoff, is set to the value of 0.4 (in this embodiment).
Late Reverb Module
There are eight delay lines 140a–h in the FDN 140, each having a different length. The total length of the delay lines 140a–h is specified by the Modal Density parameter. The outputs of all the lateDelay delay lines 140a–h are multiplied by the fbscale coefficients 141, which are used to control the Decay Time of the reverberation and to normalize the Feedback Matrix 143. Once these eight signals have the delayed input signals added to them they are passed through 1-pole low-pass filters 146 and enter the feedback matrix 148. Each low-pass filter has its own filter coefficient (Ipcoff0 for the 0th, etc.). The low-pass filter coefficients are adjusted according to the settings of the parameters Decay Time and Decay HF Ratio to control the decay time at high frequencies. Whenever one of the lateDelay read pointers 140a–h is updated, the corresponding fbscale and lpcoff coefficients are updated according to the current settings of Decay Time and Decay HF Ratio.
Continuous Control of Diffusion in the Feedback Matrix
After the outputs of each lateDelay delay line have been multiplied by an fbscale coefficient they are added to a delayed input signal, and filtered by a low-pass filter. The resulting signals are mixed together by a unitary mixing feedback matrix 143 before being fed back to the lateDelay delay line inputs. This feedback matrix can be changed from a diagonal matrix, which sends each of the inputs through unaffected, to a completely diffuse matrix, which mixes all of the input signals into each of the output signals. The amount of mixing affects the echo density of the reverb output, and is under control of the Diffusion parameter. A low Diffusion parameter value causes the feedback matrix to become diagonal, and a high value causes the matrix to become diffuse. An intermediate Diffusion value causes the matrix to be more or less diffuse.
The preferred implementation uses a recursive rotation matrix. A recursive rotation matrix can be made by applying a 2×2 rotation matrix to each pair of inputs, and then applying the same rotation to the outputs of the first rotation until all of the inputs have been mixed into each of the outputs, as described in
The feedback matrix is made unitary by applying a normalizing gain to the values of the fbscale mutlipliers. This value is dependant on the diffusion of the feedback matrix and is calculated as follows:
FdnFbScale=cos3 (diffnorm*π/4)
Producing and Controlling a Repeating Echo in the Reverberation Decay
The lengths of the lateDelay delay lines 140 are distributed across a range of values. A repeating echo effect is achieved by reducing the range of lengths across which the delay lines are distributed. As the range becomes diminished, the repeating echo effect becomes more distinct.
The locations of the read pointers for the lateDelay lines in the FDN are determined by the values of Modal Density and Echo Depth according to the following equations (DelayLen[i] is the location in msec of the ith read pointer):
In a preferred embodiment, DMIN=39.1 msec, DMAX 150.1 msec.
As depicted in
In addition to this muting of four of the eight signals entering the FDN by means of the halfecho coefficient, it is necessary to have each of the matrix outputs feed the next delay line (instead of the delay line which matches the matrix input), as shown in
The Sequence for Changing Reverberation Delays
There are five different parts of the reverberation processor which have delay line reads which must be updated without causing artifacts. In the preferred embodiment, many of the delay lines are modified when the Reverb Delay parameter is changed. This parameter controls the delay between the early reflections and late reverberation by changing the location of reverbRead. Changing the Reverb Delay parameter also spreads out the early reflections so that they span the time between Reflections Delay and the onset of the late reverberation. This is done by changing the EarlyDiff reads 122b–d, and by changing the early all-pass delay lengths 121a–d.
The parts that are changed are:
The program which controls the DSP includes a function called GlitchlessSequence which performs the steps necessary to update all the aforementioned delay read pointers. GlitchlessSequence uses a timer callback when it is necessary to wait CBTIME milliseconds for the next step. The delay line reads are changed according to the following pseudo code sequence:
Else if lateDelay read pointers need to be changed go to step #12;
If a change is made to Reflections Delay and the sequence is not in progress GlitchlessSequence will be called starting at step #1. If the sequence is in progress a flag will be set which is used by steps #3, #11, and #20 to make sure the correct delay lines are changed the next time through the sequence. Similarly, changes to Reverb Delay and changes to the lateDelay read pointers (caused by changes in Modal Density, Echo Depth or Echo Time) will start the GlitchlessSequence at steps #2 and #12, respectively, or, if the sequence is already in progress, set their own flags.
The Specifics of Changing One Delay Line Read Pointer
The reverberation processor uses an interpolate instruction to ramp a coefficient for one value, called Ramp1Sub, to another value, called Ramp1Dest, at a rate determined by a variable called RampConst. For example, when the ReflRead tap to the roomDelay delay line 15 is to be moved then the ReflLevel coefficient would be initially equal to Ramp 1 Sub and would be ramped as described below.
Specifically, at each sample period:
Ramp1 Sub=(RampConst*Ramp 1 Sub)+((1−RampConst)*Ramp1Dest)
By replacing the ReflLevel coefficient with the new value Ramp1Sub at each sample period, the delayed signal can be ramped down or up, depending on the value of Ramp1Dest.
The value of RampConst determines how fast Ramp 1Sub approaches the value of Ramp1Dest, and therefore how much time each step of the ramping sequence will take. For a given ramp time in milliseconds, say CBTIME, RampConst is set to:
RampConst=2^(−15/(CBTIME*SampleRate/1000))
If Ramp1 Dest were set to zero, for example, it would take Ramp 1Sub CBTIME milliseconds to reach 15 bits below its original value. In the preferred embodiment CBTIME is 20 milliseconds.
The primary steps to move one delay line read pointer, reflRead for example, are as follows: ramp down ReflLevel, move reflRead, and ramp ReflLevel back up. However, to accomplish these steps there is a more detailed sequence of events that must take place.
1. Set Ramp1Dest to the value of ReflLevel. This must be done so that the ramper does not ramp Ramp1Sub away before we can substitute it in for ReflLevel.
2. Set Ramp1Sub to the value of ReflLevel, so that there will be no level jump when the substitution is made.
3. Replace ReflLevel with Ramp1Sub in the instruction that multiplies the signal read from reflRead.
4. Set Ramp 1 Dest to zero, so that the ramper starts ramping down.
5. Wait CBTIME milliseconds while Ramp1Sub ramps down.
6. Move reflRead to the new location.
7. Set Ramp1Dest to the value of ReflLevel to start ramping up.
8. Wait CBTIME milliseconds while Ramp1Sub ramps up.
9. If ReflLevel has been changed since step 7,
10. If ReflLevel has not been changed, replace Ramp1Sub with ReflLevel in the multiply instruction
If another delay line read pointer must be changed it can begin its own similar sequence immediately. It is not necessary to wait another time step.
The invention has now been described with reference to the preferred embodiments. Alternatives and substitutions will now be apparent to persons of skill in the art. For example, the particular ramping algorithm or delay configurations described can be modified by persons of skill in the art while practicing the principles of the invention. Accordingly, it is not intended to limit the invention except as provided by the appended claims.
Number | Name | Date | Kind |
---|---|---|---|
4338581 | Morgan | Jul 1982 | A |
4731835 | Futamase et al. | Mar 1988 | A |
4731848 | Kendall et al. | Mar 1988 | A |
4817149 | Myers | Mar 1989 | A |
5131051 | Kishnaga et al. | Jul 1992 | A |
5491754 | Jot et al. | Feb 1996 | A |
5553150 | Kozuki | Sep 1996 | A |
5619579 | Ando et al. | Apr 1997 | A |
5657476 | O'Connell et al. | Aug 1997 | A |
5689571 | Kitamura | Nov 1997 | A |
5781461 | Jaffe et al. | Jul 1998 | A |
6032235 | Hoge | Feb 2000 | A |
6188769 | Jot et al. | Feb 2001 | B1 |
6483922 | Limacher | Nov 2002 | B1 |