Audio Channel Conversion

Abstract

A device (1) for converting a first number (M) of input audio channels into a second, larger number (N) of output audio channels comprises: decorrelation units (3) for decomposing the input audio channels into a set of decorrelated auxiliary channels, at least one upmix unit (4) for combining the decorrelated auxiliary channels into the output audio channels, and at least one pre-processing unit (2) for pre-processing the input audio channels and feeding the pre-processed input audio channels to the decorrelation units (3). The pre-processing unit (2) and the upmix unit (4) are preferably controlled by audio parameters.

Description

The present invention will further be explained below with reference to exemplary embodiments illustrated in the accompanying drawings, in which:

FIG. 1 schematically shows a channel conversion device according to the Prior Art.

FIG. 2 schematically shows a first embodiment of a channel conversion device according to the present invention.

FIG. 3 schematically shows a second embodiment of the channel conversion device according to the present invention.

FIG. 4 schematically shows a third embodiment of the channel conversion device according to the present invention.

FIG. 5 schematically shows a fourth embodiment of the channel conversion device according to the present invention.

FIG. 6 schematically shows an audio system according to the present invention.

The Prior Art device 1′ shown in FIG. 1 comprises an array 3 of decorrelation units and an upmix unit 4. The device has M inputs 5 and N outputs 6, which are all coupled to the upmix unit 4. Each input 5 receives an audio channel of a set of audio channels which together constitute a multiple-channel audio signal.

The number of output channels (N outputs 6) is greater than the number of input channels (M inputs 5). Exemplary values are N=6 and M=2, as when a stereo audio signal is converted into a 5.1 audio signal, or N=2 and M=1, as when a stereo signal is encoded as a mono signal plus additional information, although other values of M and N are also possible. The output channels typically have (mutual) correlations defined by parameters fed to the upmix unit 4. To produce output channels having the desired correlations, a set of mutually uncorrelated channels is derived from the input channels. To this end, decorrelation units 3 are coupled to each input 5 so as to produce sets of uncorrelated input channels. The actual number of decorrelation filters, which are well known in the art, may vary and is not limited to the number shown in the drawings.

The decorrelation units 31, . . . , 39 typically include filters having all-pass characteristics. Such filters substantially maintain the spectral envelope of the audio signal. However, the all-pass characteristics have the disadvantage of introducing a time delay. In addition, they often cause a “smearing” of the input signal, that is, the temporal envelope of the decorrelated signal is less well-defined than the temporal envelope of the original signal. Both the time delay and the “smearing” result in a discrepancy between the audio signal and the corresponding parameters: some signal parts (that is, time segments of the signal produced by decorrelation filters) reach the upmix unit later than the corresponding parameters. As a result, the wrong parameters are applied to these signal parts and the audio signal is processed incorrectly, leading to a perceptible signal distortion, for example cross-talk. It will be understood that this is highly undesirable.

It is noted that the parameters could be delayed (e.g. be a delay unit) so as to better match the timing of the parameters and the signals. However, the upmix unit 4 also receives the un-decorrelated input signals, which have not been delayed. In addition, the “smearing” may be frequency-dependent. As a result, it is difficult to match the parameters and the corresponding signal parts.

The present invention solves this problem by processing the audio signal prior to the decorrelation. That is, a substantial part of the signal processing is performed before the audio signal is fed to the decorrelation filters. In this way, the mismatch caused by the decorrelation filters is largely avoided.

The device 1 according to the present invention and illustrated merely by way of non-limiting example in FIG. 2 also comprises an array 3 of decorrelation filters (31, . . . ) and an upmix unit 4. In contrast to the Prior Art device 1′ of FIG. 1, however, the device 1 of the present invention additionally comprises a pre-processing unit 2 for pre-processing the audio signal prior to the decorrelation.

The pre-processing unit 2 receives the M input channels of the audio signal through the M inputs 5. The unit 2 also receives parameters relating to the audio signal, which are indicative of desired signal properties. Using these parameters, the pre-processing unit 2 performs signal processing such as adjusting the power ratios of the audio channels and predicting some audio channels on the basis of other audio channels. As a result, power ratio adjustment and prediction are carried out without being influenced by the decorrelation filters 3, and any time mismatch between the audio signal and the parameters relating to these operations is avoided.

It will be understood that not all signal processing can be performed by the pre-processing unit. Setting the desired correlations of the audio channels typically requires the availability of uncorrelated channels as produced by the decorrelation filters 3. Accordingly, setting the correlations is performed by the upmix unit 4. In addition, additional signal adjustments may be made by the upmix unit 4, such as an additional adjustment of the power levels of the audio channels. In this case, the power adjustment may be carried out in both the pre-processing unit 2 and the upmix unit 4, although it is very well possible to perform this operation in only one of these units.

An additional advantage of the present invention is the possibility to choose which of the units 2 and 4 is best suitable for performing a certain signal processing operation. By providing two units (2 and 4) instead of a single unit (4), a greater design flexibility is achieved, and the unfavorable effects of the decorrelation units can be avoided to the greatest extent possible.

In the preferred embodiments of the present invention, the pre-processing unit 2 and the upmix unit 4 are both time-variant: their signal processing properties are controlled by signal parameters which may vary in time. The decorrelation filters 3, however, are preferably time-invariant: their properties are not time-dependent and are preferably not controlled by signal parameters that vary over time. Embodiments can be envisaged in which either the pre-processing unit 2 or the upmix unit 4 is time-invariant.

In further advantageous embodiments, the processing performed by the pre-processing unit 2 and/or the upmix unit 4 is frequency-dependent: the signal processing properties of these units may be controlled by parameters which vary in dependence of the frequency.

As mentioned above, the number of output channels (N) is greater than the number of input channels (M). For example, there may be two input channels and five or six output channels, or there may be a single input channel and two or more output channels, although other combinations are possible.

It is also possible that the number of output channels 6 is equal to the number of input channels 5 (that is, M=N), in which case the device of the present invention provides a remix of the audio channels. This may be useful to adjust certain signal properties and to enhance the audio signal.

It is noted that the audio signal may be constituted by a series of signal parts contained in consecutive time segments. Such time segments may be time frames or other units defining a time-limited signal part. Due to the decorrelation units the synchronization between the time segments and the corresponding parameters may be lost. This problem is solved by the present invention.

A merely exemplary embodiment of the device of the present invention is shown in more detail in FIG. 3. The device 1 of FIG. 3 receives a single channel audio input signal (M=1). In the exemplary embodiment of FIG. 3 the pre-processing unit 2 comprises two gain units 22 and 23 having respective gains G₂ and G₃. The gain units 22 and 23 set the levels of the audio auxiliary channels before these auxiliary channels are decorrelated by respective decorrelation units 31, 32, 33 of a set (array) 3 of decorrelation units. Each of the decorrelation units 31, 32 and 33 has a respective transfer function H₁, H₂ and H₃ and produces a respective decorrelated auxiliary channel S₁, S₂ and S₃.

A (first) gain unit 21 having a gain G₁ could be added between the input terminal and the first decorrelation unit 31 but has been omitted from the embodiment shown where the first gain G₁ is equal to 1.

The upmix unit 4 comprises, in the example shown, three mixing units 41, 42 and 43 which mix the input channel and its three auxiliary channels to produce four output channels Lf (Left front), Ls (Left surround), Rf (Right front) and Rs (Right surround). The mixing unit 41 receives the (time-dependent) parameters IID_lr (Inter-channel Intensity Difference left-right) and ICC_lr (Inter-channel Cross-Correlation left-right), the mixing unit 42 receives the (time-dependent) parameters IID_l (Inter-channel Intensity Difference left front-left surround) and ICC_l (Inter-channel Cross-Correlation left front-left surround), while the mixing unit 43 receives the (time-dependent) parameters IID_r (Inter-channel Intensity Difference right front-right surround) and ICC_r (Inter-channel Cross-Correlation right front-right surround).

The parameters mentioned above are typically used in a so-called mixing matrix to determine the desired output signals. For example, the output signals Rf (Right front) and Rs (Right surround) may be determined by a mixing matrix M of mixing unit 43:

$\begin{array}{cc}\left[\begin{array}{c}\mathrm{Rf}\\ \mathrm{Rs}\end{array}\right]=\left[\begin{array}{cc}{m}_{11}& m_{12}\\ m_{21}& m_{22}\end{array}\right]\left[\begin{array}{c}R\\ H_3\left(G_3·S\right)\end{array}\right]& \left(1\right)\end{array}$

where the matrix M has coefficients m₁₁ . . . m₂₂, and where H₃(G₃·S)=S₃ is the output signal of decorrelation unit 33. The normalized correlation coefficient ICC of the signals Rf and Rs is given by:

$\begin{array}{cc}\mathrm{ICC}\left(\mathrm{Rf},\mathrm{Rs}\right)=\frac{m_{11}m_{21}{\sigma }_R^2+m_{12}m_{22}{\sigma }_{S3}^2}{\sqrt{\left(m_{11}^2{\sigma }_R^2+m_{12}^2{\sigma }_{S3}^2\right)\left(m_{21}^2{\sigma }_R^2+m_{22}^2{\sigma }_{S3}^2\right)}}& \left(2\right)\end{array}$

where σ²_x is the power of signal x. The intensity ratio IID is given by:

$\begin{array}{cc}\mathrm{IID}\left(\mathrm{Rf},\mathrm{Rs}\right)=\frac{\left(m_{11}^2{\sigma }_R^2+m_{12}^2{\sigma }_{S3}^2\right)}{\left(m_{21}^2{\sigma }_R^2+m_{22}^2{\sigma }_{S3}^2\right)}& \left(3\right)\end{array}$

As the total power should be unaltered, it follows that:

σ_R²=m₁₁² σ_R²+m₁₂² σ_S3²+m₂₁² σ_R²+m₂₂² σ_H3²   (4)

It has been found that the further constraint m₁₂=−m₂₂ is effective. In other words, the power of the intermediate signal (auxiliary channel) S₃ in both signals Rf and Rs is equal but has opposite signs (anti-phase). If m₁₂=−m₂₂ holds, the factors m₁₂ and m₂₂ can be moved upstream of decorrelator unit 33, for example to gain unit 23, to allow processing prior to decorrelation. Equation (1) can then be rewritten as:

$\begin{array}{cc}\left[\begin{array}{c}\mathrm{Rf}\\ \mathrm{Rs}\end{array}\right]=\left[\begin{array}{cc}{m}_{11}& 1\\ m_{21}& -1\end{array}\right]\left[\begin{array}{c}R\\ H_3\left(G_3·m_{12}·S\right)\end{array}\right]& \left(1^{\prime }\right)\end{array}$

Equation (1′) can be generalized using a parameter c:

$\begin{array}{cc}\left[\begin{array}{c}\mathrm{Rf}\\ \mathrm{Rs}\end{array}\right]=\left[\begin{array}{cc}{m}_{11}& c\\ m_{21}& -c\end{array}\right]\left[\begin{array}{c}R\\ H_3\left(G_3·\frac{m_{12}}{c}·S\right)\end{array}\right]& \left(1^{″}\right)\end{array}$

For c=1 all time-variant processing of the decorrelator signal path is performed upstream of the decorrelator, while for c=G₃·m₁₂ all time-variant processing of the decorrelator signal path is performed downstream of the decorrelator. In accordance with the present invention, the parameter c will preferably have a value approximately or substantially equal to 1.

In the exemplary embodiment described above the upmix unit 4 sets both the cross-correlation and the intensity difference of the four output channels. This is, of course, not essential and in some embodiments the inter-channel intensity may be set in the pre-processing unit 2. This may be accomplished by performing all mixing operations in the pre-processing unit 2, for example directly using the input signal S.

It can be seen from FIG. 3 that in accordance with the present invention a pre-processing operation is carried out, in the example shown a gain (that is, power) adjustment.

Another example of a device 1 according to the present invention is illustrated in FIG. 4 where an audio signal comprised of two input audio channels L₀ and R₀ is converted into an audio signal consisting of five output audio channels Lf, Ls, C (Center), Rf and Rs. The pre-processing unit 2 comprises a single mixing unit 25 which receives the (time-dependent) signal parameters c_1 and c_2. The parameters c_1 and c_2 are prediction parameters for predicting the intermediate signals L, C and R output by the mixing unit 25 on the basis of the input signals L₀ and R₀. The decorrelation units 31 and 32 produce uncorrelated counterparts of the intermediate channels L and R which are then fed to the upmix unit 4. The operation of the mixing units 41 and 42 of the upmix unit 4 is similar to the operation of the mixing units 41-43 in the embodiment of FIG. 3.

As can be seen from FIG. 4, part of the processing is carried out by the processing unit 4, prior to the decorrelation. This is particularly advantageous when prediction is used as decorrelators tend to distort the original waveform, while a correct prediction requires the original waveforms to be unaltered. Prediction carried out before decorrelation therefore yields much better results. It will be understood that instead of a single pre-processing unit 2, two or more of such units may be present, for example one pre-processing unit performing prediction operations and another pre-processing unit performing mixing and/or scaling operations.

An exemplary stereo decoder in accordance with the present invention is illustrated in FIG. 5. The stereo decoder of FIG. 5 is essentially a device 1 according to the present invention having a single input (M=1) and two outputs (N=2). The pre-processing unit 2 performs a scaling operation (gain G) and produces two intermediate channels, one of which is decorrelated by the decorrelation unit 3 (transfer function H). An upmix unit 4 performs a rotation operation (Rot) so to rotate the spatial orientation of the signal. It is noted that multiple channel signal rotation is well known in the art. Signal rotation is discussed in more detail in International Patent Application WO 03/090206 (Applicant's Reference PHNL020639EPP), the entire contents of which are herewith incorporated in this document.

An audio system 10 according to the present invention is schematically illustrated in FIG. 6. The audio system 10 is shown to comprise a device 1 for converting a first number of input audio channels into a second number of output audio channels as discussed above.

Accordingly, the present invention may be used in audio amplifiers and/or systems. Such audio systems may include one or more audio sources, an amplifier and loudspeaker units or their equivalents. The audio sources may include a CD player, a DVD player, an MP3 or AAC player, a radio tuner, a hard disk, and/or other sources. The audio system may be incorporated in an entertainment center or in a computer system.

As discussed above, the present invention provides both a device and a method. The method steps are evident from FIG. 2, where the step of pre-processing the input audio channels prior to the step of decomposing the input audio channels into a set of decorrelated auxiliary channels is carried out by the pre-processing unit 2, the step of decomposing the input audio channels into a set of decorrelated auxiliary channels is carried out by the array 3 of decorrelation units (31, 32, . . . ), and the step of converting the decorrelated auxiliary channels, preferably in combination with the input audio channels and/or any intermediate channels, into the output audio channels is carried out by the upmix unit 4.

The present invention is based upon the insight that the time delay and possible “smearing” caused by the decorrelation in an audio decoder may cause temporal alignment discrepancies between the signal parameters and the corresponding signal parts. The present invention benefits from the further insight that this discrepancy can be eliminated, at least for certain signal processing operations, by carrying out these operations prior to the decorrelation.

It is noted that any terms used in this document should not be construed so as to limit the scope of the present invention. In particular, the words “comprise(s)” and “comprising” are not meant to exclude any elements not specifically stated. Single (circuit) elements may be substituted with multiple (circuit) elements or with their equivalents.

It will be understood by those skilled in the art that the present invention is not limited to the embodiments illustrated above and that many modifications and additions may be made without departing from the scope of the invention as defined in the appending claims.

Claims

1-13. (canceled)

14. A device (1) for converting a first number (M) of input audio channels into a second number (N) of output audio channels, where the first number (M) is smaller than the second number (N), the device comprising: at least one decorrelation unit (3) for producing a set of decorrelated auxiliary channels from an input audio channel, the set of decorrelated auxiliary channels including one or more decorrelated auxiliary channels; andat least one upmix unit (4) for combining channels into output audio channels, wherein the at least one upmix unit (4) is operative to combine an input audio channel or a pre-processed input audio channel and a decorrelated auxiliary channel based on a time-varying inter-channel cross correlation parameter, said device further comprising:at least one pre-processing unit (2) for pre-processing the input audio channel prior to feeding the input audio channel to the at least one decorrelation unit (3), wherein the at least one pre-processing unit (2) is operative to perform a time-varying signal processing other than setting correlations.

15. The device as claimed in claim 14, wherein the at least one pre-processing unit (2) and the at least one upmix unit (4) are controlled by audio parameters.

16. The device as claimed in claim 14, wherein the at least one pre-processing unit (2) is arranged for time-variant pre-processing.

17. The device as claimed in claim 14, wherein the at least one decorrelation unit (3) is arranged for time-invariant decorrelation.

18. The device as claimed in claim 14, wherein the upmix unit (4) is arranged for time-variant decorrelation.

19. The device as claimed in claim 14, wherein the pre-processing unit (2) is arranged for setting power ratios of audio channels and/or for prediction.

20. The device as claimed in claim 14, wherein the first number (M) is equal to one.

21. The device as claimed in claim 14, wherein the first number (M) is equal to two.

22. An audio system (10), comprising a device as claimed in claim 14.

23. A method of converting a first number (M) of input audio channels into a second number (N) of output audio channels, where the first number (M) is smaller than the second number (N), the method comprising the steps of: producing a set of decorrelated auxiliary channels from an input audio channel, the set of decorrelated auxiliary channels including one or more decorrelated auxiliary channels; andcombining channels into output audio channels, wherein the step of combining comprising combining an input audio channel or a pre-processed input audio channel and a decorrelated auxiliary channel based on a time-varying inter-channel cross correlation parameter,

24. The method as claimed in claim 23, wherein audio parameters are used in the combining step and the pre-processing step.

25. The method as claimed in claim 23, wherein the step of pre-processing comprises the sub-steps of setting power ratios of audio channels and/or of prediction.

26. A computer program product for carrying out the method as claimed in claim 23.