This application is a National Stage application under 35 U.S.C. § 371 of International Application No. PCT/JP2019/033870, having an International Filing Date of Aug. 29, 2019, the disclosure of which is considered part of the disclosure of this application, and is incorporated in its entirety into this application.
The present invention relates to a signal processing device, a signal processing method, and a signal processing program.
In recent years, reproduction schemes with multiple arranged speakers have been popularized for public viewing and at home. With popularization of video technologies, such as 3D (three-dimensional) videos and wide videos, efforts have also been made to achieve acoustic reproduction with a higher sense of presence. Specifically, directions of arrival and loudness of sounds are controlled by the speakers for scenes of videos. In particular, a desired sound field is reproduced through a sound field reproduction technology using a speaker array including multiple disposed speakers.
As a general sound field reproduction technology, there is a Pressure-Matching based approach of solving an inverse problem of matching the desired sound field and a sound field to be reproduced. Inverse problems, however, are ill-conditioned problems, and tend to give unstable solutions. In contrast, approaches based on analytic approaches with a spherical speaker array using spherical harmonics or a linear speaker array using an angle spectrum may often give stabler solutions than the inverse problems, and many such approaches have been proposed.
There is an approach of reproducing directional characteristics of the spherical harmonics through superposition of multipole sources (see Patent Literature 1). Patent Literature 1 applies a spherical harmonics expansion coefficient to the directional characteristics of the spherical harmonics reproduced through the superposition of the multipole sources, and thus reproduces the directional characteristics generated by the spherical harmonics, through the superposition of the multipole sources. The spherical harmonics expansion coefficient is obtained through an inverse problem such as a least square method, or spherical harmonic expansion of the sound field.
There is a method of reproducing the desired sound field through a mode-matching approach (Non-Patent Literature 1). Non-Patent Literature 1 collects sounds with a spherical microphone array, and reproduces an expanded sound field with the spherical speaker array.
There is also the multipole source as a method of controlling directivity of the sounds emitted from the speakers (Non-Patent Literature 2). The multipole source is an approach of expressing the directivity of the sounds with a combination of primitive directivities such as a dipole and a quadrupole. Each primitive directivity is achieved with a combination of sound sources having different polarities in proximity to one another.
None of the literatures, however, discloses or suggests a method of reproducing the desired sound field with a multipole speaker array including multiple speakers that provide output outwardly. Patent Literature 1 and Non-Patent Literature 2 only reproduce the directional characteristics, and do not reproduce the sound field. In addition, Non-Patent Literature 1 uses the spherical speaker array, and does not use the multipole speaker array including the multiple speakers that provide the output outwardly.
In addition, the multipole source is not an orthogonal function, and thus cannot expand the sound field as with the spherical harmonics that are orthogonal functions. Accordingly, the reproduction of the sound field with the multipole speaker array requires derivation of a weight factor for superposition of multipoles. While the derivation through the Pressure-Matching based approach may be conceived as a general approach, the approach calculates the inverse problem and thus is likely to undesirably give unstable solutions.
In this way, the conventional art cannot derive the weight factor for the superposition of the multipoles, and thus cannot reproduce the desired sound field with the multipole speaker array including the multiple speakers that provide the output outwardly.
An object of the present invention, which has been accomplished in view of the above situation, is to provide a technology of reproducing the desired sound field with the multipole speaker array including the multiple speakers that provide the output outwardly.
A signal processing device of one aspect of the present invention includes an expansion coefficient calculation unit that, from an outward sound field to be reproduced, calculates a spherical harmonics expansion coefficient for reproducing the sound field; an expansion coefficient conversion unit that converts the calculated spherical harmonics expansion coefficient into a weight factor for superposition of multipole sources; a filter coefficient calculation unit that, from the weight factor, calculates a filter coefficient corresponding to each speaker included in a multipole speaker array, the speaker providing output outwardly; and a convolution operation unit that convolves the filter coefficient corresponding to each speaker into an input acoustic signal to calculate an output acoustic signal for each speaker.
A signal processing method of one aspect of the present invention includes a step of calculating, by a computer from an outward sound field to be reproduced, a spherical harmonics expansion coefficient for reproducing the sound field; a step of converting, by the computer, the calculated spherical harmonics expansion coefficient into a weight factor for superposition of multipole sources; a step of calculating, by the computer from the weight factor, a filter coefficient corresponding to each speaker included in a multipole speaker array, the speaker providing output outwardly; and a step of convolving, by the computer, the filter coefficient corresponding to each speaker into an input acoustic signal to calculate an output acoustic signal for each speaker.
One aspect of the present invention is a signal processing program for causing a computer to function as the above described signal processing device.
According to the present invention, it is possible to provide the technology of reproducing the desired sound field with the multipole speaker array including the multiple speakers that provide the output outwardly.
An embodiment of the present invention will be described below with reference to the drawings. In the drawings, the same reference signs are attached to the same portions and the description thereof will be omitted.
A signal processing device 1 according to the embodiment of the present invention generates, from an input acoustic signal, an output acoustic signal that reproduces a desired sound field with a multipole speaker array.
With reference to
As shown in
The signal processing device 1 reproduces the desired sound field identified in
The embodiment of the present invention derives an analytic expansion coefficient of spherical harmonics for reproducing the outward sound field with spherical harmony (spherical speaker array). The reproduction of the outward sound field with the multipole speaker array is achieved through analytic conversion of the derived expansion coefficient into a weight factor for superposition of multipole sources.
With reference to
The expansion coefficient calculation unit 11 calculates, from the outward sound field to be reproduced, a spherical harmonics expansion coefficient for reproducing this outward sound field.
The sound field to be reproduced is calculated in accordance with Expression (1). Expression (1) expresses the sound field in polar coordinates as shown in
The spherical harmonics in Expression (1) is defined in Expression (2).
The spherical harmonics expansion coefficient in Expression (1) is defined in Expression (3). Expression (3) is referred to as spherical harmonic expansion. The spherical harmonics expansion coefficient is obtained through the spherical harmonic expansion.
[Math. 3]
Anm(ω)=∫02π∫0πS(θ,ϕ,ω)Ynm(θ,ϕ)*sin θdϕdθ. Expression (3)
It should be noted that an example of the spherical harmonics up to degree n=3 is shown in
The expansion coefficient conversion unit 12 converts the spherical harmonics expansion coefficient, which has been calculated by the expansion coefficient calculation unit 11, into the weight factor for the superposition of the multipole sources.
The multipole sources will be described here. The multipole sources are sound sources including an opposite phase distribution of point sources having the same amplitude in positions extremely close to the origin. By way of example, Expression (4) expresses a sound pressure distribution of the multipole sources where the point sources are arranged at very small intervals of 2d on an x-y plane, as follows.
[Math. 5]
xμ,α=xc+(μ−2α)d . . . (0≤α≤μ)
yν,β=yc+(ν−2β)d . . . (0≤β≤ν) Expression (5)
The sound pressure of the point source with respect to the x-axis direction is defined in Expression (6). The sound pressure of the point source with respect to the y-axis direction is also defined similarly.
The sound pressure of the point source constituting the multipole sources of order (μ,ν) is defined below.
[Math. 7]
gμ,αν,β=gμ,α·gν,β Expression (7)
The sound field obtained through the superposition of the multipole sources is expressed in Expression (8).
In accordance with Expression (4) and Expression (8), the sound field with the multipole speaker array is defined by Expression (9).
In addition, the outward sound field is defined with the spherical harmonics in Expression (10).
With respect to Expression (10), Euler's theorem shown in Expression (11) and a binominal theorem may be used to modify Expression (10) as shown in Expression (12).
In addition, let μ in Expression (9) be m−ν. Then comparison of coefficients of cosm−νϕ sinν ϕ between in Expression (9) and in Expression (12) derives Expression (13).
Furthermore, let m in Expression (13) be μ+ν. Then the expression is arranged to give Expression (14). The expansion coefficient conversion unit 12 converts the spherical harmonics expansion coefficient into the weight factor for the superposition of the multipole sources, in accordance with Expression (14).
The filter coefficient calculation unit 13 calculates, from the weight factor, a filter coefficient corresponding to each speaker that is included in the multipole speaker array and provides the output outwardly. The filter coefficient calculation unit 13 obtains the filter coefficient corresponding to each speaker by multiplying the weight factor for the superposition of the multipoles, which has been output by the expansion coefficient conversion unit 12, by the gain of each speaker constituting the multipole sources.
The convolution operation unit 14 convolves the filter coefficient corresponding to each speaker into the input acoustic signal to calculate the output acoustic signal for each speaker. The convolution operation unit 14 calculates the output acoustic signal for each speaker, from the input acoustic signal that is input, and the filter coefficient corresponding to each speaker constituting the multipole speaker array.
The output acoustic signal output by the signal processing device 1 is input to each speaker constituting the multipole speaker array. The output acoustic signal is reproduced at each speaker to thereby reproduce the desired sound field.
(Signal Processing Method)
With reference to
At step S1, the signal processing device 1 first acquires data of the sound field to be reproduced.
At step S2, from the data of the sound field acquired at step S1, the signal processing device 1 next calculates the spherical harmonics expansion coefficient. At step S3, the signal processing device 1 converts the spherical harmonics expansion coefficient calculated at step S2, into the weight factor for the superposition of the multipole sources.
At step S4, the signal processing device 1 calculates the filter coefficient for each speaker from the weight factor for the superposition of the multipole sources calculated at step S3. At step S5, the signal processing device 1 convolves the filter coefficient for each speaker calculated at step S4, into the input acoustic signal to calculate the output acoustic signal for each speaker.
Instead of deriving the weight factor directly from the multipole sources, the signal processing device 1 according to the embodiment of the present invention compares the sound field expressed with the spherical harmony to the sound field expressed with the multipole sources, and thereby analytically converts the sound field with the spherical harmonics into the weight factor for the superposition of the multipoles. The signal processing device 1 can thus generate the acoustic signal that reproduces the desired sound field with the multipole speaker array.
As the above described signal processing device 1 of the present embodiment, for example, a general-purpose computer system is used, which includes a CPU (Central Processing Unit, processor) 901, a memory 902, a storage 903 (HDD: Hard Disk Drive, SSD: Solid State Drive), a communication device 904, an input device 905, and an output device 906. In this computer system, each function of the signal processing device 1 is implemented by the CPU 901 executing a predetermined signal processing program loaded on the memory 902.
It should be noted that the signal processing device 1 may be implemented in one computer, or may be implemented in multiple computers. The signal processing device 1 may also be a virtual machine implemented in the computer.
The signal processing program for implementing each function of the signal processing device 1 may be stored in a computer-readable recording medium, such as an HDD, an SSD, a USB (Universal Serial Bus) memory, a CD (Compact Disc), and a DVD (Digital Versatile Disc), and may also be delivered via a network.
It should be noted that the present invention is not limited to the above described embodiment, and numerous modifications can be made within the scope of the gist of the present invention.
Filing Document | Filing Date | Country | Kind |
---|---|---|---|
PCT/JP2019/033870 | 8/29/2019 | WO |
Publishing Document | Publishing Date | Country | Kind |
---|---|---|---|
WO2021/038782 | 3/4/2021 | WO | A |
Number | Name | Date | Kind |
---|---|---|---|
10433093 | Setiawan | Oct 2019 | B2 |
Number | Date | Country |
---|---|---|
2012169895 | Sep 2012 | JP |
Entry |
---|
Haneda et al., “Directivity synthesis using multipole sources based on spherical harmonic expansion,” Journal of the Acoustical Societyof Japan, 2013, 69(11):577-588, 25 pages (with English Translation). |
Poletti, “Three-Dimensional Surround Sound Systems Based on Spherical Harmonics,” Journal of the Audio Engineering Society, 2005, 53(11):1004-1025. |
Number | Date | Country | |
---|---|---|---|
20220312145 A1 | Sep 2022 | US |