The invention relates generally to sound field modeling and creation of a sound event based on a modeled sound field, and more particularly to a method and apparatus for capturing a sound field with a plurality of sound capture devices located on an enclosing surface, modeling and storing the sound field and subsequently creating a sound event based on the stored information.
Existing sound recording systems typically use two or three microphones to capture sound events produced by a sound source, e.g. a musical instrument. The captured sounds can be stored and subsequently played back. However, various drawbacks exist with these types of systems. These drawbacks include the inability to capture accurately three dimensional information concerning the sound and spatial variations within the sound (including full spectrum “directivity patterns”). This leads to an inability to accurately produce or reproduce sound based on the original sound event. A directivity pattern is the resultant sound field radiated by, a sound source (or distribution of sound sources) as a function of frequency and observation position around the source (or source distribution). The possible variations in pressure amplitude and phase as the observation position is changed are due to the fact that different field values can result from the superposition of the contributions from all elementary sound sources at the field points. This is correspondingly due to the relative propagation distances to the observation location from each elementary source location, the wavelengths or frequencies of oscillation, and the relative amplitudes and phases of these elementary sources. It is the principle of superposition that gives rise to the radiation patterns characteristics of various vibrating bodies or source distributions. Since existing recording systems do not capture this 3-D information, this leads to an inability to accurately model, produce or reproduce 3-D sound radiation based on the original sound event.
On the playback side, prior systems typically use “Implosion Type” (IMT) sound fields. That is, they use two or more directional channels to create a “perimeter effect” sound field. The basic IMT method is “stereo,” where a left and a right channel are used to attempt to create a spatial separation of sounds. More advanced IMT methods include surround sound technologies, some providing as many as five directional channels (left, center, right, rear left, rear right), which creates a more engulfing sound field than stereo. However, both are considered perimeter systems and fail to fully recreate original sounds. Perimeter systems typically depend on the listener being in a stationary position for maximum effect. Implosion techniques are not well suited for reproducing sounds that are essentially a point source, such as stationary sound sources (e.g., musical instruments, human voice, animal voice, etc.) that radiate sound in all or many directions.
Other drawbacks and disadvantages of the prior art also exist.
An object of the present invention is to overcome these and other drawbacks of the prior art.
Another object of the present invention is to provide a system and method for capturing a sound field, which is produced by a sound source over an enclosing surface (e.g., approximately a 360° spherical surface), and modeling the sound field based on predetermined parameters (e.g., the pressure and directivity of the sound field over the enclosing space over time), and storing the modeled sound field to enable the subsequent creation of a sound event that is substantially the same as, or a purposefully modified version of, the modeled sound field.
Another object of the present invention is to model the sound from a sound source by detecting its sound field over an enclosing surface as the sound radiates outwardly from the sound source, and to create a sound event based on the modeled sound field, where the created sound event is produced using an array of loud speakers configured to produce an “explosion” type acoustical radiation. Preferably, loudspeaker clusters are in a 360° (or some portion thereof) cluster of adjacent loudspeaker panels, each panel comprising one or more loudspeakers facing outward from a common point of the cluster. Preferably, the cluster is configured in accordance with the transducer configuration used during the capture process and/or the shape of the sound source.
According to one object of the invention, an explosion type acoustical radiation is used to create a sound event that is more similar to naturally produced sounds as compared with “implosion” type acoustical radiation. Natural sounds tend to originate from a point in space and then radiate up to 360° from that point.
According to one aspect of the invention, acoustical data from a sound source is captured by a 360° (or some portion thereof) array of transducers to capture and model the sound field produced by the sound source. If a given soundfield is comprised of a plurality of sound sources, it is preferable that each individual sound source be captured and modeled separately.
A playback system comprising an array of loudspeakers or loudspeaker systems recreates the original sound field. Preferably, the loudspeakers are configured to project sound outwardly from a spherical (or other shaped) cluster. Preferably, the soundfield from each individual sound source is played back by an independent loudspeaker cluster radiating sound in 360° (or some portion thereof). Each of the plurality of loudspeaker clusters, representing one of the plurality of original sound sources, can be played back simultaneously according to the specifications of the original soundfields produced by the original sound sources. Using this method, a composite soundfield becomes the sum of the individual sound sources within the soundfield.
To create a near perfect representation of the soundfield, each of the plurality of loudspeaker clusters representing each of the plurality of original sound sources should be located in accordance with the relative location of the plurality of original sound sources. Although this is a preferred method for EXT reproduction, other approaches may be used. For example, a composite soundfield with a plurality of sound sources can be captured by a single capture apparatus (360° spherical array of transducers or other geometric configuration encompassing the entire composite soundfield) and played back via a single EXT loudspeaker cluster (360° or any desired variation). However, when a plurality of sound sources in a given soundfield are captured together and played back together (sharing an EXT loudspeaker cluster), the ability to individually control each of the independent sound sources within the soundfield is restricted. Grouping sound sources together also inhibits the ability to precisely “locate” the position of each individual sound source in accordance with the relative position of the original sound sources. However, there are circumstances which are favorable to grouping sound sources together. For instance, during a musical production with many musical instruments involved (i.e., full orchestra). In this case it would be desirable, but not necessary, to group sound sources together based on some common characteristic (e.g., strings, woodwinds, horns, keyboards, percussion, etc.).
These and other objects of the invention are accomplished according to one embodiment of the present invention by defining an enclosing surface (spherical or other geometric configuration) around one or more sound sources, generating a sound field from the sound source, capturing predetermined parameters of the generated sound field by using an array of transducers spaced at predetermined locations over the enclosing surface, modeling the sound field based on the captured parameters and the known location of the transducers and storing the modeled sound field. Subsequently, the stored sound field can be used selectively to create sound events based on the modeled sound field. According to one embodiment, the created sound event can be substantially the same as the modeled sound event. According to another embodiment, one or more parameters of the modeled sound event may be selectively modified. Preferably, the created sound event is generated by using an explosion type loudspeaker configuration. Each of the loudspeakers may be independently driven to reproduce the overall soundfield on the enclosing surface.
Other embodiments, features and objects of the invention will be readily apparent in view of the detailed description of the invention presented below.
According to one embodiment of the present invention, when a sound field is produced by a sound source, the plurality of transducers measures predetermined parameters of the sound field at predetermined locations on the enclosing surface over time. As detailed below, the predetermined parameters are used to model the sound field.
For example, assume a spherical enclosing surface Γa with N transducers located on the enclosing surface Γa. Further consider a radiating sound source surrounded by the enclosing surface, Γa (
While various types of transducers may be used for sound capture, any suitable device that converts acoustical data (e.g., pressure, frequency, etc.) into electrical, or optical data, or other usable data format for storing, retrieving, and transmitting acoustical data may be used.
Processor module 120 may be central processing unit (CPU) or other processor. Processor module 120 may perform various processing functions, including modeling sound received from capture module 110 based on predetermined parameters (e.g. amplitude, frequency, direction, formation, time, etc.), directing information, and other processing functions. Processor module 120 may direct information between various other modules within a system, such as directing information to one or more of storage module 130, modification module 140, or driver module 150.
Storage module 130 may store information, including modeled sound. According to an embodiment of the invention, storage module may store a model, thereby allowing the model to be recalled and sent to modification module 140 for modification, or sent to driver module 150 to have the model reproduced.
Modification module 140 may permit captured sound to be modified. Modification may include modifying volume, amplitude, directionality, and other parameters. While various aspects of the invention enable creation of sound that is substantially identical to an original sound field, purposeful modification may be desired. Actual sound field models can be modified, manipulated, etc. for various reasons including customized designs, acoustical compensation factors, amplitude extension, macro/micro projections, and other reasons. Modification module 140 may be software on a computer, a control board, or other devices for modifying a model.
Driver module 150 may instruct reproduction modules 160 to produce sounds according to a model. Driver module 150 may provide signals to control the output at reproduction modules 160. Signals may control various parameters of reproduction module 160, including amplitude, directivity, and other parameters.
Preferably there are N transducers located over the enclosing surface Γa of the sphere for capturing the original sound field and a corresponding number N of transducers for reconstructing the original sound field. According to an embodiment of the invention, there may be more or less transducers for reconstruction as compared to transducers for capturing. Other configurations may be used in accordance with the teachings of the present invention.
According to an embodiment of the invention, as illustrated in
So, the two cases are as follows:
1. To reproduce the Carnegie Hall event, one needs to know the total reverberatory sound field within a volume, and fit that field with the array subject to spatial Nyquist convergence criteria. There would be no guarantee however that the field would converge anywhere outside this volume.
2. To reproduce the original instrument alone, one needs to know the outgoing (or propagating) field only over a circumscribing sphere, and fit that field with the array subject to convergence criteria on the sphere surface. If this field is fit with sufficient convergence, the field will continue to propagate within the playback environment as if the original instrument were actually playing within this volume.
Thus, in one case, an outgoing sound field on enclosing surface Γa has either been obtained in an anechoic environment or reverberatory effects of a bounding medium have been removed from the acoustic pressure P(α). This may be done by separating the sound field into its outgoing and incoming components. This may be performed by measuring the sound event, for example, within an anechoic environment, or by removing the reverberatory effects of the recording environment in a known manner. For example, the reverberatory effects can be removed in a known manner using techniques from spherical holography. For example, this requires the measurement of the surface pressure and velocity on two concentric spherical surfaces. This will permit a formal decomposition of the fields using spherical harmonics, and a determination of the outgoing and incoming components comprising the reverberatory field. In his event, we can replace the original source with an equivalent distribution of sources within enclosing surface Γa. Other methods may also be used.
By introducing a function Hi,j(ω), and defining it as the transfer function between source point “i” (of the equivalent source distribution) to field point “j” (on the enclosing surface Γa), and denoting the column vector of inputs to the sources xi(ω), i=1, 2 . . . N, as X, the column vector of acoustic pressures P(α)j j=1, 2, . . . N, on enclosing surface Γa as P, and the N×N transfer function matrix as H, then a solution for the independent inputs required for the equivalent source distribution to reproduce the acoustic pressure P(α) on enclosing surface Γa may be expressed as follows
X=H−1P. (Eqn. 1)
Given a knowledge of the acoustic pressure P(α) on the enclosing surface Γa, and a knowledge of the transfer function matrix (H), a solution for the inputs X may be obtained from Eqn. (1), subject to the condition that the matrix H−1 is nonsingular.
The spatial distribution of the equivalent source distribution may be a volumetric array of sound sources, or the array may be placed on the surface of a spherical structure, for example, but is not so limited. Determining factors for the relative distribution of the source distribution in relation to the enclosing surface Γhd a may include that they lie within enclosing surface Γa, that the inversion of the transfer function matrix, H−1, is nonsingular over the entire frequency range of interest, or other factors. The behavior of this inversion is connected with the spatial situation and frequency response of the sources through the appropriate Green's Function in a straightforward manner.
The equivalent source distributions may comprise one or more of:
Concerning the spatial sampling criteria in the measurement of acoustic pressure P(α) on the enclosing surface Γ8, from Nyquist sampling criteria, a minimum requirement may be that a spatial sample be taken at least one half the highest wavelength of interest. For 20 kHz in air, this requires a spatial sample to be taken every 8 mm. For a spherical enclosing Γa surface of radius 2 meters, this results in approximately 683,600 sample locations over the entire surface. More or less may also be used.
Concerning the number of sources in the equivalent source distribution for the reproduction of acoustic pressure P(α), it is seen from Eqn. (1) that as many sources may be required as there are measurement locations on enclosing surface Γa. According to an embodiment of the invention, there may be more or less sources when compared to measurement locations. Other embodiments may also be used.
Concerning the directivity and amplitude variational capabilities of the array, it is an object of this invention to allow for increasing amplitude while maintaining the same spatial directivity characteristics of a lower amplitude response. This may be accomplished in the manner of solution as demonstrated in Eqn. 1, wherein now we multiply the matrix P by the desired scalar amplitude factor, while maintaining the original, relative amplitudes of acoustic pressure P(α) on enclosing surface Γa.
It is another object of this invention to vary the spatial directivity characteristics from the actual directivity pattern. This may be accomplished in a straightforward manner as in beamforming methods.
According to another aspect of the invention, the stored model of the sound field may be selectively recalled to create a sound event that is substantially the same as, or a purposely modified version of, the modeled and stored sound. As shown in
One advantage of the present invention is that once a sound source has been modeled for a plurality of sounds and a sound library has been established, the sound reproduction equipment can be located where the sound source used to be to avoid the need for the sound source, or to duplicate the sound source, synthetically as many times as desired.
The present invention takes into consideration the magnitude and direction of an original sound field over a spherical, or other surface, surrounding the original sound source. A synthetic sound source (for example, an inner spherical speaker cluster) can then reproduce the precise magnitude and direction of the original sound source at each of the individual transducer locations. The integral of all of the transducer locations (or segments) mathematically equates to a continuous function which can then determine the magnitude and direction at any point along the surface, not just the points at which the transducers are located.
According to another embodiment of the invention, the accuracy of a reconstructed sound field can be objectively determined by capturing and modeling the synthetic sound event using the same capture apparatus configuration and process as used to capture the original sound event. The synthetic sound source model can then be juxtaposed with the original sound source model to determine the precise differentials between the two models. The accuracy of the sonic reproduction can be expressed as a function of the differential measurements between the synthetic sound source model and the original sound source model. According to an embodiment of the invention, comparison of an original sound event model and a created sound event model may be performed using processor module 120.
Alternatively, the synthetic sound source can be manipulated in a variety of ways to alter the original sound field. For example, the sound projected from the synthetic sound source can be rotated with respect to the original sound field without physically moving the spherical speaker cluster. Additionally, the volume output of the synthetic source can be increased beyond the natural volume output levels of the original sound source. Additionally, the sound projected from the synthetic sound source can be narrowed or broadened by changing the algorithms of the individually powered loudspeakers within the spherical network of loudspeakers. Various other alterations or modifications of the sound source can be implemented.
By considering the original sound source to be a point source within an enclosing surface Γa, simple processing can be performed to model and reproduce the sound.
According to an embodiment, the sound capture occurs in an anechoic chamber or an open air environment with support structures for mounting the encompassing transducers. However, if other sound capture environments are used, known signal processing techniques can be applied to compensate for room effects. However, with larger numbers of transducers, the “compensating algorithms” can be somewhat more complex.
Once the playback system is designed based on given criteria, it can, from that point forward, be modified for various purposes, including compensation for acoustical deficiencies within the playback venue, personal preferences, macro/micro projections, and other purposes. An example of macro/micro projection is designing a synthetic sound source for various venue sizes. For example, a macro projection may be applicable when designing a synthetic sound source for an outdoor amphitheater. A micro projection may be applicable for an automobile venue. Amplitude extension is another example of macro/micro projection. This may be applicable when designing a synthetic sound source to perform 10 or 20 times the amplitude (loudness) of the original sound source. Additional purposes for modification may be narrowing or broadening the beam of projected sound (i.e., 360° reduced to 180°, etc.), altering the volume, pitch, or tone to interact more efficiently with the other individual sound sources within the same soundfield, or other purposes.
The present invention takes into consideration the “directivity characteristics” of a given sound source to be synthesized. Since different sound sources (e.g., musical instruments) have different directivity patterns the enclosing surface and/or speaker configurations for a given sound source can be tailored to that particular sound source. For example, horns are very directional and therefore require much more directivity resolution (smaller speakers spaced closer together throughout the outer surface of a portion of a sphere, or other geometric configuration), while percussion instruments are much less directional and therefore require less directivity resolution (larger speakers spaced further apart over the surface of a portion of a sphere, or other geometric configuration).
According to another embodiment of the invention, a computer usable medium having computer readable program code embodied therein for an electronic competition may be provided. For example, the computer usable medium may comprise a CD ROM, a floppy disk, a hard disk, or any other computer usable medium. One or more of the modules of system 100 may comprise computer readable program code that is provided on the computer usable medium such that when the computer usable medium is installed on a computer system, those modules cause the computer system to perform the functions described.
According to one embodiment, processor module 120, storage module 130, modification module 140, and driver module 150 may comprise computer readable code that, when installed on a computer, perform the functions described above. Also, only some of the modules may be provided in computer readable code.
According to one specific embodiment of the present invention, a system may comprise components of a software system. The system may operate on a network and may be connected to other systems sharing a common database. According to an embodiment of the invention, multiple analog systems (e.g. cassette tapes) may operate in parallel to each other to accomplish the objections and functions of the invention. Other hardware arrangements may also be provided.
Other embodiments, uses and advantages of the present invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. The specification and examples should be considered exemplary only. The intended scope of the invention is only limited by the claims appended hereto.
This application is a continuation of U.S. patent application Ser. No. 10/705,861, filed Nov. 13, 2003, which is a continuation of U.S. patent application Ser. No. 10/230,989, filed Aug. 30, 2003, which is a continuation of U.S. patent application Ser. No. 09/864,294, filed May 25, 2001 (now U.S. Pat. No. 6,444,892), which is a continuation of U.S. patent application Ser. No. 09/393,324, filed Sep. 10, 1999 (now U.S. Pat. No. 6,239,348). Each of the aforementioned U.S. Patents and Patent Applications are incorporated herein by reference in their entirety.
Number | Name | Date | Kind |
---|---|---|---|
257453 | Ader | May 1882 | A |
572981 | Goulvin | Dec 1896 | A |
1765735 | Phinney | Jun 1930 | A |
2352696 | De Boer et al. | Jul 1944 | A |
2819342 | Becker | Jan 1958 | A |
3158695 | Camras | Nov 1964 | A |
3540545 | Herleman et al. | Nov 1970 | A |
3710034 | Murry | Jan 1973 | A |
3944735 | Willcocks | Mar 1976 | A |
4072821 | Bauer | Feb 1978 | A |
4096353 | Bauer | Jun 1978 | A |
4105865 | Guillory | Aug 1978 | A |
4196313 | Griffiths | Apr 1980 | A |
4377101 | Santucci | Mar 1983 | A |
4393270 | Van Den Berg | Jul 1983 | A |
4408095 | Ariga et al. | Oct 1983 | A |
4422048 | Edwards | Dec 1983 | A |
4433209 | Kurosawa et al. | Feb 1984 | A |
4481660 | De Koning et al. | Nov 1984 | A |
4675906 | Sessler et al. | Jun 1987 | A |
4683591 | Dawson et al. | Jul 1987 | A |
4782471 | Klein | Nov 1988 | A |
5027403 | Short et al. | Jun 1991 | A |
5033092 | Sadaie | Jul 1991 | A |
5046101 | Lovejoy | Sep 1991 | A |
5058170 | Kanamori et al. | Oct 1991 | A |
5142586 | Berkhout | Aug 1992 | A |
5150262 | Hosokawa et al. | Sep 1992 | A |
5212733 | DeVitt et al. | May 1993 | A |
5225618 | Wadhams | Jul 1993 | A |
5260920 | Ide et al. | Nov 1993 | A |
5315060 | Paroutaud | May 1994 | A |
5367506 | Inanaga et al. | Nov 1994 | A |
5400405 | Petroff | Mar 1995 | A |
5400433 | Davis et al. | Mar 1995 | A |
5404406 | Fuchigami et al. | Apr 1995 | A |
5452360 | Yamashita et al. | Sep 1995 | A |
5465302 | Lazzari et al. | Nov 1995 | A |
5497425 | Rapoport | Mar 1996 | A |
5506907 | Ueno et al. | Apr 1996 | A |
5506910 | Miller et al. | Apr 1996 | A |
5521981 | Gehring | May 1996 | A |
5524059 | Zurcher | Jun 1996 | A |
5627897 | Gagliardini et al. | May 1997 | A |
5657393 | Crow | Aug 1997 | A |
5740260 | Odom | Apr 1998 | A |
5768393 | Mukojima et al. | Jun 1998 | A |
5781645 | Beale | Jul 1998 | A |
5790673 | Gossman | Aug 1998 | A |
5796843 | Inanaga et al. | Aug 1998 | A |
5809153 | Aylward et al. | Sep 1998 | A |
5812685 | Fujita et al. | Sep 1998 | A |
5822438 | Sekine et al. | Oct 1998 | A |
5850455 | Arnold et al. | Dec 1998 | A |
5857026 | Scheiber | Jan 1999 | A |
6021205 | Yamada et al. | Feb 2000 | A |
6041127 | Elko | Mar 2000 | A |
6072878 | Moorer | Jun 2000 | A |
6084168 | Sitrick | Jul 2000 | A |
6154549 | Arnold et al. | Nov 2000 | A |
6219645 | Byers | Apr 2001 | B1 |
6239348 | Metcalf | May 2001 | B1 |
6356644 | Pollak | Mar 2002 | B1 |
6444892 | Metcalf | Sep 2002 | B1 |
6574339 | Kim et al. | Jun 2003 | B1 |
6608903 | Miyazaki et al. | Aug 2003 | B1 |
6664460 | Pennock et al. | Dec 2003 | B1 |
6686531 | Pennock et al. | Feb 2004 | B1 |
6738318 | Harris | May 2004 | B1 |
6740805 | Metcalf | May 2004 | B2 |
6826282 | Pachet et al. | Nov 2004 | B1 |
6829018 | Lin et al. | Dec 2004 | B2 |
6925426 | Hartmann | Aug 2005 | B1 |
6959096 | Boone et al. | Oct 2005 | B2 |
6990211 | Parker | Jan 2006 | B2 |
7138576 | Metcalf | Nov 2006 | B2 |
7206648 | Fujishita | Apr 2007 | B2 |
7289633 | Metcalf | Oct 2007 | B2 |
7383297 | Atsmon et al. | Jun 2008 | B1 |
7572971 | Metcalf | Aug 2009 | B2 |
7636448 | Metcalf | Dec 2009 | B2 |
20010055398 | Pachet et al. | Dec 2001 | A1 |
20030123673 | Kojima | Jul 2003 | A1 |
20040111171 | Jang et al. | Jun 2004 | A1 |
20040131192 | Metcalf | Jul 2004 | A1 |
20050141728 | Moorer | Jun 2005 | A1 |
20050195998 | Yamamoto et al. | Sep 2005 | A1 |
20060117261 | Sim et al. | Jun 2006 | A1 |
Number | Date | Country |
---|---|---|
0 593 228 | Apr 1994 | EP |
1 416 769 | May 2004 | EP |
Number | Date | Country | |
---|---|---|---|
20050223877 A1 | Oct 2005 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 10705861 | Nov 2003 | US |
Child | 11131275 | US | |
Parent | 10230989 | Aug 2003 | US |
Child | 10705861 | US | |
Parent | 09864294 | May 2001 | US |
Child | 10230989 | US | |
Parent | 09393324 | Sep 1999 | US |
Child | 09864294 | US |