Patent Grant
7289633
The invention generally relates to methods and apparatus for recording and reproducing a sound event by separately capturing each object within a sound event, transferring the separately captured objects for storage and/or reproduction, and reproducing the original sound event by discretely reproducing each of the separately captured objects and selectively controlling the interaction between the objects based on relationships therebetween.
Methods and systems for recording and reproducing sounds produced by a plurality of sound sources are generally known. In the musical context, for example, systems for recording and reproducing live performances of bands and orchestras are known. In those cases, the sound sources include the musical instruments and performers' voices.
Recording and reproducing sound produced by a sound source typically involves detecting the physical sound waves produced by the sound source, converting the sound waves to audio signals (digital or analog), storing the audio signals on a recording medium and subsequently reading and amplifying the stored audio signals and supplying them as an input to one or more loudspeakers to reconvert the audio signals back to physical sound waves.
Audio signals are typically electrical signals that correspond to actual sound waves, however this correspondence is “representative”, not “congruent”, due to various limitations intrinsic to the process of capturing and converting acoustical data. Other forms of audio signals (e.g., optical), although more reliable in the transmission of acoustical data, encounter similar limitations due to capturing and converting the acoustical data from the original sound field.
The quality of the sound produced by a loudspeaker partly depends on the quality of the audio signal input to the loudspeaker, and partly depends on the ability of the loudspeaker to respond to the signal accurately. Ideally, to enable precise reproduction of sound, the audio signals should correspond exactly to (i.e., be a perfect representation of) the original sound, including its spatial (3D) properties, and the reconversion of the audio signals back to sound should be a perfect conversion of the audio signal to sound waves including its spatial (3D) properties. In practice however, such perfection has not been achieved due to various phenomenon that occur in the various stages of the recording/reproducing process, as well as deficiencies that exist in the design concept of “universal” loudspeakers.
Additional problems are presented when trying to precisely record and reproduce sound produced by a plurality of sound sources. One significant problem encountered when trying to reproduce sounds from a plurality of sound sources is the inability of the system to recreate what is referred to as sound staging. Sound staging is the phenomena that enables a listener to perceive the apparent physical size and location of a musical presentation. The sound stage includes the physical properties of depth and width. These properties contribute to the ability to listen to an orchestra, for example, and be able to discern the relative position of different sound sources (e.g., instruments). However, many recording systems fail to precisely capture the sound staging effect when recording a plurality of sound sources. One reason for this is the methodology used by many systems. For example, such systems typically use one or more microphones to receive sound waves produced by a plurality of sound sources (e.g., drums, guitar, vocals, etc.) and convert the sound waves to electrical audio signals. When one microphone is used, the sound waves from each of the sound sources are typically mixed (i.e., superimposed on one another) to form a composite signal. When a plurality of microphones are used, the plurality of audio signals are typically mixed (i.e., superimposed on one another) to form a composite signal. In either case the composite signal is then stored on a storage medium. The composite signal can be subsequently read from the storage medium and reproduced in an attempt to recreate the original sounds produced by the sound sources. However, the mixing of signals, among other things, limits the ability to recreate the sound staging of the plurality of sound sources. Thus, when signals are mixed, the reproduced sound fails to precisely recreate the field definition and source resolution of the original sounds. This is one reason why an orchestra sounds different when listened to live as compared with a recording. This is one major drawback of prior sound systems. Other problems are caused by mixing as well.
While attempts have been made to address these drawbacks, none has adequately overcome the problem. For example, in some cases, the composite signal includes two separate channels (e.g., left and right) in an attempt to spatially separate the composite signal. In some cases, a third (e.g., center) or more channels (e.g., front and back) are used to achieve greater spatial separation of the original sounds produced by the plurality of sound sources. Two popular methodologies used to achieve a degree of spatial separation, especially in home theater audio Systems, are Dolby Surround and Dolby Pro Logic. Dolby Pro Logic is the more sophisticated of the two and combines four audio channels into two for storage and then separates those two channels into four for playback over five loudspeakers. Specifically, a Dolby Pro Logic system starts with left, center and right channels across the front of the viewing area and a single surround channel at the rear. These four channels are stored as two channels, reconverted to four and played back over left, center and right front loudspeakers and a pair of monaural rear surround loudspeakers that are fed from a single audio channel. While this technique provides some measure of spatial separation, it fails to precisely recreate the sound staging and suffers from other problems, including those identified above.
Other techniques for creating spatial separation have been tried using a plurality of channels. However, regardless of the number of channels, such systems typically involve mixing source signals to form one or more composite signals. Even systems touted as “discrete multi-channel”, typically base the discreteness of each channel on a “directional component” (i.e., Dolby's AC-3, discrete 5.1 multi-channel surround sound is based on five discrete directional channels and one low-frequency effect channel). Surround sound using discrete channels for directional cues help create a more engulfing acoustical effect, but do not address the critical losses of veracity within the representative audio signal nor does it address the reproduction of the intraspace dynamics created by individual sound sources interacting with one another in a defined space.
Other separation techniques are commonly used in an attempt to enhance the recreation of sound. For example, each loudspeaker typically includes a plurality of loudspeaker components, with each component dedicated to a particular frequency band to achieve a frequency distribution of the reproduced sounds. Commonly, such loudspeaker components include woofer or bass (lower frequencies), mid-range (moderate frequencies) and tweeters (higher frequencies). Components directed to other specific frequency bands are also known and may be used. When frequency distributed components are used for each of multiple channels (e.g., left and right), the output signal can exhibit a degree of both spatial distribution and frequency distribution in an attempt to reproduce the sounds produced by the plurality of sound sources. However, maximum recreation of the original sounds is not fully achieved because the source signals continue to be a composite signal as a result of the “mixing” process.
Another problem resulting from the mixing of either sounds produced by sound sources or the corresponding audio signals is that this mixing typically requires that these composite sounds or composite audio signals be played back over the same loudspeaker(s). It is well known that effects such as masking preclude the precise recreation of the original sounds. For example, masking can render one sound inaudible when accompanied by a louder sound. For example, the inability to hear a conversation in the presence of loud amplified music is an example of masking. Masking is particularly problematic when the masking sound has a similar frequency to the masked sound. Other types of masking include loudspeaker masking, which occurs when a loudspeaker cone is driven by a composite signal as opposed to an audio signal corresponding to a single sound source. Thus, in the later case, the loudspeaker cone directs all of its energy to reproducing one isolated sound, as opposed to, in the former, the loudspeaker cone must “time-share” its energy to reproduce a composite of sounds simultaneously.
Another problem with mixing sounds or audio signals and then amplifying the composite signal is intermodulation distortion. Intermodulation distortion refers to the fact that when a signal of two (or more) frequencies is input to an amplifier, the amplifier will output the two frequencies plus the sum and difference of these frequencies. Thus, if an amplifier input is a signal with a 400 Hz component and a 20 KHz component, the output will be 400 Hz and 20 KHz plus 19.6 KHz (20 KHz−400 Hz) and 20.4 KHz (20 KHz+400 Hz).
The mixing of signals can also dictate the use of “universal loudspeakers”, meaning that a given loudspeaker must be capable of reproducing a full or broad spectrum of possible sounds. With the exception of frequency range breakout (e.g., electronic crossovers), loudspeakers are typically capable of reproducing a full range of sound sources. Subwoofers and tweeters are exceptions to this rule but their mandate for separation is based on frequency, not “sound source type”. The drawbacks with “universal” and “frequency dependent” loudspeakers is that they are not capable of being configured to achieve a full integral sound wave (including full directivity patterns) for a given sound source. By being “universal” and “non-configurable”, they can not be optimized for the reproduction of a specific sound source.
More specifically, existing sound recording systems typically use two or three microphones to capture sound events produced by a sound source, e.g., a musical instrument. The captured sounds can be stored and subsequently played back. However, various drawbacks exist with these types of systems. These drawbacks include the inability to capture accurately three dimensional information concerning the sound and spatial variations within the sound (including full spectrum “directivity patterns”). This leads to an inability to accurately produce or reproduce sound based on the original sound event.
A directivity pattern is the resultant sound field radiated by a sound source (or distribution of sound sources) as a function of frequency and observation position around the source (or source distribution). The possible variations in pressure amplitude and phase as the observation position is changed are due to the fact that different field values can result from the superposition of the contributions from all elementary sound sources at the field points. This is correspondingly due to the relative propagation distances to the observation location from each elementary source location, the wavelengths or frequencies of oscillation, and the relative amplitudes and phases of these elementary sources.
It is the principle of superposition that gives rise to the radiation patterns characteristics of various vibrating bodies or source distributions. Since existing recording systems do not capture this 3-D information, this leads to an inability to accurately model, produce or reproduce 3-D sound radiation based on the original sound event.
On the playback side, prior systems typically use “Implosion Type” (IMT) sound fields. That is, they use two or more directional channels to create a “perimeter effect” sound field. The basic IMT method is “stereo,” where a left and a right channel are used to attempt to create a spatial separation of sounds. More advanced IMT methods include surround sound technologies, some providing as many as five directional channels (left, center, right, rear left, rear right), which creates a more engulfing sound field than stereo. However, both are considered perimeter systems and fail to fully recreate original sounds. Perimeter systems typically depend on the listener being in a stationary position for maximum effect. Implosion techniques are not well suited for reproducing sounds that are essentially a point source, such as stationary sound sources or sound sources in the nearfield (e.g., musical instruments, human voice, animal voice, etc.) that should retain their full spectrum directivity patterns and radiate sound in all or many directions.
Despite significant improvements over the last two decades in signal processing and equipment design, the goal of “perfect sound reproduction” remains elusive.
Another problem with the existing systems of sound reproduction are the paradigmatic and other distortions created in an original event right from the beginning of the recording and reproduction process. Such distortions include: (1) lack of true field definition (source signals are mixed together and rely on perceptual effects for definition); (2) lack of source resolution (source rendering is via plane wave transducers, not integral wave transducers); (3) lack of spatial congruency (when source signals are mixed together, sound staging is an approximation at best, once again relying heavily on perceptual effects). These distortions are passed down through the recording and reproduction chain, so that each phase of the chain creates its own colorations on the original distortions created by the paradigm itself.
For example, in a typical stereo reproduction system, when an original event is captured, a multi-dimensional sound wave is represented by a two-dimensional (left/right) signal which is then mixed together with other two-dimensional signals representing other original sound sources within the same sound event, creating a mixture of two-dimensional signals. Once “spatial” and “mixing” distortions have been captured and processed they are passed along to the storage, recall, and reproduction parts of the recording and reproduction chain where additional colorations may be added, compounding the nature of the paradigmatic distortions.
Other contextual issues such as paradigms within paradigms (or sub-paradigms), often are a result of protocol and/or design issues. An example of a sub-paradigm issue is that of “perceptual” effects versus “physical” effects. Perceptual methods of sound reproduction are designed to trick the ear into perceiving certain elements such as spatial qualities and sound stage. Physical objectives for reproduction are focused on physically reproducing source dynamics including primary sources (sound producing entities) and secondary sources (sound effecting entities like room acoustics).
Yet another problem in sound reproduction is amplification. The current amplification of sound concept has remained essentially unchanged for over 40 years, in that, the output signal equals the input signal but at an elevated level. The problem with this approach is that the input signal may be a distorted representation of the original event and most of the time is a compilation of mixed signals representing the original event. When these signals are amplified, the distortions that are present due to the paradigm are amplified and as a result become more noticeable and have a greater impact on the reproduced event.
Another aspect of the problem relates to the issue of “film” paradigm versus the “music” paradigm. The film paradigm utilizes surround sound very well because, with the exception of dialog, most of the soundtrack is a far-field, moving, dynamic type of sound field (e.g., traffic, outdoor environments, etc.) or ambiance-related sound field (e.g., indoor venue, etc.) both of which do well with surround sound formats. Music, on the other hand, is typically a stationary sound event, usually in the near-field, and usually with a more intimate divergent type wave front as opposed to a convergent type wave front created from mid-field and far-field reproductions used in the film industry. Sub-paradigm issues such as these must be harmonized in accordance with the goals of the broader reproduction paradigm if the paradigmatic context is to be optimized and the paradigmatic distortion minimized or eliminated.
Another issue in the present state of sound recording and reproduction is the objectivism vs. subjectivism issue on how close the reproduced event matches the original sound event. Within the current state-of-the-art paradigm, objective measurements can be made (e.g., input signal vs. output signal), but the comprehensive evaluation of a given sound event remains somewhat subjective primarily because of a flawed context—comparison is between an integral form (original event) and a facsimile form (reproduced event). Only when the reproduction system can generate a synthetic sound event in the same integral form as an original event can we expect to render an objective evaluation of the reproduced event. Subjectivity will always play a role in determining which variations, deviations, etc. to an original event are preferable from one person to the next, but the quantifiable evaluation of a reality event and its corresponding synthetic event, should ultimately be an objective analysis.
The problem with trying to use a term like “realism” as a reference standard is not that it is inherently subjective (“reality” is actually inherently objective—it can be objectively measured and modeled, e.g., acoustical holography), but rather that it cannot be adequately synthesized in the same integral form as the original event. The subjective element arises when the audio community attempts to compare various distorted synthetic realities (reproduced events) to their corresponding undistorted original realities (original events), or worse yet, to one another. Even if perfection is interpreted differently by different people, that should not change the fact that the comparison of a reproduced event A to its corresponding original event A, should be an objective analysis. Even if an original source is unnatural or a hybrid of a natural sound, the objective is still to reproduce the source's integral state as determined by an artist and/or producer. A drawback of current systems is the lack of a means for developing reference standards for the articulation of all definable sound sources, and a means for describing derivatives, hybrids, and any other type of deviation from a given reference sound.
Thus, despite significant research and development, prior systems suffer various drawbacks and fail to maximize the ability of the system to precisely reproduce the original sounds.
The invention addresses these and other issues with known sound recording and reproduction systems and presents new methods and systems for more realistically reproducing an original sound event.
One embodiment of the invention relates to a system and method for capturing and reproducing sounds from a plurality of sound sources to more closely recreate actual sounds produced by the sound sources, where sounds from each of a plurality of sound sources (or a predetermined group of sources) are captured by separate sound detectors, and where the separately captured sounds are converted to audio signals, recorded, and played back by separately retrieving the stored audio signals from the recording medium and transmitting the retrieved audio signals separately to a separate loudspeaker system for reproduction of the originally captured sounds.
Another embodiment of the invention relates to a system and method for reproducing sounds produced by a plurality of sound sources, where sounds from each sound source (or a predetermined group of sources) are captured by separate sound detectors, and where the separately captured sounds are converted to audio signals, each of which is transmitted separately to a separate loudspeaker system for reproduction of the originally captured sounds.
According to another embodiment of the invention, each loudspeaker system comprises a plurality of loudspeakers or a plurality of groups of loudspeakers (e.g., loudspeaker clusters) customized for reproduction of specific types of sound sources or group(s) of sound sources. Preferably the customization is based at least in part on characteristics of the sounds to be reproduced by the loudspeaker or based on the dynamic behavior of the sounds or groups of sounds.
According to another embodiment of the invention, each signal path is connected to a separate amplification systems to separately amplify audio signals corresponding to the sounds from each source (or predetermined group of sources). The amplifier systems may be customized for the particular characteristics of the audio signals that it will be amplifying.
According to another embodiment of the invention the amplifier systems are separately controlled by a controller so that the relationship among the components of the power (amplifier) network and those of the loudspeaker network can be selectively controlled. This control can be automatically implemented based on the dynamic characteristics of the audio signals (or the produced sounds) or a user can manually control the reproduction of each sound (or predetermined groups of sounds). For example, the amplifier and loudspeaker systems for each signal path may be automatically controlled by a dynamic controller that controls the relationship among the amplifier systems, the components of the amplifier systems, the loudspeaker systems and the components of the of the loudspeaker systems. For example, the controller can individually turn on/off individual amplifiers of an amplifier system so that increased/decreased power levels can be achieved by using more or less amplifiers for each audio signal instead of stretching the range of a single amplifier. Similarly, the controller can control individual loudspeakers within a loudspeaker system.
If done manually, this may be done through a user interface that enables the user to independently adjust the input power levels of each sound (or predetermined group of sounds) from “off” to relatively high levels of corresponding output power levels without necessarily affecting the power level of any of the other independently controlled audio signals.
If desired, the audio signals output from the sound detectors may be recorded on a recording medium for subsequent readout prior to being transmitted to the loudspeaker systems for reproduction. If recorded, preferably the recording mechanism separately records each of the audio signals on the recording medium without mixing the audio signals. Subsequently, the stored audio signals are separately retrieved and are provided over separate signal paths to individual amplifier systems and then to the separate loudspeaker systems. Preferably, the audio signals are separately controllable, either automatically or manually. The loudspeaker systems preferably are each made up of one or more loudspeakers or loudspeaker clusters and are customized for reproduction of specific types of sounds produced by the respective sound source or group of sound sources associated with the signal path. For example, a loudspeaker system may be customized for the reproduction of violins or stringed instruments. The customization may take into account various characteristics of the sounds to be reproduced, including, frequency, directivity, etc. Additionally, the loudspeakers for each signal path may be configured in a loudspeaker cluster that uses an explosion technique, i.e., sound radiating from a source outwards in various directions (as naturally produced sound does) rather than using an implosion technique, i.e., sound projecting inwardly toward a listener (e.g., from a perimeter of speakers as with surround sound or from a left/right direction as with stereo). In other circumstance, an implosion technique or a combination of explosion/implosion may be preferred.
One embodiment of the invention relates to a system and method for capturing a sound field, which is produced by a sound source over an enclosing surface (e.g., approximately a 360° spherical surface), and modeling the sound field based on predetermined parameters (e.g., the pressure and directivity of the sound field over the enclosing space over time), and storing the modeled sound field to enable the subsequent creation of a sound event that is substantially the same as, or a purposefully modified version of, the modeled sound field.
Another aspect of the invention relates to a system and method for modeling the sound from a sound source by detecting its sound field over an enclosing surface as the sound radiates outwardly from the sound source, and to create a sound event based on the modeled sound field, where the created sound event is produced using an array of loud speakers configured to produce an “explosion” type acoustical radiation. Preferably, loudspeaker clusters are in a 360° (or some portion thereof) cluster of adjacent loudspeaker panels, each panel comprising one or more loudspeakers facing outward from a common point of the cluster. Preferably, the cluster is configured in accordance with the transducer configuration used during the capture process and/or the shape of the sound source.
According to one aspect of the invention, acoustical data from a sound source is captured by a 360° (or some portion thereof) array of transducers to capture and model the sound field produced by the sound source. If a given sound field is comprised of a plurality of sound sources, it is preferable that each individual sound source be captured and modeled separately.
Preferably, a playback system comprising an array of loudspeakers or loudspeaker systems recreates the original sound field. According to one aspect of the invention, an explosion type acoustical radiation is used to create a sound event that is more similar to naturally produced sounds as compared with “implosion” type acoustical radiation. Preferably, the loudspeakers are configured to project sound outwardly from a spherical (or other shaped) cluster. Preferably, the sound field from each individual sound source is played back by an independent loudspeaker cluster radiating sound in 360° (or some portion thereof). Each of the plurality of loudspeaker clusters, representing one of the plurality of original sound sources, can be played back simultaneously according to the specifications of the original sound fields produced by the original sound sources. Using this method, a composite sound field becomes the sum of the individual sound sources within the sound field.
To create a near perfect representation of the sound field, each of the plurality of loudspeaker clusters representing each of the plurality of original sound sources should be located in accordance with the relative location of the plurality of original sound sources. Although this is a preferred method for EXT reproduction, other approaches may be used: For example, a composite sound field with a plurality of sound sources can be captured by a single capture apparatus (360° spherical array of transducers or other geometric configuration encompassing the entire composite sound field) and played back via a single EXT loudspeaker cluster (360° or any desired variation).
These and other aspects of the invention are accomplished according to one embodiment of the invention by defining an enclosing surface (spherical or other geometric configuration) around one or more sound sources, generating a sound field from the sound source, capturing predetermined parameters of the generated sound field by using an array of transducers spaced at predetermined locations over the enclosing surface, modeling the sound field based on the captured parameters and the known location of the transducers and storing the modeled sound field. Subsequently, the stored sound field can be used selectively to create sound events based on the modeled sound field. According to one embodiment, the created sound event can be substantially the same as the modeled sound event. According to another embodiment, one or more parameters of the modeled sound event may be selectively modified. Preferably, the created sound event is generated by using an explosion type loudspeaker configuration. Each of the loudspeakers may be independently driven to reproduce the overall sound field on the enclosing surface.
Another aspect of the invention relates to a system and method for reproducing a sound event includes means for retrieving a plurality of separately stored audio signals for a sound event, where at least one of the audio signals comprises an ambiance sound field of an environment of the sound event and where at least one of the audio signals comprises a sound field for a sound source, amplification means for separately amplifying each audio signal and a loudspeaker network comprising a plurality of loudspeaker means. At least one loudspeaker means comprises a convergent speaker system for reproducing the ambiance sound field and where at least one loudspeaker means comprises a divergent speaker system for reproducing the sound field for the sound source.
In another aspect of the invention, a system and method for creating a holographic or three-dimensional sound event includes storing first data for an integral reality model of a sound source, the data including a plurality of predetermined parameters for creating a holographic or three-dimensional sound for the sound source, inputting second data for a sound event, where the sound event comprises a sound source and where the second data comprises information on a portion of a sound field for the sound source and rendering holographic or three-dimensional sound data for the sound event by extrapolating the second data using the plurality of parameters from the first data, where the holographic or three-dimensional sound data includes information for outputting audio signals to a plurality of loudspeakers positioned in a predetermined three-dimensional arrangement.
Another aspect of the invention relates to a method for objectively comparing a reproduced sound event to an original sound event includes retrieving data representing a modeled sound field of a first radiating sound field of an original sound event, the modeled sound field including a first set of predetermined parameters, converting the data to a plurality of separate audio signals representing the first radiating sound field, separately amplifying each audio signal, communicating each amplified audio signal to a respective loudspeaker of a cluster of loudspeakers, where each respective loudspeaker is arranged along a predetermined geometric position to create a reproduced sound event comprising a second radiating sound field emanating from the cluster of loudspeakers and recording the second radiating sound field via a plurality of transducers arranged on a predetermined geometric surface at least partially surrounding the cluster of loudspeakers. The second radiating sound field includes a second set of predetermined parameters. The method also further includes comparing the second set of predetermined parameters to the first set of predetermined parameters, where a difference between the second set of predetermined parameters and the first set of predetermined parameters establishes an objective determination on a similarity between the reproduced sound event to the original sound event.
Other aspects of the invention include computer instruction and computer readable medium including computer instructions for performing methods according to the above aspects of the invention.
Other embodiments, features and objects of the invention will be readily apparent in view of the detailed description of the invention presented below and the drawings attached hereto. It is also to be understood that both the foregoing general description and the following detailed description are exemplary and not restrictive of the scope of the invention.
If desired, the N audio signals output from the N sound detectors (SD1-SDN) may be input to an acoustical manifold 10 and/or an annunciator 20 prior to being input to encoder 30. The acoustical manifold 10 is an input/output device that receives audio signal inputs, indexes them (e.g., by assigning an identifier to each data stream) and determines which of the inputs to the manifold have a data stream (e.g., audio signals) present. The manifold then serves as a switching mechanism for distributing the data streams to a particular signal path as desired (detailed below). The annunciator 20 can be used to enable flexibility in handling different numbers of audio signals and signal paths. Annunciators are active interface modules for transferring or combining the discrete data streams (e.g., audio signals) conveyed over the plurality of signal paths at various points within the system from sound capture to sound reproduction. For example, when the number of signal paths output from the sound detectors is equal to the number of amplifier systems and/or loudspeaker systems, the function of the annunciator can be passive (no combining of signals is necessarily performed). When the number of outputs from the sound detectors is greater than the number of amplifier systems and/or loudspeaker systems, the annunciator can combine selected signal paths based on predetermined criteria, either automatically or under manual control by a user. For example, if there are N sound sources and N sound detectors, but only N-i inputs to the encoder are desired, a user may elect to combine two signal paths in a manner described below. The operation and advantages of these components are further detailed below.
For simplicity, it will be assumed that N audio signals are input to annunciator 60 and that N audio signals are output therefrom. It is to be understood, however, that different numbers of signals can be input to and output from annunciator 20. If, for example, only five audio signals are output from annunciator 60, only five amplifier systems and five loudspeaker systems are necessary. Additionally, the number of audio signals output from annunciator 60 may be dictated by the number of amplifier or loudspeaker systems available. For example, if a system only has four amplifier systems and four loudspeaker systems, it may be desirable for the annunciator to output only four audio signals. For example, the user may elect to build a system modularly (i.e., adding amplifier systems and loudspeaker systems one or more at a time to build up to N such systems). In this event, the annunciator facilitates this modularity. The user interface 55 enables the user to select which audio signals should be combined, if they are to be combined, and to control other aspects of the systems as detailed below.
Referring to
Preferably, each loudspeaker or loudspeaker cluster is customized for the specific types of sounds produced by the sound source or groups of sound sources associated with its signal path. Preferably, each of the amplifier systems and loudspeaker systems are separately controllable so that the audio signals sent over each signal path can be controlled individually by the user or automatically by the system as detailed below. More preferably, each of the individual amplifiers (A-N) and each of the individual loudspeakers (A-N) are each separately controllable. For example, it is preferable that each of amplifiers A-N for amplifier system AS1 is separately controllable to be on or off, and if on to have variable levels of amplification from low to high. In this way, power levels of audio signals on that signal path may be stepped up or down by turning on specific amplifiers within an amplifier system and varying the amplification level of one or more of the amplifiers that are on. Preferably, each of the amplifiers of an amplifier system is customized to amplify the audio signals to be transmitted through that amplifier system. For example, if the amplifier system is connected in a signal path that is to receive audio signals corresponding to sounds that consist of primarily low frequencies (e.g., bass sounds from a drum), each of the amplifiers of that amplifier system may be designed to optimally amplify low frequency audio signals. This is an advantage over using amplifiers that are generic to a broad range of frequencies. Moreover, by providing multiple amplifiers within one amplifier system for a specific type of audio signal (e.g., sounds that consist of primarily low frequencies), the power level output from the amplifier system can be stepped up or down by turning on or off individual amplifiers. This is an advantage over using a single amplifier that must be varied from very low power levels to very high power levels. Similar advantages are achieved by using multiple loudspeakers within each loudspeaker system. For example, two or more loudspeakers operating at or near a middle portion of a power range will reproduce sounds with less distortion than a single loudspeaker at an upper portion of its power range. Additionally, loudspeaker arrays may be used to effect directivity control over 360 degrees or variations thereof.
As also shown in
Preferably, the user interface 55 includes a master volume control (MC) and N separate controls (C1-CN) for the N signal paths. A dynamics override control (DO) may also be provided to enable a user to manually override the automatic dynamic control of dynamic controller 90.
Also shown in
According to one aspect of the invention, dynamics control module 90 includes a controller 91, one or more annunciator interfaces 92, one or more amplifier system interfaces 93, one or more loudspeaker interfaces 94 and a feedback control interface 95. The annunciator interface 92 is connected to one or more annunciators (20, 60). The amplifier interface 93 is operatively connected to the amplifier network 70. The loudspeaker interface 94 is connected to the loudspeaker network 80. Dynamics control module 90 controls the relationship among the amplifier systems and loudspeaker systems and the individual components therein. Dynamics control module 90 may receive feedback via the feedback control interface 95 from the amplification network 70 and/or the loudspeaker network 80. Dynamics control module 90 processes signals from amplification network 70 and/or sounds from loudspeaker network 80 to control amplification network 70 and loudspeaker network 80 and the components thereof. Dynamics control module 90 preferably controls the power relationship among the amplifier systems of the amplification network 70. For example, as power or volume of an amplifier system is increased, the dynamic response of a particular audio signal amplified by that amplifier system may vary according to characteristics of that audio signal. Moreover, as the overall power of the amplifier network is increased or decreased, the dynamic relationship among the audio signals in the separate signal paths may change. Dynamics control module 90 can be used to discretely adjust the power levels of each amplifier system based on predetermined criteria. An example of the criteria on which dynamics control module 90 may base its adjustment is the individual sound signal power curves (e.g., optimum amplification of audio signals when ramping power up or down according to the power curves of the original sound event). Module 90 can discretely activate, deactivate, or change the power level of, any of the amplification systems 70 AS1-ASN and preferably, the individual components (A-N) of any given amplifier system AS1-AS1.
Module 90 can also control the loudspeaker network 80 based on predetermined criteria. Preferably, module 90 can discretely activate, deactivate, or adjust the performance level of each individual loudspeaker system and/or the individual loudspeakers or loudspeaker clusters (A-N) within a loudspeaker system (LS1-LSN Thus, the system components are capable of being individually manipulated to optimize or customize the amplification and reproduction of the audio signals in response to dynamic or changing external criteria (e.g., power), sound source characteristics (e.g., frequency bandwidth for a given source), and internal characteristics (e.g., the relationship between the audio signals of the different signal paths).
The user interface 55 and/or dynamic controller 90 enables any signal path or component to be turned on/off or to have its power level controlled either automatically or manually. The dynamic controller 90 also enables individual amplifiers or loudspeakers within an amplifier system or loudspeaker system to be selectively turned on depending, for example, on the dynamics of the signals. For example, it is advantageous to be able to turn on two amplifiers within one system to increase the power level of a signal rather than maxing out the amplification of a single amplifier which can cause undesired distortion.
As will be apparent from the foregoing description, whether the N separate audio signals are recorded first and then reproduced or reproduced without first being recorded, the invention enables various types of control to be effected to enable the reproduced sounds to have desired characteristics. According to one embodiment, the N separate audio signals output from the sound detectors (SD1-SDN) are maintained as N separate audio signals throughout the system and are provided as N separate inputs to the N loudspeaker systems. Typically, it is desired to do this to accurately reproduce the originally captured sounds and avoid problems associated with mixing of audio signals and/or sounds. However, as detailed herein various types of selective control over the audio signals can be effected by using acoustical manifold 10, one or more annunciators (20, 60), a user interface 55 and a dynamic controller 90 to enable various types of desired mixing of audio signals to permit modular expansion of a system. For example, one or more acoustical manifolds 10 can be used at various points in the system to enable audio signals on one signal path to be switched to another signal path. For example, if the sounds produced by SS1 are captured by SD1 and converted to audio signals on signal path SP1, it may be desired to ultimately provide these audio signals to loudspeaker system LS4 (e.g., since the loudspeakers may be customized for a particular type of sound source). If so, then the audio signals input to the acoustical manifold 10 on SP1 are routed to output 4 of the acoustical manifold 10. Other signals may be similarly switched to other signal paths at various points within the system. Thus, if the characteristics of the sounds produced by a sound source (SS) as captured by a sound detector (SD) change, the acoustical manifold 10 enables those signals to be routed to an amplifier system and/or loudspeaker system that is customized for those characteristics, without reconfiguring the entire system.
One or more annunciators (e.g., 20, 60) may be used to selectively combine two or more audio signals from separate signal paths or it can permit the N separate audio signals to pass through all or portions of the system without any mixing of the audio signals. One advantage of this is where there are more sound detectors then there are amplifier systems or loudspeaker systems. Another is when there are less amplifier systems and/or loudspeaker systems than there are signal paths. In either case (or in other cases) it may be desired to selectively combine audio signals corresponding to the sounds produced by two or more sound sources. Preferably, if such sounds or audio signals are mixed, selective mixing is performed so that signals having common characteristics (e.g., frequency, directivity, etc.) are mixed. This also enables modular expansion of the system.
As will be apparent from the foregoing, during the entire process from the detection of the sound to its reproduction by the loudspeakers, each of the audio signals corresponding to sounds produced by a sound source are preferably maintained separate from other sounds/audio signals produced by another sound source. Unless specifically desired to do so, the signals are not mixed. In this way, many of the problems with prior systems are avoided. While the foregoing discussion addresses the use of separate signal paths to keep the audio signals separate, it is to be understood that this may also be accomplished by multiplexing one or more signals over a signal path while maintaining the information separate (e.g., using time division multiplexing).
If desired, a feedback system 51 (
By way of example,
For purposes of example only, the sound sources SS1-SSN may include keyboards (e.g., a piano), strings (e.g., a guitar), bass (e.g., a cello), percussion (e.g., a drum), woodwinds (e.g., a clarinet), brass (e.g., a saxophone), and vocals (e.g., a human voice). These seven identified sound sources represent the seven major groups of musical sound sources. The invention does not require seven sound sources. More or less can be used. Of course, other sound sources or groups of sound sources may be also be used as indicated by box SSN. In the general case, N sound sources may be used where N is an integer greater than 1, or equal, but preferably greater than 1. It is well known that each of these seven major groups of musical sound sources have different audio characteristics and that, while each individual sound source within a group may have significant tonal differences (i.e., the violin and guitar), the sound sources within a group may have one or more common characteristics.
According to one aspect of the invention, the sounds produced by each of the N sound sources SS1-SSN are separately detected by one of a plurality of sound detectors SD1-SDN, for example, N microphones or microphone sets. Preferably, the sound detectors are directional to detect sound from substantially only one or selected ones of the plurality of sound sources. Each of the N sound detectors preferably detect sounds produced by one of the N sound sources and converts the detected sounds to audio signals. If each of the N sound sources simultaneously produces sound, then N separate audio signals will exist. Each sound detector may comprise one or more sound detection devices. For example, each sound detector may comprise more than one microphone. According to a preferred embodiment, three microphones (left, right and center) are used for each sound source. As detailed below, the use of these microphones is just one example of the use of a plurality of sound detection devices for each sound source. In other situations, more or less may be desired. For example, it may be desirable to surround a source with a plurality of microphones to obtain more directional information. The audio signals output from each of the N sound detectors or sound detection devices are supplied over a separate signal path as described above.
Each signal path may comprise multiple channels. For example, as shown in
The number of channels for a particular signal path need not be limited to three. More or fewer channels may be incorporated as desired. For example, a plurality of channels may be used to provide directional control (e.g., left, right and center). However, some or all of the channels may be used to provide frequency separation or for other purposes. For example, if three channels are used, each of the three channels could represent one musical instrument within a given group. For example, the musical group may be “strings” (e.g., if the event being recorded has two violins and one acoustical guitar). In this case, one channel could be used for one violin, another channel could be used for the second violin, and the third channel could be used for the acoustical guitar. Another use of separate channels is to enable power stepping, where one channel is used for audio signals up to a first level, then a second channel is added as the power level is increased above the first level, and so on. This method helps regulate the optimum efficiency level for each of the loudspeakers used in the loudspeaker network.
The recording process, if used, generally involves separately recording the M×N audio signals onto the recording medium 40 to enable the M×N signals to be subsequently read out and reproduced separately. The recording and read out may be accomplished in a standard manner by providing independent recording/reading heads for each signal path/channel or by time-division multiplexing the audio signals through one or more recording/reading heads onto or from M×N tracks of the recording medium.
According to another aspect of the invention, the separately recorded audio signals are separately reproduced. As shown in
According to one embodiment of the invention, each sound source may be a group of sound sources instead of an individual source. Preferably, each group includes sound sources with one or more similar characteristics. For example, these characteristics may include musical groupings (keyboards, strings, bass, percussion, woodwinds, brass group, and vocals), frequency bandwidth, or other characteristics. Thus, if more than one type of string instruments is used, it may be acceptable to use one signal path for the string instruments and separate signal paths, etc. for other sound sources or groups of sound sources. This still enables recognition of the advantages derived from the use of customized loudspeaker systems since sounds with common characteristics are produced by the same loudspeaker system.
According to one embodiment, the criteria used for grouping sound sources is related to a common dynamic behavior of particular audio signals when they are amplified. For example, a particular amplifier may have different distortion effects on different audio signals having different characteristics (e.g., frequency bandwidth). Thus, it also may be preferable to use a different type of amplifier system for different types of audio signals. Another criteria used for grouping sound sources is common directivity patterns. For instance, “horns” are very directional and can be grouped together while “keyboard instruments” are less directional than horns and would not be compatible with the “horns” customized speaker configuration, and therefore would not be grouped together with horns.
The sound system need not be limited to any particular number of signal paths. The number of signal paths can be increased or decreased to accommodate larger or smaller numbers of individual sound sources or sound groups. Further, application of the system is not limited to musical instruments and vocals. The sound system has many applications including standard movie theater sound systems, special movie theaters (e.g., OmniMax, IMAX, Expos) cyberspace/computer music, home entertainment, automobile and boat sound systems, modular concert systems (e.g., live concerts, virtual concerts), auto system electronic crossover interface, home system electronic crossover interface, church systems, audio/visual systems (e.g., advertising billboards, trade shows), educational applications, musical compositions, and HDTV applications, to name but a few.
Preferably, loudspeaker network 80 consists of several loudspeaker systems, each including a plurality of loudspeakers or loudspeaker clusters each of which is used for one of the signal paths. Each loudspeaker cluster includes one or more loudspeakers customized for the type of sounds that it is used to reproduce. A given loudspeaker cluster may be responsive to the power change of the corresponding amplification system. For example, if the power level supplied to a given loudspeaker network is below a first predetermined level, one or a group of loudspeaker components may be active to reproduce sound. If the power level exceeds the first predetermined level, a second or second group of loudspeaker components may become active to reproduce the sound. This avoids overloading the first loudspeaker (or first group of loudspeakers) and also avoids under powering the loudspeakers(s). Thus, depending on the power level of the audio signals on one (or more) of the signal paths, the individual loudspeakers within a given loudspeaker cluster can be automatically activated or deactivated (e.g., manually or automatically under control of the dynamics control module 90). Furthermore, a control signal embedded in the audio signal can identify the type of sound being delivered and thus trigger the precise group(s) of speakers, within a loudspeaker cluster, that most closely represents the characteristics of that signal (e.g., actual directivity pattern(s) of the sound source(s) being reproduced). For example, if the sound source being reproduced is a trumpet, the embedded control signal would trigger a very narrow group of speakers within the larger loudspeaker network, since the directivity of an actual trumpet is relatively narrow. Similar control can occur for other characteristics.
The audio signals, if digital, preferably are encoded and decoded at a sample rate of at least 88.2 KHz and 20-bit linear quantitization. Other sample rates and quantitization rates can be used however.
According to one embodiment of the invention, when a sound field is produced by a sound source, the plurality of transducers measures predetermined parameters of the sound field at predetermined locations on the enclosing surface over time. As detailed below, the predetermined parameters are used to model the sound field.
For example, assume a spherical enclosing surface Γa with N transducers located on the enclosing surface Γa. Further consider a radiating sound source surrounded by the enclosing surface, Γa (
While various types of transducers may be used for sound capture, any suitable device that converts acoustical data (e.g., pressure, frequency, etc.) into electrical, or optical data, or other usable data format for storing, retrieving, and transmitting acoustical data” may be used.
As illustrated in
Storage module 130 may store information, including modeled sound. According to an embodiment of the invention, storage module may store a model, thereby allowing the model to be recalled and sent to modification module 140 for modification, or sent to driver module 150 to have the model reproduced.
Modification module 140 may permit captured sound to be modified. Modification may include modifying volume, amplitude, directionality, and other parameters. While various aspects of the invention enable creation of sound that is substantially identical to an original sound field, purposeful modification may be desired. Actual sound field models can be modified, manipulated, etc. for various reasons including customized designs, acoustical compensation factors, amplitude extension, macro/micro projections, and other reasons. Modification module 140 may be software on a computer, a control board, or other devices for modifying a model.
Driver module 150 may instruct reproduction modules 160 to produce sounds according to a model. Driver module 150 may provide signals to control the output at reproduction modules 160. Signals may control various parameters of reproduction module 160, including amplitude, directivity, and other parameters.
Preferably there are N transducers located over the enclosing surface Γa of the sphere for capturing the original sound field and a corresponding number N of transducers for reconstructing the original sound field. According to an embodiment of the invention, there may be more or less transducers for reconstruction as compared to transducers for capturing. Other configurations may be used in accordance with the teachings of the invention.
According to an embodiment of the invention, as illustrated in
So, the two cases are as follows:
1. To reproduce the Carnegie Hall event, one needs to know the total reverberatory sound field within a volume, and fit that field with the array subject to spatial Nyquist convergence criteria. There would be no guarantee however that the field would converge anywhere outside this volume.
2. To reproduce the original instrument alone, one needs to know the outgoing (or propagating) field only over a circumscribing sphere, and fit that field with the array subject to convergence criteria on the sphere surface. If this field is fit with sufficient convergence, the field will continue to propagate within the playback environment as if the original instrument were actually playing within this volume.
Thus, in one case, an outgoing sound field on enclosing surface Γa has either been obtained in an anechoic environment or reverberatory effects of a bounding medium have been removed from the acoustic pressure P(a). This may be done by separating the sound field into its outgoing and incoming components. This may be performed by measuring the sound event, for example, within an anechoic environment, or by removing the reverberatory effects of the recording environment in a known manner. For example, the reverberatory effects can be removed in a known manner using techniques from spherical holography. For example, this requires the measurement of the surface pressure and velocity on two concentric spherical surfaces. This will permit a formal decomposition of the fields using spherical harmonics, and a determination of the outgoing and incoming components comprising the reverberatory field. In this event, we can replace the original source with an equivalent distribution of sources within enclosing surface Γa. Other methods may also be used.
By introducing a function Hi,j(ω), and defining it as the transfer function between source point “i” (of the equivalent source distribution) to field point “j” (on the enclosing surface Γa), and denoting the column vector of inputs to the sources χi(ω), i=1, 2 . . . N, as X, the column vector of acoustic pressures P(a)j j=1, 2, . . . N, on enclosing surface Γa as P, and the N×N transfer function matrix as H, then a solution for the independent inputs required for the equivalent source distribution to reproduce the acoustic pressure P(a) on enclosing surface Γa may be expressed as follows
X=H−1P. (Eqn. 1)
Given a knowledge of the acoustic pressure P(a) on the enclosing surface Γa, and a knowledge of the transfer function matrix (H), a solution for the inputs X may be obtained from Eqn. (1), subject to the condition that the matrix H−1 is nonsingular.
The spatial distribution of the equivalent source distribution may be a volumetric array of sound sources, or the array may be placed on the surface of a spherical structure, for example, but is not so limited. Determining factors for the relative distribution of the source distribution in relation to the enclosing surface Γa may include that they lie within enclosing surface Γa, that the inversion of the transfer function matrix, H−1, is nonsingular over the entire frequency range of interest, or other factors. The behavior of this inversion is connected with the spatial situation and frequency response of the sources through the appropriate Green's Function in a straightforward manner.
The equivalent source distributions may comprise one or more of:
Concerning the spatial sampling criteria in the measurement of acoustic pressure P(a) on the enclosing surface Γa, from Nyquist sampling criteria, a minimum requirement may be that a spatial sample be taken at least one half the highest wavelength of interest. For 20 kHz in air, this requires a spatial sample to be taken every 8 mm. For a spherical enclosing Γa surface of radius 2 meters, this results in approximately 683,600 sample locations over the entire surface. More or less may also be used.
Concerning the number of sources in the equivalent source distribution for the reproduction of acoustic pressure P(a), it is seen from Eqn. (1) that as many sources may be required as there are measurement locations on enclosing surface Γa. According to an embodiment of the invention, there may be more or less sources when compared to measurement locations. Other embodiments may also be used.
Concerning the directivity and amplitude variational capabilities of the array, it is an aspect of this invention to allow for increasing amplitude while maintaining the same spatial directivity characteristics of a lower amplitude response. This may be accomplished in the manner of solution as demonstrated in Eqn. 1, wherein now we multiply the matrix P by the desired scalar amplitude factor, while maintaining the original, relative amplitudes of acoustic pressure P(a) on enclosing surface Γa.
It is another aspect of this invention to vary the spatial directivity characteristics from the actual directivity pattern. This may be accomplished in a straightforward manner as in beamforming methods.
According to another aspect of the invention, the stored model of the sound field may be selectively recalled to create a sound event that is substantially the same as, or a purposely modified version of, the modeled and stored sound. As shown in
One advantage of the invention is that once a sound source has been modeled for a plurality of sounds and a sound library has been established, the sound reproduction equipment can be located where the sound source used to be to avoid the need for the sound source, or to duplicate the sound source, synthetically as many times as desired.
The invention takes into consideration the magnitude and direction of an original sound field over a spherical, or other surface, surrounding the original sound source. A synthetic sound source (for example, an inner spherical speaker cluster) can then reproduce the precise magnitude and direction of the original sound source at each of the individual transducer locations. The integral of all of the transducer locations (or segments) mathematically equates to a continuous function which can then determine the magnitude and direction at any point along the surface, not just the points at which the transducers are located.
According to another embodiment of the invention, the accuracy of a reconstructed sound field can be objectively determined by capturing and modeling the synthetic sound event using the same capture apparatus configuration and process as used to capture the original sound event. The synthetic sound source model can then be juxtaposed with the original sound source model to determine the precise differentials between the two models. The accuracy of the sonic reproduction can be expressed as a function of the differential measurements between the synthetic sound source model and the original sound source model. According to an embodiment of the invention, comparison of an original sound event model and a created sound event model may be performed using processor module 120.
Alternatively, the synthetic sound source can be manipulated in a variety of ways to alter the original sound field. For example, the sound projected from the synthetic sound source can be rotated with respect to the original sound field without physically moving the spherical speaker cluster. Additionally, the volume output of the synthetic source can be increased beyond the natural volume output levels of the original sound source. Additionally, the sound projected from the synthetic sound source can be narrowed or broadened by changing the algorithms of the individually powered loudspeakers within the spherical network of loudspeakers. Various other alterations or modifications of the sound source can be implemented.
By considering the original sound source to be a point source within an enclosing surface Γa, simple processing can be performed to model and reproduce the sound.
According to an embodiment, the sound capture occurs in an anechoic chamber or an open air environment with support structures for mounting the encompassing transducers. However, if other sound capture environments are used, known signal processing techniques can be applied to compensate for room effects. However, with larger numbers of transducers, the “compensating algorithms” can be somewhat more complex.
Once the playback system is designed based on given criteria, it can, from that point forward, be modified for various purposes, including compensation for acoustical deficiencies within the playback venue, personal preferences, macro/micro projections, and other purposes. An example of macro/micro projection is designing a synthetic sound source for various venue sizes. For example, a macro projection may be applicable when designing a synthetic sound source for an outdoor amphitheater. A micro projection may be applicable for an automobile venue. Amplitude extension is another example of macro/micro projection. This may be applicable when designing a synthetic sound source to perform 10 or 20 times the amplitude (loudness) of the original sound source. Additional purposes for modification may be narrowing or broadening the beam of projected sound (i.e., 360° reduced to 180°, etc.), altering the volume, pitch, or tone to interact more efficiently with the other individual sound sources within the same soundfield, or other purposes.
The invention takes into consideration the “directivity characteristics” of a given sound source to be synthesized. Since different sound sources (e.g., musical instruments) have different directivity patterns the enclosing surface and/or speaker configurations for a given sound source can be tailored to that particular sound source. For example, horns are very directional and therefore require much more directivity resolution (smaller speakers spaced closer together throughout the outer surface of a portion of a sphere, or other geometric configuration), while percussion instruments are much less directional and therefore require less directivity resolution (larger speakers spaced further apart over the surface of a portion of a sphere, or other geometric configuration).
Another aspect of the invention relates to a system and method for integral transference. Integral transference includes the process of transferring a sound event from one place, space, and time, to another place, space, and time, with little or no distortion to the integral form of the original event. The reproduced sound event should be nearly equivalent in every detail to the original sound event. Desired modifications to the original event may be made, but the applied modifications should be specified in terms of how they deviate from the integral form of the original event. By establishing a protocol such as that provided by various aspects of the invention, the integral form of the original event becomes a reference standard by which all reproductions may be gauged and by which all modifications may be specified. Accordingly, an overview of an integral transference system 300 is shown in
The integral reality of an acoustical event may be defined as the acoustical image projected onto an imaginary (or real) surface area (e.g., sphere) circumventing the event. Near field acoustical holography has been used to model the holographic acoustical dynamics of specified sound sources, usually as part of an engineering or design study for improving the acoustical characteristics of a given sound source (e.g., engine noise). As illustrated in
The invention takes into consideration the magnitude and direction of an original sound field over a spherical, or other surface area, surrounding the original sound source over, preferably, a 360 degree area. A synthetic sound source (for example, an inner spherical speaker cluster) modeled after the original sound field reproduces the precise magnitude and direction of the original sound source at each of the individual transducer locations. The integral of all of the transducer locations (or segments) mathematically equates to a continuous function which then determines the magnitude and direction at any point along the surface, not just the points at which the transducers are located. Such a system reproduces a sound event in a form that a listener is not able to determine whether the event is live or recorded.
To capture an original sound source (e.g., a musical instrument), the outgoing (or propagating) field is determined over a circumscribing area, and fitted with a transducer array subject to convergence criteria on the sphere surface. If this field is fit within sufficient convergence, the field will continue to propagate within the playback environment as if the original instrument were actually playing within this volume. Some aspects of the invention create a mathematical model of the captured source which may be stored in a sound source library as discussed herein or otherwise.
According to one aspect of the invention, integral transference starts with modularization, which relates to the breaking down of a sound event into its integral parts (
Object modules 24 relate to discrete sound producing entities (primary sources 25) and/or discrete sound affecting entities (secondary sources 27) within a given sound event. Object modules 24 are captured discretely, transferred discretely, and then reproduced discretely as synthetic objects in a reproduced event (
In terms of capturing an object module 24, recording transducers are placed along a grid that covers the surface area of an object and each piece of the grid is a sector, as shown in
According to another embodiment, element modules 28 are the most basic modules, consisting preferably of a single sound producing component (or power producing component) whether it be a tweeter, midrange, or mid-bass speaker, or in the power domain, an analog or digital amplifier. Element modules 28 may work together to change the dynamics of a sector module 26 which may also work together to change the dynamics of an object module 24.
Space modules 30 are somewhat different because they do not rely on the pyramid relationship associated with the element sector and object modules. Space modules 30 are a different type of modular component related to space, spatial qualities, spatial movement, relative location, and the like. For instance, if object module 24 is in the near-field close to the listener, then the space module 30 would be a near-field rendering apparatus. If object module 24 is in the far-field, then the rendering apparatus would be a far-field apparatus, considered a far-field space rendering apparatus. Other forms of space modules 30 exist when a space is divided into left, right, or surround sound directional components as is common is the discrete 5.1 (or 7.1) surround-sound format. Space modules 30 can also be used based on a spherical coordinate system for describing any point in space and the acoustical properties that exist at that point. Space modules 30 can also relate to movement algorithms that have to do with the relative position and location of object modules 24 and how they move in space relative to the listener and relative to one another.
Space modules 30 may operate independently of the object, sector, element modules (according to the modeling of the original event that is to be reproduced) and the engineering of the reproduced event based on the given resources. Space modules 30 also play an important role in the rendering of complex sound fields where primary and secondary sound sources co-exist in both the near field and far field, some moving while others may be stationary.
Intelligent modules 34 are an important component of integral transference. With intelligent modules 34, the integral transference technology can be engineered to be practical and eloquent while retaining the ability to render unique integral wave fronts for each discrete sound source within a given sound event, with less data than recording a full holographic or three-dimensional sound image of a given sound event. An overview of the use of intelligent modules 34 is illustrated in
The discrete transfer architecture according to the invention not only selectively segregates sound sources, it also serves as a transfer mechanism for segregated intelligent modules 34 and other forms of metadata that may apply to each segregated object module 24, as well as for control of “sector modules” 26, “element modules” 28 and “space modules” 32. Accordingly, a stored model of a sound field from an original sound source may be selectively recalled using the invention to create a sound event that is substantially the same as, or a purposely modified version of, the modeled and stored sound. The created sound event may be implemented by defining a predetermined geometrical surface (e.g., the spherical surface in
Thus, an advantage of the invention is that once a sound source has been modeled for a plurality of sounds, a sound library may be established, and the sound reproduction equipment can be located where the sound source used to be to avoid the need for the sound source, or to duplicate the sound source, synthetically as many times as desired.
According to one aspect of the invention, five primary intelligent module 34 categories are used in integral transference system 300: (1) source related intelligent module—data about a given sound source, (for example, its holographic acoustical “DNA” or fingerprint); (2) event related intelligent module—data regarding a given sound event (e.g., the spatial relationships of a plurality of sound sources in a given event); (3) system related intelligent module—data regarding a reproduction system's capabilities so it can be matched up with the content structure (e.g., number and type of rendering channels); (4) rendering appliance related intelligent module—data regarding a rendering appliance's capabilities; and (5) consumer related intelligent module—data regarding a consumer's preferences and other personal settings, adaptations, etc. More or less categories may be used.
Using intelligent modules 34, each sound source may be holographically captured and modeled resulting in an integral reality model which can then be used to synthesize a rendering appliance for projecting the same integral reality model on the same circumventing surface as the original sound source. The integral reality model is also used as a mechanism for building filters that allow spherical rendering apparatus to change dynamics based on the sound source being reproduced at the time.
Source intelligent modules may be used to streamline the process of transferring and recording acoustical code from the original event through the transfer process to the reproduction system for rendering. This process, called single node capture (
The design function according to the invention also plays a role in the engineering and development of the recording and reproduction system. Since the number of sound sources per acoustical event changes and the system characteristics within a home or automobile or other venue usually remains the same, intelligent module functions are required in order to coordinate the number of sources, the number of available transfer channels, and the number of available reproduction channels. Preferably, each sound source retains a discrete reproduction system for reproducing the integral wave form of each original sound source and each reproduction system retains a rendering mechanism that is capable of such.
Preferably, the state spherical rendering appliance according to the invention includes intelligent modules 34 built into it, or an intelligent module 34 driving it, which allows the appliance to change its filtering dynamics in order to render virtually any type of integral wave form produced by any type of sound source. For practical reasons, however, these types of segregation in number of channels and sources and reproduction mechanism may not be feasible and therefore some form of combining integral reality models and integral reality rendering mechanism is generally considered. The intelligent module functions play a vital role in how this done efficiently and effectively.
Modularization is another element that is impacted by intelligent module functions. Because modularization covers the discrete object models for each sound event, the role of the sector modules and element modules within each object module and the spatial modules including near field and far field rendering architectures are all preferably controlled by the intelligent module function. These control schemes may be hard coded into the signal during the recording process or they can be programmed into a delta Dynamics module as part of the reproduction process. The discrete transfer architecture not only transfers discrete acoustical code in the form of object modules 24 but also transfers intelligent module code corresponding to each discrete acoustical code and other intelligent module operations that must be transferred from the recording process to the reproduction process.
As stated earlier, when applying modularization, the original event is 32 deconstructed into object modules 24, sector modules 26, element modules 28 and space modules 32 and then transferred to a reproduction system that reconstructs these modules and reproduces the event. Each module may be controlled by the integral command and control system (
Programmable functions also exist which include the ability to program a reproduction system to match the ideal operating parameters for a given consumer, a process called E-modeling. The specific programs are called E-gorithms.
Accordingly, with the invention, for example, the performance of a four piece band (three instruments, one vocal) is recorded and reproduced in its integral form including the same macro/micro dynamics as the original event (
Thereafter, the DVD may be played on a DVD-A player 40 (for example) via a sound reproduction system 42 according to the invention which decodes both the intelligent module data and the sound source, feeding the decoded data into a dynamic controller 44 which controls how each of the separate sound sources is discretely amplified through amplifiers 46 and reproduced via sector module 26.
In the invention, the amplification process focuses on the amplification of the output, not the input. The output based on integral transference is a duplication of the integral wave input. In other words, if the original event consisted of three sound entities and those sound entities are captured in their integral form and transferred to the reproduction process and reproduced in their integral form, then the amplification process would be an amplified version of each integral wave, or an amplified integral wave form. This process called integral amplification may be first accomplished in the modeling domain. Once an integral reality model is captured and processed for a given sound source, the amplification of that model can take place in the modeling domain and the engineered rendering appliance can be used to create the amplified integral wave with little or no distortion.
Also important to the amplification process is the discrete nature of the transfer architecture (i.e., each sound source in the original event is captured and transferred and reproduced as a discrete entity) therefore the amplification process can be customized for that specific entity rather than using universal type components that are capable of amplifying and rendering any type of sound (usually in a planar wave form). By focusing on discrete entities for amplification, not only can the rendering appliance reproduce an amplified version of an integral wave form, but the definition between sound sources can also remain intact and the amplification curves (in terms of how each sound source is amplified relative to the other sound source and relative to the overall system elevated volume) can be customized and adjusted to match an individual persons taste.
In conjunction with integral amplification is integral scalability, both of which operate within the subheading of integral hyperization (i.e., that the integral wave of an original event is used and projecting into domains beyond its natural domain). For example, if an acoustical guitar is capable of producing an integral wave at a certain natural amplification, then if the integral wave is made ten times more elevated than normal, it would be beyond the natural ability of the guitar to produce a loudness of that magnitude. Through electronics in the invention, however, a hyper domain is created which is beyond the natural domain but retains the integral wave form.
The same concept applies towards scalability. An integral wave can be scaled down into a micro domain or it can be scaled up into a macro domain yet retaining the integral wave form of the original event. Thus, the individual sound entities may be spaced according to the original sound events spatial relationships and may be sized according to the venue designated for playback. For example, if a five piece band is recorded in a studio but played back in a automobile, then the integral transference rendering system 300 may be scaled down to match the venue size. On the other hand, if the reproduction venue is an outdoor amphitheatre, the rendering appliances may be scaled up in size and scope to meet the reproduction requirement of a large environment, all of this taking place without any distortions to the integral wave form of the original event. Deviations may also be engineered or created as desired or as mandated by resources, but preferably, the projection up and down in scale would take place with no distortions to the original wave form of the original event.
In terms of playback, in personal systems E-gorithms are specific ways of processing sound or configuring reproduction systems that appeal to specific preferences by specific people as opposed to E-models which appeal to a broader spectrum of people within certain broader type parameters. E-gorithms may be programmed into each individual system once his or her preferences are determined. For instance, someone might like the percussion to be stronger than someone else and therefore most of the sound reproduction that they experience will have an elevated percussion level. Some may desire to hear full integral wave form reproductions while others may require half-spherical reproduction mechanism. Some may require certain ambiance to be reproduced others may prefer no ambiance to be reproduced. These E-gorithms may be easily programmed or adjusted during the playback process according to each individuals criteria.
The MDF is based on the concept of modularization as discussed earlier and the fact that a sound reproduction system, according to the invention, may be gradually pieced together over time to achieve an ideal state system. Since each of the rendering appliances are modular, and since a discrete transfer architecture transfers sound sources discretely from the original event to the reproduction event, a system may be built up one source at a time and integrated with old technology as needed. For example, if someone cannot afford a seven channel discrete whole sound playback system they can first buy the percussion and bass breakout systems that would breakout the bass guitar and the drums and the bass drum and utilize special rendering appliances for those sound sources, while down-mixing the other sound sources together and playing them over a traditional stereo type format. Over time, as resources permit, the consumer can add additional rendering appliances and change the down-mix to apply to whatever sound sources do not have special integral transference rendering appliances. Furthermore each rendering appliance may be modular as well and gradually be built up from a partial integral form to a full integral form over time.
Also, it is a feature in the invention that the sector modules 26 and element modules 28 can be replaced as needed. This allows for more inexpensive components to be used at first to make it affordable for the masses, relying on the novel configuration for the sound improvement. Over time, more expensive better quality components can be changed out as element modules 28 in the system improve in terms of minor improvement in fidelity based on the quality of the elements like loudspeakers and amplifiers.
While commercial recording applications typically take into consideration the specifications and limitations of a recording medium (e.g., the number of available channels), live sound applications are not bound by the same limitations. Yet most live sound reproduction mechanism are configured remarkably similar to a recording studio. Inputs from discrete sound producing entities are usually routed into a central mixing board where some or all of the sound signals are mixed together and then outputted to a bank of amplifiers and loudspeakers, usually stacked on two sides of a stage resulting in a left/right stereo mix, similar to the stereo mix that is encoded onto a recording medium. The problem with this can be traced back to the paradigmatic context of the paradigm in use, in this case the stereo paradigm. By mixing sound source signals together and then sharing output devices like amplifiers and loudspeakers, many of the key components for rendering precise reproductions are dismissed (e.g., precise source definition, customized integral wave form rendering, integral wave form amplification, scalability, and hyperization mechanism, to name a few).
Integral transference of the invention proposes a novel approach for engineering and building live sound reproduction mechanism. The formula is the same as it would be for recording and reproducing sound events under ideal circumstances, only without the recording medium. Integral transference concept applies because the original event (unamplified) is transferred to a larger space, even though the time and place components remain the same. The objective is to amplify and render the original event while retaining the original event's distinct unamplified qualities, like discrete source definition, integral wave rendering, integral wave amplification, integral wave scalability, integral spatial congruency of discrete sound sources, and tonal accuracy. In short, the electronically amplified version of the original event becomes an enlarged version of the unamplified event.
An electronically enhanced version of the original event may maintain the same pure, undistorted qualities of the unenhanced version, only with broader reach and higher intensity. If modifications are desired, for instance because of the acoustics of a given venue, then the modification may be described in terms of how it deviates from the ideal state integral form of the undistorted, electronically enhanced, original event. As described earlier, this provides an objective reference point for describing and evaluating modifications and other deviations from a sound event's integral form.
Another component of the integral command and control process is a diagnostic component 500 (
Accordingly, if one of the segregated reproduction mechanism is malfunctioning or needing calibrating, the diagnostic system detects the problem independent of the other segregated reproduction mechanism. The diagnostic system 500 includes, for example, a plurality of diagnostic transducers (DT1-DTN), an active feedback module 54, an AI (acoustic intelligence) module 56, a sound recognition library 58, remote I/O 61, and an exterior sound sampler 62. A resolution to such problems may be segregated as well.
The diagnostics may also be used to create an objective reference standard by which reproductions can be completely and objectively compared. Accordingly, a reality reference standard is created by juxtaposing the integral reality models of the original event with the integral reality models of the reproduced event. Thus, sound events may be analyzed objectively by comparing in the proper context—their integral form. Furthermore, all modifications and derivatives in terms of how the sound deviates from the integral reality reference standard may be realized. For example, if a full spherical rendering mechanism is not required or desired then a half sphere system or quarter sphere system may be used and classified as a half integral reality system or a quarter integral reality system, respectively. Such modification protocol can be established in detail and applied to the commercialization process of integral transference systems 300.
Also related to integral standardization is the optimization protocol for optimizing components, sectors, object modules, and space modules according to predetermined criteria. Development of such reference standards and modification protocols makes it feasible for a sonic language that allows all reproductions to be described and all components to be described in terms of what role they play in the integral transference process.
The integral wave form of a near-field source in the invention is projected in its holographic or three-dimensional form in all directions just as it is in the natural domain. As a source gets further from the listener it becomes a midfield or far-field source then the integral form of the wave becomes less important because based on the Huygens' Principle: as a spherical wave propagates other spherical wave fronts form upon that wave front and as the wave front propagates further from its source the shape of the wave front becomes more planar.
In the near field, the integral wave form is important, especially for musical instruments. Musical instruments are designed to appeal to the total body sensory elements (music is felt in addition to being heard). The warmth and emotion generated by a live performance or a precise reproduction forms a unique listening experience. Thus, the three-dimensional aspects of a near field rendering, especially when amplified, play a key role in elevating the natural pleasure one receives while listening to music.
Accordingly, one embodiment of the invention presents a compound rendering architecture 600 (shown in
Far field sound sources may sometimes be rendered using a near field architecture due to scaling and other special perceptual effects. However, it is difficult for a far field rendering mechanism to effectively, in its integral form, render a near field source. Thus, the present embodiment of the invention allows for near field sources to be rendered using a equipment optimized for the near field while far field sources may be rendered using equipment optimized for the far field. Moreover, other rendering perspectives can also exist. Using the integral transference protocol, multiple rendering perspectives can be engineered into a compound rendering architecture.
In cases of macro sound events where a plurality of sound sources are activated simultaneously (e.g., musical event) the integral reality of the macro event can be determined as a whole (spherical boundary circumventing the macro event) or as a compilation of multiple micro events (integral reality models for each individual sound source). The latter case is the most proficient mechanism for calculating the macro integral reality because it proposes a more modular approach and operates within the near field domain which provides better definition and resolution in terms of modeling individual integral realities. Integral transference relies on an integrated modular approach, reproducing discrete integral realities, based on the distributive principle that a macro sound event is comprised of the sum of its primary and secondary sound sources.
While the ideal state approach implies that each primary sound source (sound producing entities) and secondary sound source (sound affecting entities) should retain a discrete capture, transfer, and reproduction mechanism, the invention includes methods in which certain entities may be combined together in the modeling domain and ultimately in the rendering domain based on predetermined criteria. For instance, if a given reproduction system maintains a limited rendering mechanism, say three discrete channels, and the original sound event is comprised of six discrete sources. The discrete integral reality models of common sound sources can be combined together and rendered through a composite integral wave rendering appliance.
Accordingly, integral transference reproduction system 300 with a limited number of reproduction sources operates as follows. A controller senses the number of sound sources that are required to reproduce the sound event from the recording medium and also senses the number of available amplification channels and number of sector modules available to reproduce the sound event. For optimum field definition and source resolution, each discrete sound source is preferably maintained with a segregated rendering mechanism. If combinations do have to occur, it is preferable that the grouping takes place among sources with common integral wave characteristics. One such solution, for example, is a standard seven channel system with each channel dedicated to one of the following musical groups: (1) strings, (2) brass, (3) horns, (4) woodwinds, (5) bass, (6) percussion, and (7) vocals. Each group may utilize a rendering mechanism customized according to the composite dynamics of all or most of the sources that fall into that group. A universal rendering mechanism for each group is then used accordingly. There are many other ways in which common sound sources can be combined together to produce composite integral waves according to the combined integral wave models of the original sources. Hybrid systems which combine integral transference appliances with more traditional type appliances (e.g., plane wave speakers) can be easily derived and utilized when necessary.
According to another embodiment of the invention, a computer usable medium having computer readable program code embodied therein for an electronic competition may be provided. For example, the computer usable medium may comprise a CD ROM, a floppy disk, a hard disk, or any other computer usable medium. One or more of the modules of system 100 may comprise computer readable program code that is provided on the computer usable medium such that when the computer usable medium is installed on a computer system, those modules cause the computer system to perform the functions described.
According to one embodiment, processor module 120, storage module 130, modification module 140, and driver module 150 may comprise computer readable code that, when installed on a computer, perform the functions described above. Also, only some of the modules may be provided in computer readable code.
According to one specific embodiment of the invention, system 300 may comprise components of a software system. System 300 may operate on a network and may be connected to other systems sharing a common database. According to an embodiment of the invention, multiple analog systems (e.g., cassette tapes) may operate in parallel to each other to accomplish the objections and functions of the invention. Other hardware arrangements may also be provided.
Having now described a few embodiments of the invention, it should be apparent to those skilled in the art that the foregoing is merely illustrative and not limiting, having been presented by way of example only. Numerous modifications and other embodiments are within the scope of ordinary skill in the art and are contemplated as falling within the scope of the invention as defined by the appended claims and equivalents thereto. The contents of all references, issued patents, and published patent applications cited throughout this application are hereby incorporated by reference. The appropriate components, processes, and methods of those patents, applications and other documents may be selected for the invention and embodiments thereof.
Other embodiments, uses and advantages of the invention will be apparent to those skilled in the art from consideration of the specification and practice of the invention disclosed herein. The specification and examples should be considered exemplary only.
This application is a continuation of U.S. patent application Ser. No. 10/673,232 filed Sep. 30, 2003 now abandoned which claims priority to provisional application No. 60/414,423 filed Sep. 30, 2002, the subject matter of which is incorporated by reference herein in its entirety. This application is related to U.S. patent application Ser. No. 08/749,766, filed Nov. 20, 1996, and U.S. patent application Ser. No. 09/393,324, filed Oct. 9, 1999, the subject matter of which is incorporated by reference herein in its entirety.
| Number | Name | Date | Kind |
|---|---|---|---|
| 257453 | Ader | May 1882 | A |
| 572981 | Goulvin | Dec 1896 | A |
| 1765735 | Phinney | Jun 1930 | A |
| 2352696 | De Boer et al. | Jul 1944 | A |
| 3158695 | Camras | Nov 1964 | A |
| 3540545 | Herleman et al. | Nov 1970 | A |
| 3710034 | Murry | Jan 1973 | A |
| 3944735 | Willcocks | Mar 1976 | A |
| 4072821 | Bauer | Feb 1978 | A |
| 4096353 | Bauer | Jun 1978 | A |
| 4105865 | Guillory | Aug 1978 | A |
| 4196313 | Griffiths | Apr 1980 | A |
| 4377101 | Santucci | Mar 1983 | A |
| 4393270 | Van Den Berg | Jul 1983 | A |
| 4408095 | Ariga et al. | Oct 1983 | A |
| 4422048 | Edwards | Dec 1983 | A |
| 4433209 | Kurosawa et al. | Feb 1984 | A |
| 4481660 | De Koning et al. | Nov 1984 | A |
| 4675906 | Sessler et al. | Jun 1987 | A |
| 4683591 | Dawson et al. | Jul 1987 | A |
| 4782471 | Klein | Nov 1988 | A |
| 5027403 | Short et al. | Jun 1991 | A |
| 5033092 | Sadaie | Jul 1991 | A |
| 5046101 | Lovejoy | Sep 1991 | A |
| 5058170 | Kanamori et al. | Oct 1991 | A |
| 5150262 | Hosokawa et al. | Sep 1992 | A |
| 5225618 | Wadhams | Jul 1993 | A |
| 5260920 | Ide et al. | Nov 1993 | A |
| 5315060 | Paroutaud | May 1994 | A |
| 5367506 | Inanaga et al. | Nov 1994 | A |
| 5400405 | Petroff | Mar 1995 | A |
| 5400433 | Davis et al. | Mar 1995 | A |
| 5404406 | Fuchigami et al. | Apr 1995 | A |
| 5452360 | Yamashita et al. | Sep 1995 | A |
| 5465302 | Lazzari et al. | Nov 1995 | A |
| 5497425 | Rapoport | Mar 1996 | A |
| 5506907 | Ueno et al. | Apr 1996 | A |
| 5506910 | Miller et al. | Apr 1996 | A |
| 5524059 | Zurcher | Jun 1996 | A |
| 5627897 | Gagliardini et al. | May 1997 | A |
| 5657393 | Crow | Aug 1997 | A |
| 5740260 | Odom | Apr 1998 | A |
| 5768393 | Mukojima et al. | Jun 1998 | A |
| 5781645 | Beale | Jul 1998 | A |
| 5790673 | Gossman | Aug 1998 | A |
| 5796843 | Inagawa et al. | Aug 1998 | A |
| 5809153 | Aylward et al. | Sep 1998 | A |
| 5812685 | Fujita et al. | Sep 1998 | A |
| 5822438 | Sekine et al. | Oct 1998 | A |
| 5850455 | Arnold et al. | Dec 1998 | A |
| 6021205 | Yamada et al. | Feb 2000 | A |
| 6041127 | Elko | Mar 2000 | A |
| 6154549 | Arnold et al. | Nov 2000 | A |
| 6219645 | Byers | Apr 2001 | B1 |
| 6239348 | Metcalf | May 2001 | B1 |
| 6356644 | Pollak | Mar 2002 | B1 |
| 6444892 | Metcalf | Sep 2002 | B1 |
| 6574339 | Kim et al. | Jun 2003 | B1 |
| 6608903 | Miyazaki et al. | Aug 2003 | B1 |
| 6664460 | Pennock et al. | Dec 2003 | B1 |
| 6686531 | Pennock et al. | Feb 2004 | B1 |
| 6738318 | Harris | May 2004 | B1 |
| 6740805 | Metcalf | May 2004 | B2 |
| 6829018 | Lin et al. | Dec 2004 | B2 |
| 6925426 | Hartmann | Aug 2005 | B1 |
| 6990211 | Parker | Jan 2006 | B2 |
| 20010055398 | Pachet et al. | Dec 2001 | A1 |
| 20030123673 | Kojima | Jul 2003 | A1 |
| Number | Date | Country |
|---|---|---|
| 0 593 228 | Apr 1994 | EP |
| Number | Date | Country | |
|---|---|---|---|
| 20060029242 A1 | Feb 2006 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60414423 | Sep 2002 | US |
| Number | Date | Country | |
|---|---|---|---|
| Parent | 10673232 | Sep 2003 | US |
| Child | 11247239 | US |
