This application claims priority to United Kingdom Patent Application No. 1112676.0 filed on Jul. 22, 2011, the entire content of which is incorporated herein by reference.
The present invention relates to methods of sound analysis and associated sound synthesis, for example in real time; for example, the present invention relates to methods of analyzing sounds for determining their timbral characteristics, and then applying the timbral characteristics onto another sound in real time. Moreover, the present invention also concerns apparatus operable to execute aforesaid methods. Furthermore, the present invention relates to software products recorded on machine-readable data storage media, wherein the software products are executable upon computing hardware for implementing aforesaid methods.
When a band of musicians or an individual musical artist is desirous to making a recording, for example a record or album, it is beneficial to use facilities included in a recording studio. In its most basic form, a recording studio includes a room in which the band or artist is located when making music, a control room for a recording engineer, together with recording gear. The recording gear includes equipment such as microphones, cables, monitor speakers and a multitrack recorder. Optionally, the multitrack recorder is implemented digitally using an AD-converter, a DA-converter and a personal computer executing appropriate multitrack recording software. Alternatively, the multitrack recorder is implemented as a more conventional electromechanical device using magnetic recording tape.
It is contemporarily feasible to implement a professional home recording studio provided that a skilled recording engineer is employed to operate the studio and other associated equipment such as microphones are of sufficiently high quality. In the year 2011, it is estimated that professional-quality recording equipment needed for implementing a small home recording studio is in an order of Euro 5000. If a personal computer is employed for the multitrack recorder, for example a laptop computer, the equipment for implementing the recording studio is potentially highly portable. However, expensive home recording studios are also purchasable, for example a proprietary Pyramix mastering workstation is estimated to cost in an order of 20000 USD (USD=United States dollars).
When employing an aforementioned contemporary studio, an actual recording process involves each musician playing his or her part separately to provide a plurality of “takes”, and then the takes are combined in a mixing process where characteristics of each take is individually adjustable so as to obtain a preferred balance between the takes. For example, in a case of recording a rock band at a home studio, a drummer of the band would be recorded first to provide a drummer take, typically whilst hearing a demo guitar and demo bass via headphones so as to obtain a correct duration of musical bars to the recording. Optionally, the demo guitar and demo bass, via their respective musicians, are played together with the drummer in a manner such that guitar and bass amplifiers are located in a separate room to avoid their sound contribution being included into the drummers take; the sound of the guitar and bass is transduced using microphones and corresponding microphone signals mixed together to generate a corresponding mixed signal which is then employed to drive headphones of the drummer, and of the musicians playing the bass and guitar. However, the guitar and bass signals are not recorded at this point to provide corresponding takes of the bass and the guitar, because their sole purpose is to guide the drummer when playing to provide the drummer take.
Often, after several repetitions and corresponding recordings, a recording engineer and members of the rock band are satisfied with the drummer take, the activities are then focused to generate a bass take, namely bass guitar take. During recording of the bass take, the bass guitar musician is provided with a replay of the drummer take via headphones, optionally together with the demo guitar. This process of progressing recording takes is repeated until takes for all members of the rock band have been recorded by the recording engineer.
After the takes have been completed, a mixing engineer fine-tunes each of the takes individually, normally starting with the drummer take. Optionally, the takes are mixed to generate a composite track by way of a mixing process that is optionally executed in the aforesaid control room; beneficially, the control room is an acoustically treated room including high-quality loudspeakers and a computer. For example, the control room is acoustically treated room, which is substantially devoid of natural reverberation. The mixing engineer is operable to add various sound effects to the takes, for example signal limiting, equalization, dynamic range compression and so forth when generating the composite track until the musicians in the band are satisfied with the composite track. Whilst adjusting a given take, namely “track”, the mixing engineer ensures that respective timbres of the takes are mutually compatible when mixing to generate the composite track. The composite track substantially corresponds to a final mix which is eventually for broadcast, sale via data carriers such as CD's and records, or otherwise disseminated to the public, although certain mastering adjustments to the composite track are often implemented in practice for obtaining a best rendition in the final mix. A total number of takes mixed together to form a corresponding composite track often includes several dozen takes, and preparation of a composite track take often require hours, days, even weeks of work. The composite tracks are included together by a mastering engineer to provide a final album for dissemination to the public.
The mastering engineer has a task of finalizing an overall sound of the album. The mastering engineer is thus operable to execute a mastering process, which is usually implemented much faster than aforesaid mixing activities implemented by the mixing engineer. Typically, the mastering process is executed within a couple of days. For example, the mastering engineer has a task of making the album sound as loud as possible when program material pertains to rock music. Human appreciation of sound, namely a combination of human ear activity and human brain activity, finds louder sounds more interesting than quieter sounds. Since the album producing process executed by the mastering engineer cannot in practice influence a volume setting of a consumers earpiece, the mastering engineer is operable to apply certain audio effects, which cause the sound to be perceived on listening to be louder than it actually is in reality. These effects include dynamic compression as well as an addition of subtle distortion effects.
Since given rock bands and record producers desire that listeners, namely customers, to find their particular albums more interesting in comparison to competing artists and albums, a generally similar loudness enhancing maximization is applied on all contemporary rock records and similar, with a consequent result that most contemporary rock band albums sound mutually equally loud, too mutually similar and fatiguing to listeners.
Contemporary albums involve slow and tedious manual work on the part of the recording engineer, the mixing engineer and the mastering engineer, as well as the musicians, for example during mastering and especially mixing of takes. Such work involves experimenting with different mixes, whereas work involved with overall sounds of rock bands or artists is kept to a minimum. Consequently, artistic freedom becomes limited on account of mixing and mastering engineers not being inclined to take risks and potentially jeopardize several days' work. Moreover, since home recording studios have become more common, the artist, the recording engineer, the mixing engineer, the mastering engineer, as well as the produce for albums produced by the home studio are often implemented by one person.
Clearly, a contemporary need arises for sound processing methods which enable recordings in albums to be enhanced which enables them compete better against other albums.
In a published U.S. Pat. No. 4,984,495 (“Musical Tone Signal Generating Apparatus”, Applicant—Yamaha Corp.; inventor—Fujimori), there is described a musical tone signal generating apparatus. The apparatus is implemented such that first sampling data and second sampling data are multiplied together by a convolution operation, wherein the first sampling data indicates instantaneous amplitude values of a musical tone waveform generated from a keyboard, for example. The second sampling data is obtained from an impulse response waveform signal indicative of a reverberation characteristic of a room or an acoustic characteristic of an amplifier or musical instrument such as a guitar or a piano. Alternatively, the second data can be obtained from a waveform signal indicative of an animal sound, a natural sound or the like. Then, the multiplication result of the first and second sampling data is combined together into the musical tome waveform data, whereby a musical tone signal corresponding to this musical tone waveform data is generated. Thus, the musical tone is modulated with another sound such that the reverberation or acoustic characteristic will be simulated in the musical tone to be generated, whereby the variable musical effect can be applied to the musical tone.
The various embodiments of the present invention seeks to provide an improved sound analysis and associated sound synthesis method which is capable of copying timbral characteristics from one signal onto another by way of one or more impulse responses.
According to a first aspect, there is provided a sound analysis and associated sound synthesis method as claimed in appended claim 1: there is provided a sound analysis and associated sound synthesis method, wherein the method includes:
The embodiment is of advantage in that the method is capable of applying timbral nuances to the second signals when generating the corresponding output sound signal.
Optionally, the method is implemented in real time using software products executing upon computing hardware.
Optionally, the method is implemented such that multiple impulse responses from (b) are stored on a database, and are user-selectable for applying to the second input sound signal.
Optionally, the method is implemented such that steps (b) and (e) employ at least one of: signal delay functions, signal resonance functions, non-linear functions, Fourier transform functions.
Optionally, the method is implemented such that step (d) includes generating a “pink noise” equivalent frequency spectrum of the second input sound signal. More optionally, the method is implemented such that step (d) includes adding distortion of a form associated with magnetic tape recorders.
Optionally, the method is implemented such that steps (b) and (d) include a signal loudness estimation, for use in step (e) for adjusting the loudness of the processed sound.
Optionally, the method is adapted for applying timbral characteristics corresponding to thermionic electron tube amplifiers.
According to a second aspect, there is provided an apparatus operable to execute a method pursuant to the first aspect of the invention, wherein the apparatus includes:
According to a third aspect, there is provided a software product recorded on a machine-readable data storage medium, wherein the software product is executable upon computing hardware for implementing a method pursuant to the first aspect of the invention.
It will be appreciated that features of the invention are susceptible to being combined in various combinations without departing from the scope of the invention as defined by the appended claims.
Embodiments of the present invention will now be described, by way of example only, with reference to the following drawings wherein:
In the accompanying drawings, an underlined number is employed to represent an item over which the underlined number is positioned or an item to which the underlined number is adjacent. A non-underlined number relates to an item identified by a line linking the non-underlined number to the item. When a number is non-underlined and accompanied by an associated arrow, the non-underlined number is used to identify a general item at which the arrow is pointing.
In overview, the various embodiments of the present invention are concerned with methods of sound analysis and sound synthesis, for example methods of analyzing sounds for determining timbre as represented by a set of parameters, and then applying these parameters via sound synthesis to process other sounds to impart thereto the analyzed timbre. The method is, for example, potentially applicable to takes used for producing albums for imparting greater interest to other sounds included in the albums.
If one hears two equally loud notes, for example middle-C played on a clarinet and on a guitar, it is possible for a human being to distinguish between the sounds even though they are of nominally similar pitch and amplitude. Such distinguishing is governed by several factors such as harmonic development, sound attack characteristics, sound decay characteristics and subtle harmonic instabilities arising when the notes are being sounded. The notes are beneficially analyzed as a series of harmonic components in a spectrum, wherein the harmonic components are a function of time from when the note is initially sounded.
For example, sounds that change with time are a piano tone. Firstly, there is a thump of a piano hammer hitting a piano string, immediately followed by a bright sustained ringing tone of the piano string, which gradually becomes mellower and finally fades out. Thus, if a spectrum of an entire piano note were graphically plotted as a function of time, it would contain a sonic average of the initial thump, the bright ringing part, and the mellower fading part, all mixed together in a temporally changing sequence. Thus, the tone of the piano can be represented by Equation 1 (Eq. 1):
wherein
It will be appreciated from Equation 1 (Eq. 1) that faithful representation of a piano tone is potentially highly complex. It is thus commonplace for contemporary electronic musical instruments such as digital pianos to employ sampled sounds of real pianos, rather than attempting to solve Equation 1 (Eq. 1) for a piano tone.
Pursuant to the present embodiment of the invention, the embodiment provides a method of sound analysis to determine timbral characteristics of a first sound signal to derive parameters representative of the timbre, and then thereafter to apply the parameters to a second sound signal to impose upon the second sound signal the timbral characteristics to modify the second sound signal to generate a third sound signal. The third sound signal has timbral nuances to the first sound signal. Practical applications of the method include mimicking the effect of sound processing devices, for example a record mastering effects chain, or non-distorting parts of a guitar amplifier-loudspeaker combination. The parameters representative of the timbre are conveniently, for example, derived by way of obtaining an impulse response. An impulse, for example a Dirac-type pulse, is characterized by having a broad flat spectrum of harmonic components. However, when a system has restricted dynamic range, it is alternatively possible, to a first approximation, to employ a swept frequency signal as a substitute for a Dirac-type pulse.
Tonal coloration caused by acoustic or electrical systems is susceptible to being expressed explicitly using one or more impulse responses. As its name implies, an impulse response represents a system's response to being stimulated by an impulse signal. For an acoustic system, for example a concert hall, the stimulating impulse signal is a temporally abrupt sound of a start pistol, and the corresponding impulse response is the sound of reverberation of the start pistol being fired within the concert hall. By analyzing the impulse response, it is possible to computer parameters representative of the reverberation characteristics of the concert hall, and then to apply the parameters via a mathematical function to sound signals to make them sound as if they were being performed in the concert hall.
In practice, the impulse response is not measured using an impulse signal on account of a low signal-to-noise ratio which would pertain when computing the aforesaid one or more parameters. Alternatively, the impulse response may be derived using a broadband signal source to generate a stimulating signal; “broadband” here means a signal concurrently including a plurality of sinusoidal signal components, for example several thousands sinusoidal components, spread over a broad frequency range from low frequencies, for example 20 Hz, to high frequencies, for example 20 kHz.
With regard to signal processing, when a broadband stimulating signal is employed to stimulate an acoustic system, a corresponding impulse response may be obtained by de-convolving a measured response from the acoustic system to the stimulating signal. The impulse response, represented by one or more parameters is then beneficially applied, pursuant to the present invention, to other sound signals to mimic an acoustic effect of the system. The impulse response can, for example, be represented as a series of signal time delays and signal resonances with associated signal gains and resonance Q-factors. However, computations become more difficult when the system is non-linear when transforming the broadband stimulating signal into an output signal from the system. Such complications arise when the system includes a guitar amplifier output stage, for example when implemented using thermionic vacuum tubes as power amplifying components. Equalization circuitry of an amplifier and microphones tends to add very little non-linearity. However, loudspeakers can add considerable non-linearity when driven at high power levels such that loudspeaker diaphragm movement is a major part of a total mechanical movement range which is possible for the diaphragm (for example defined by diaphragm surround support and voice-coil support arrangement known as a “spiders web support”). A conventional Volterra convolution is susceptible to being employed when the system exhibits non-linearity in its dynamic response, whereas other convolutions are beneficially employed when only linear effects occur.
Pursuant to the present embodiment, it is desirable to copy a timbre of a sound (represented by parameters describing its equivalent impulse response) and apply it to another sound, for example to generate interesting sonic effects in aforesaid albums to render them more appealing in comparison to competing albums. For example, many guitarists are desirous to copy guitar tones from some earlier famous guitarist. Moreover, record producers may be desirous to copy an overall timbre of some given classic record onto a new record on which they are working.
It will be appreciated from the foregoing that exact copying and re-applying timbral characteristics is a technically difficult problem that has hitherto not been adequately addressed, especially not in real time. The present invention enables timbral characteristics to be copied in real time and applied to another sound.
The present embodiment will now be further elucidated with reference to
The impulse response as derived in an arrangement illustrated in
Additionally in
The present embodiment is beneficially employed as a record mastering tool, for example for use in aforesaid recording studios. Moreover, the synthesis steps depicted in
Optionally, the present embodiment is applied to generate an impulse library of several pre-existing musical records. The synthesis steps of
As aforementioned, the present embodiment is especially suitable when synthesizing the timbre of “valve” power amplifiers and associated loudspeaker combinations in association with guitar. A users own signal derived from a dummy load applied to an output of an amplifier driven from a guitar is fed through steps of
The present embodiment is capable of providing considering benefits in comparison to known software products for applying sound modification, for example proprietary Nebula effect samplers and similar. Nebula effect samplers employ Volterra-based modeling techniques are not well suited for simulating “valve” amplifiers or distortion effects pedals, due to the computational overcomplexity of synthesizing strongly saturating distortions using the Volterra technique. In the context of the electric guitar, the present invention, used in conjunction with a valve amplifier, is capable of providing a major benefit of only requiring a clip of sound to work with to generate a corresponding impulse response for synthesis, whereas known Volterra-based effects require access to actual devices that are to be simulated.
The present embodiment is susceptible to being manufactured as software products executable upon computing hardware. Moreover, the present invention has technical effect by processing real signals to generate corresponding processing signals having unusual technical characteristics that are also aesthetically pleasing and beneficial when producing tangible products such as albums.
Modifications to embodiments of the invention described in the foregoing are possible without departing from the scope of the invention as defined by the accompanying claims. Expressions such as “including”, “comprising”, “incorporating”, “consisting of”, “have”, “is” used to describe and claim the present invention are intended to be construed in a non-exclusive manner, namely allowing for items, components or elements not explicitly described also to be present. Reference to the singular is also to be construed to relate to the plural. Numerals included within parentheses in the accompanying claims are intended to assist understanding of the claims and should not be construed in any way to limit subject matter claimed by these claims.
Number | Name | Date | Kind |
---|---|---|---|
4984495 | Fujimori | Jan 1991 | A |
8538042 | Yang et al. | Sep 2013 | B2 |
20050108004 | Otani et al. | May 2005 | A1 |
20070193435 | Hardesty et al. | Aug 2007 | A1 |
20110038490 | Yang et al. | Feb 2011 | A1 |
20130343573 | Yang et al. | Dec 2013 | A1 |
Number | Date | Country |
---|---|---|
0505949 | Sep 1992 | EP |
05027770 | Feb 1993 | JP |
2003050582 | Feb 2003 | JP |
0028521 | May 2000 | WO |
2011019339 | Feb 2011 | WO |
Entry |
---|
European Search Report. Application No./Patent No. 12005290.7-2225. Date of completion of the search: Nov. 21, 2012. |
Dolphin Music, “Universal Audio Premiers Studer A800 Multichannel Tape Recorder Plug in for UAD-2”, Dec. 7, 2010, http://www.dolphinmusic.co.uk/article/4639-universal-audio-premiers-studer-a800-multichannel-tape-recorder-plug-in-for-uad-2.html [accessed Nov. 14, 2011]. |
United Kingdom Intellectual Property Office Search Report. Application No. GB1112675.2. Date of search: Nov. 15, 2011. |
United Kingdom Intellectual Property Office Search Report. Application No. GB1112676.0. Date of search: Nov. 14, 2011. |
Number | Date | Country | |
---|---|---|---|
20130019739 A1 | Jan 2013 | US |