Patent Grant
7470849
The present invention relates generally to frequency modulation (FM) synthesis, and, more particularly, to a method and system for generating audio waveforms used in FM music sound synthesis.
The reproduction of common waveforms in general, and of instrument sounds in particular, requires a collection of the primary components of that sound that when appropriately processed, can create a replica of that sound. The most accurate but impractical method would be a recording of that sound that includes all of its variants in frequency, attack, etc. Practical methods reduce the set of parameters needed to reproduce the sound. In the case of wavetable synthesis, each instrument is recorded and sampled over a small number of pitch cycles, over a subset of octaves. These sampled recordings are stored in a wavetable, and reproduction of the sound involves looping over this table.
FM synthesis replicates instrument sounds and therefore can be used as a synthesizer in music reproduction. Compared to the other methods of music generation, FM synthesis requires the least amount of memory for the music reproduction process while still maintaining an acceptable integrity of the instrument sound. It requires substantially less ROM and/or RAM memory for this synthesis method since it needs only a small set of pre-defined waveforms stored as a set of look-up wavetables. Wavetable synthesis, on the other hand, requires a much greater amount of memory in order to achieve an acceptable level of performance. In an example case where FM synthesis requires about 24 KB of instrument synthesis data comprising of the waveform tables, wave shaping data, and the wavetable synthesizer would require at least 512 KB of wavetable memory. This is a factor of 21 times the data size requirement for FM synthesis.
Besides the distinctive amplitude envelopes of sound produced from an instrument, sidetones create the timbre that distinguishes one instrument sound from another. Since sidetones are a naturally occurring and analytically derivable effect from performing frequency modulation on predefined waveforms, audio FM synthesis can be used to simulate instrument sounds. This is accomplished by matching the sidetones of the FM synthesized waveforms with the real instrument sidetones. A most basic FM synthesis tone generator uses a modulator frequency for self-modulation and to modulate a carrier frequency.
and a portion of the modulator signal, βr[n−1]. The waveform wavetable index is then used with a waveform look-up wavetable 106 to generate the first sequence of output samples W1[n]. A gain factor (Am[n]) is applied to this output resulting in r[n], where r[n]=Am[n]W1[n]. The delayed portion of this signal, βr[n−1], is then fed back to the modulator to calculate the next sample's modulator wavetable index. The amount fed back is determined by a gain factor (β), representing the modulator's frequency deviation. A portion of modulator signal, αr[n], is also fed forward to modulate the carrier frequency after a carrier gain factor, α, is applied. The carrier frequency,
is summed with αr[n] resulting in the wavetable index, φc[n]. This value is used with a wavetable 108 to yield the second sequence of output samples, W2[n]. A carrier gain factor, (Ac[n]), is then applied to obtain the final simulated instrument sound, SFM[n], where SFM[n]=Ac[n]W2[n]. Using this type of synthesis requires only a few waveform wavetables. For example, six waveform tables may be used to reproduce all of the 128 General MIDI instruments and 47 General MIDI drums.
FM synthesis may be chosen as the tone generator for music reproduction because of the economical benefits resulting from a smaller wavetable size requirement. This small set of waveforms placed in look-up wavetables would need substantially less ROM and/or RAM memory for this synthesis method.
In one implementation, the look-up waveform wavetables contain a complete cycle of all the necessary waveforms. The software algorithm computes a wavetable step index calculated from the carrier and modulator frequencies. This step index is accumulated and used to acquire each sample of the carrier and modulator waveforms from the appropriate wavetable. A simple wrapping algorithm (cycle-modulo arithmetic) is used to reproduce the continuous stream of the waveform. This single-cycle wrapping algorithm requires the minimum number of instruction cycles needed to recreate the carrier and modulator waveforms.
In another implementation, the symmetry of the waveforms can be used to reduce the memory size significantly since only part of a cycle (e.g., ¼ of a cycle or ½ of a cycle) of one of the waveforms is needed to generate all of the waveforms. Complex waveforms can be further created through segmentation of the cycle stored. This requires a more complex software wavetable look-up algorithm. The segments of the accumulated wavetable index has to be calculated, modulo arithmetic over the reduced cycle and full cycle needs to be performed, shifting of the index has to be done to adjust the step size, and a segment modification wavetable has to be used to scale the waveform within a segment.
However, the conventional wavetable look-up algorithm used today for FM synthesis systems experiences several drawbacks. For the aforementioned first implementation, a larger size memory device may be required since a full cycle of the waveform is stored. As such, when more wavetables are needed to be stored, a bigger memory device is needed. A large memory device can consume a larger physical area, thus increasing the die size, cost, and the power consumption of the chip. The access time to a large memory device is also higher than a small memory device, and it is also possible for timing violations to occur. A problem with timing violations is typically very costly to repair.
For the aforementioned second implementation that takes advantage of the symmetry and similarity of the waveform cycles, a smaller memory device is needed at the expense of an increase in instruction cycle usage. The increase is significant, and is doubled in FM synthesis since the wavetables are accessed twice per sample, once for the modulator frequency and once for the carrier frequency.
Therefore, it is desirable to implement a wavetable look-up algorithm that can reduce the memory size as well as the processor load.
There is a need for the following embodiments. Of course, the invention is not limited to these embodiments.
According to a first aspect of the invention, a method for generating one or more predetermined waveforms from one or more contiguous segments of at least one prototype waveform stored in one or more memory tables, comprises iterations of the following sample processing steps: reading at least one sample of the stored prototype waveform at a predetermined address, modifying the sample according to a predetermined logic, and accumulating the modified sample, wherein through a predetermined number of iterations of above steps, a cycle of a new waveform is formed by the accumulated modified samples.
According to a second aspect of the invention, a method for generating one or more predetermined waveforms from one or more contiguous segments of at least one prototype waveform stored in one or more memory tables comprising iterations of following sample processing steps: loading a first register with at least one address pointer, shifting the first register by a predetermined number of bits for providing a table address, reading at least one sample of the stored prototype waveform at the table address, providing a predetermined segment modification matrix table with a predetermined number of rows and a predetermined number of columns, selecting a column of the predetermined segment modification matrix table by an address provided by the first register, loading a second register with at least one row select address, selecting a row of the predetermined segment modification matrix table by an address provided by the second register, selecting a predetermined logic based on the content of the predetermined segment modification matrix table at the selected row and column, modifying the sample according to the predetermined logic (−1, 0, 1), and accumulating the modified sample, wherein through a predetermined number of iterations of the above steps, a cycle of new waveform is formed by the accumulated modified samples.
According to a third aspect of the invention, a waveform generating system with one or more contiguous segments of at least one prototype waveform stored in one or more memory tables, the waveform generating system comprising: at least one modification logic module for modifying at least one sample of the prototype waveform according to a predetermined logic set, at least one segment modification matrix table for selecting a logic operation from the predetermined logic set for modifying the samples, a first register for storing a column select address of the segment modification matrix table, a second register for storing a row select address of the segment modification matrix table, wherein a content at a selected column and row of the segment modification matrix table determines the selection of the logic operation, and at least one shift module for shifting the content of the first register by a predetermined number of bits for providing a table address to read a predetermined sample of the prototype waveform from the memory tables.
The construction and method of operation of the invention, however, together with additional objects and advantages thereof will be best understood from the following description of specific embodiments when read in connection with the accompanying drawings.
The drawings accompanying and forming part of this specification are included to depict certain aspects of the invention. A clearer conception of the invention, and of the components and operation of systems provided with the invention, will become more readily apparent by referring to the exemplary, and therefore non-limiting, embodiments illustrated in the drawings, wherein like reference numbers (if they occur in more than one view) designate the same elements. The invention may be better understood by reference to one or more of these drawings in combination with the description presented herein. It should be noted that the features illustrated in the drawings are not necessarily drawn to scale.
The following will provide a detailed description of a waveform generation method and system applicable in both hardware and software that can implement wavetable look-up algorithm in hardware, so that the increase in the processing cycles introduced by the more complex algorithm is eliminated. The present invention can reduce memory size, memory access time and processor instruction cycle usage at the same time.
To obtain a desired waveform, a prototype waveform is first selected, and then properly segmented and the amplitude is adjusted. The segmented and amplitude adjusted prototype waveform is then used to generate the desired and often more complex waveforms. The selection of the prototype waveform depends on applications. For example, in audio FM synthesis, the prototype waveform can be a sine, saw-tooth, ramp or exponential wave. Any waveform can be used as the prototype waveform as long as it has utilizable symmetry.
A memory table may be used to store one or more contiguous segments of the prototype waveform. The memory table used to store such information may be RAM or ROM. However, other methods of storing data (e.g., flip flops) may be implemented to meet the design criteria of a particular implementation. The memory table has a length equal to 2M, where M satisfies the relations N−K<=M<=N. For example, assuming N=11, K=2, and M=10, there will be 2048 samples for one period of the waveform and 4 segments, where each segment has 512 samples. In this example, the memory will contain 210 or 1024 samples.
Using the same prototype waveform 200 from
Note that the software implementation of this algorithm or method requires not only cycle modulo arithmetic but also partial-cycle modulo arithmetic for the fraction of the cycle actually stored. Segment modulo arithmetic needs also to be done to indicate the segment, and a new table needs to be defined to designate the polarity of each segment for all desired waveforms.
The memory interface block 304 is designed to perform the cycle, partial-cycle and segment modulo arithmetic from the wavetable phase step. A table of the reduced size waveform and a table of segment polarity are accessed to produce the sample output. Performing all the operations in software incurs a large cycle cost, so using a memory interface in hardware is preferred.
During FM synthesis, modulator sample processing occurs followed by carrier sample processing. With this method, during sample processing, the software calculates for each sample a phase step. This value and the waveform type are passed to the memory interface hardware where the cycle, partial-cycle and segment modulo arithmetic are first performed. Segment polarity modification (no change, negate or zero) is then applied to yield an output sample value.
In the beginning of a waveform generation, the processor 302 writes to the registers TableModCfgReg[0] and TableModCfgReg[1] within the register set 416. Those registers 416 may contain information that is needed throughout the processing and generation of a waveform, such as a pointer for the segment modification matrix table 412 that selects one of the rows. The TableModCfgReg[0] register is used when the modulator sample processing is performed. The TableModCfgReg[1] register is used when the carrier sample processing is performed. The segment modification matrix table 412 contains Y rows (Y is an integer), one for each required waveform, and 2K=X columns (X is an integer), one for each segment of the prototype waveform. For example, if 6 waveforms are supported and the prototype waveform is divided into 4 segments (quadrants), Y is equal to 6 and X is equal to 4. Therefore, the segment modification matrix table 412 for this example has a size of 6×4. The signal Col_sel, supplied by the TableIndexReg[0,1] 418, determines which column of the segment modification matrix table 412 will be selected. Each column of the segment modification matrix table 412 contains control bits to be applied to the modification logic 414 during its respective segment of the prototype waveform.
Each row matrix consists of the 2K-segment multipliers for reproducing a complete cycle modulator and carrier waveform. The output of the segment modification table 412 is used to modify the read memory output data. Table 1 shows a sample content of the segment modification matrix table 412. When a row and a column are selected, a certain number or key at the selected row and column will be chosen and sent to the modification logic 414.
The modification logic 414 gets the output of the segment modification matrix table 412 and modifies the data according to the selected key. In an example set of predetermined keys, 2-bit keys may be used, where 00 represents zero outputs, 01 represents no changes and 11 represents negate as shown in Table 2. Note that other options and/or keys can be used to meet the design criteria of a particular implementation.
The processor 302 calculates a wavetable step index and writes the value to one of the registers in the register set TableIndexReg[0,1] 418. As generating an FM synthesis waveform requires modulator sample processing followed by carrier sample processing, when the modulator sample processing is performed the TableIndexReg[0] register is used, and when the carrier sample processing is performed the TableIndexReg[1] register is used. The processor 302 writes to each of the registers in the register set 418 once per sampling.
The value in the register set TableIndexReg[0,1] 418 may be bit shifted by the shift logic device 408 and the shifted value becomes the address for reading the wavetable memory 420. Data output from the wavetable memory 420 may be modified by the modification logic 414, and the modified data value is put on the data-in bus of the processor 302.
For the shift logic device 408, the SFT_RIGHT operation performs a base 2 increase or decrease of the frequency of the waveform. For SFT_RIGHT=0 the frequency of the original waveform remains the same. For SFT_RIGHT=1, the frequency doubles and for the SFT_RIGHT=−1, the frequency is halved. The read operation causes a reading from the wavetable memory module 420 according to the address provided by the shift logic device 408.
The data space of processor 302 may include data memory 404 to support other functions, which are not related to this invention. The processor 302 has a direct access to the data memory 404. The read data goes to the data-in bus of the processor 302 through a multiplexer 410. Multiplexer, 410, is needed when there is more than one source that can put data on the data-in bus of the processor 302.
Note that the data memory module 404 and the wavetable memory module 420, while depicted as two logical memory blocks, can be combined into one contiguous block of physical memory. In that case, additional logic (e.g., a decoder and a multiplexer, not shown in
Generation of such complex waveforms would require a significant increase in instructions and data, however, with a modification to the memory interface and an addition of a few logic gates, the software operation stays the same, and the generation of the more complex waveforms does not increase the instruction cycle usage of the processor 302.
With this method, when the prototype waveform is divided into more segments, other FM waveforms with higher complexity can be created, possibly improving the replication of the instrument sounds.
Other modifications and various changes to the memory interface module 304 other than what is presented in the foregoing paragraphs may be possible in order to generate FM synthesized waveforms or any other type of waveforms, without departing from the spirit and scope of the invention.
In step 802, the DSP loads registers, and TableIndexReg[0,1], once per sampling. These registers determine which column of the segment modification matrix table may be selected (referring to
Since the prototype waveform has 2N (e.g., 2048) samples per cycle, then modulo-(2N−1) arithmetic is performed on the accumulated wavetable index for both modulator processing in step 804 and carrier processing in step 808. This operation is automatically executed by the processor writing an x-bit value to the TableIndexRegX[N−1:0] Register (X=0 or 1). For N=11, the modulo-arithmetic is done over a value of 0x7FF (or 2047).
Details of the waveform processing of step 806 or 810 are described as follows. The ‘N−1’ to ‘N−1−K’ bits of this register, TableIndexRegX[N−1:N−1−K], determines the segment of the waveform. These bits are used for addressing the corresponding column of the segment modification matrix, which provides the polarity for that segment. For example, for N=11 and K=2, the wavetable index will wrap around a value of 0x7FF, and the 4 segments (2K=4) are defined in Table 6:
In step 806 or 810, the TableIndexRegX[N−1:0] register is then shifted right (SFT_RIGHT) by its length or len. The SFT_RIGHT operation accomplishes two functions: (1) performing the partial-cycle modulo-(2M−1) arithmetic, and (2) adjusting the prototype waveform frequency. The modulo-(2M−1) arithmetic serves the function of wrapping the wavetable index around the size of all or a portion of the partial waveform cycle saved in memory. For example, for N=11 and M=10, only ½ of the full cycle (2N samples) is saved in memory. Therefore, len would be N−M, or 1. For integer values larger than 1, each shift serves to double the frequency of the prototype waveform. For a len=1, the frequency of the prototype waveform remains the same and the shift right operation only functions as a wrap around the ½ cycle of the waveform. For a len=2 and 3, the prototype waveform will be respectively doubled and quadrupled in frequency. The resulting value is used as the address into the stored waveform wavetable from which the sample value is extracted.
The processor reads from a predefined address. The read operation causes a reading from the wavetable memory according to the calculated address. The sample value is modified by the segment modification matrix value (a multiply of 0, 1 or −1). This results in the final sample value that is read by the processor.
In step 812, the process checks if a predetermined number of samples per block have been reached, if not, the processor will go through another round of modulator processing and carrier processing by starting at step 802 again, otherwise, the processor will exit the processing.
Note that the present invention may be used in portions of a code division multiple access (CDMA) chipset. While the present invention may be useful in CDMA designs, the present invention may be applied to generate FM waveforms or any other type of waveforms for other designs as well.
The above illustration provides many different embodiments or embodiments for implementing different features of the invention. Specific embodiments of components and processes are described to help clarify the invention. These are, of course, merely embodiments and are not intended to limit the invention from that described in the claims.
Although the invention is illustrated and described herein as embodied in one or more specific examples, it is nevertheless not intended to be limited to the details shown, since various modifications and structural changes may be made therein without departing from the spirit of the invention and within the scope and range of equivalents of the claims. Accordingly, it is appropriate that the appended claims be construed broadly and in a manner consistent with the scope of the invention, as set forth in the following claims.
The present application claims the benefit of U.S. Provisional Application Ser. 60/723,343, which was filed on Oct. 4, 2005.
| Number | Name | Date | Kind |
|---|---|---|---|
| 4440058 | Bass et al. | Apr 1984 | A |
| 4810977 | Flugstad et al. | Mar 1989 | A |
| 5541354 | Farrett et al. | Jul 1996 | A |
| 5619535 | Alvarez, Jr. | Apr 1997 | A |
| 5969703 | Surguy | Oct 1999 | A |
| 6373316 | Wu | Apr 2002 | B1 |
| 6621426 | van Nguyen | Sep 2003 | B1 |
| 20020046639 | Kikumoto | Apr 2002 | A1 |
| 20030014082 | Schu et al. | Jan 2003 | A1 |
| 20040184559 | Ballantyne | Sep 2004 | A1 |
| Number | Date | Country |
|---|---|---|
| WO2004095731 | Nov 2004 | WO |
| Number | Date | Country | |
|---|---|---|---|
| 20070079689 A1 | Apr 2007 | US |
| Number | Date | Country | |
|---|---|---|---|
| 60723343 | Oct 2005 | US |
