Patent Application
20080229915
This disclosure relates to audio devices and, more particularly, to audio devices that generate audio output based on musical instrument digital interface (MIDI) files.
Musical Instrument Digital Interface (MIDI) is a format used in the creation, communication and/or playback of audio sounds, such as music, speech, tones, alerts, and the like. A device that supports playback of MIDI files may store sets of audio information that can be used to create various “voices.” Each voice may correspond to one or more sounds, such as a musical note by a particular instrument. For example, a first voice may correspond to a middle C as played by a piano, a second voice may correspond to a middle C as played by a trombone, a third voice may correspond to a D# as played by a trombone, and so on. In order to replicate the musical note as played by a particular instrument, a MIDI compliant device may include a set of information for voices that specify various audio characteristics, such as the behavior of a low-frequency oscillator, effects such as vibrato, and a number of other audio characteristics that can affect the perception of sound. Almost any sound can be defined, conveyed in a MIDI file, and reproduced by a device that supports the MIDI format.
A device that supports the MIDI format may produce a musical note (or other sound) when an event occurs that indicates that the device should start producing the note. Similarly, the device stops producing the musical note when an event occurs that indicates that the device should stop producing the note. An entire musical composition may be encoded in accordance with the MIDI format by specifying events that indicate when certain voices should start and stop. In this way, the musical composition may be stored and transmitted in a compact file format according to the MIDI format.
MIDI is supported in a wide variety of devices. For example, wireless communication devices, such as radiotelephones, may support MIDI files for downloadable sounds such as ringtones or other audio output. Digital music players, such as the “iPod” devices sold by Apple Computer, Inc and the “Zune” devices sold by Microsoft Corporation may also support MIDI file formats. Other devices that support the MIDI format may include various music synthesizers, wireless mobile devices, direct two-way communication devices (sometimes called walkie-talkies), network telephones, personal computers, desktop and laptop computers, workstations, satellite radio devices, intercom devices, radio broadcasting devices, hand-held gaming devices, circuit boards installed in devices, information kiosks, video game consoles, various computerized toys for children, on-board computers used in automobiles, watercraft and aircraft, and a wide variety of other devices.
This disclosure describes devices that store Musical Instrument Digital Interface (MIDI) parameters for efficient access in the processing of such parameters. As described herein, a storage unit within memory may be partitioned into three regions comprising locations that can store different types of MIDI parameters. The partitioning allows for efficient access by both a processor and a hardware unit.
The MIDI parameters may be generated from MIDI events of a MIDI file. In particular, MIDI events may be converted into MIDI parameters via a processor, such as a digital signal processor (DSP). The DSP may divide the MIDI parameters into different parameter sets to be stored in a storage unit within memory. Memory may comprise a plurality of storage units. Also, the DSP may schedule the processing of the MIDI parameters in a hardware unit.
The storage unit within memory may be partitioned into at least three regions. A first region is accessible by both the processor and the hardware unit. A second region is accessible by the processor and inaccessible by the hardware unit. A third region is accessible by the hardware unit and inaccessible by the processor after initialization.
The hardware unit may access the MIDI parameters used to update new voices. The hardware unit may access one of the first and third regions of a partitioned storage unit within memory. The hardware unit may process the MIDI parameters stored in one of the first and third regions of the one of the partitioned storage units within memory and output audio samples.
In one example, this disclosure provides an apparatus comprising a processor that converts a MIDI event into MIDI parameters, a hardware unit that uses the MIDI parameters to generate audio samples, and a plurality of storage units that store the MIDI parameters, wherein the storage units are partitioned into at least three regions, wherein a first region is accessible by both the processor and the hardware unit, a second region is accessible by the processor and inaccessible by the hardware unit, and a third region is accessible by the hardware unit and inaccessible by the processor after initialization.
In another example, this disclosure provides a method comprising generating MIDI parameters for a MIDI event via a processor, generating audio samples via a hardware unit that uses the MIDI parameters, storing MIDI parameters in a plurality of storage units, partitioning one of the storage units into at least three regions, accessing a first region of the MIDI parameters via both the hardware unit and the processor, accessing a second region of the MIDI parameters via the processor, and accessing a third region of the MIDI parameters via the hardware unit, and initialized by the processor.
In another example, this disclosure provides an apparatus comprising means for converting a MIDI event into MIDI parameters, means for generating audio samples based on the MIDI parameters, and means for storing the MIDI parameters, wherein the means for storing includes a plurality of storage units, wherein each of the storage units in the means for storing is partitioned into at least three regions, wherein a first region of each of the storage units is accessible by both the means for generating and the means for converting, a second region of each of the storage units is accessible by the means for converting and inaccessible by the means for generating, and a third region of each of the storage units is accessible by the means for generating and inaccessible by the means for converting after initialization.
In another example, this disclosure provides a computer-readable medium that stores MIDI parameters, the computer readable medium comprising a first region including first MIDI parameters accessible by a hardware unit and a processor, a second region including second MIDI parameters accessible by the processor, and a third region including third MIDI parameters accessible by the hardware unit and initialized by the processor.
In another example, this disclosure provides a computer-readable medium comprising instructions that upon execution generate MIDI parameters for a MIDI event via a processor, generate audio samples via a hardware unit that uses the MIDI parameters, store MIDI parameters in a plurality of storage units, partition one of the storage units into at least three regions, access a first region of the MIDI parameters via both the hardware unit and the processor, access a second region of the MIDI parameters via the processor, and access a third region of the MIDI parameters via the hardware unit, and initialized by the processor.
In another example, this disclosure provides a circuit adapted to generate MIDI parameters for a MIDI event via a processor, generate audio samples via a hardware unit that uses the MIDI parameters, store MIDI parameters in a plurality of storage units, partition one of the storage units into at least three regions, access a first region of the MIDI parameters via both the hardware unit and the processor, access a second region of the MIDI parameters via the processor, and access a third region of the MIDI parameters via the hardware unit, and initialized by the processor.
The details of one or more examples are set forth in the accompanying drawings and the description below. Other features, objects, and advantages will be apparent from the description and drawings, and from the claims.
This disclosure describes techniques for processing audio files that comply with a musical instrument digital interface (MIDI) format. As used herein, the term MIDI file refers to any audio data or file that contains at least one audio track that conforms to the MIDI format. Examples of various file formats that may include MIDI tracks include CMX, SMAF, XMF, SP-MIDI to name a few. CMX stands for Compact Media Extensions, developed by Qualcomm Inc. SMAF stands for the Synthetic Music Mobile Application Format, developed by Yamaha Corp. XMF stands for eXtensible Music Format, and SP-MIDI stands for Scalable Polyphony MIDI. As described in greater detail below, this disclosure provides techniques for storing, accessing, and processing various MIDI events of a MIDI file.
A general processor may execute software to parse MIDI files and schedule MIDI events associated with the MIDI files. The general processor dispatches the MIDI events to a second processor, which may be a digital signal processor (DSP), in a time-synchronized manner, and the DSP processes the MIDI events according to the time-synchronized schedule in order to generate MIDI parameters. The DSP then stores the MIDI parameters in memory. The memory comprises a plurality of storage units, which are accessible by the DSP and a hardware unit.
The DSP may separate and store the MIDI parameters needed by a hardware unit to generate audio samples, and may store such MIDI parameters in a distinct region within a storage unit. The DSP may store the remaining MIDI parameters in a different region within the storage unit. The hardware unit may use the MIDI parameters stored in the storage unit to generate audio samples. The DSP may schedule the processing of hardware unit to generate audio samples. The generated audio samples are converted into analog signals, which can be used to drive speakers and output audio sounds to a user. In this way, the MIDI parameters are separated and stored in distinct parts for more efficient access.
The various components illustrated in
As illustrated in the example of
Device 4 may implement an architecture that separates MIDI processing tasks between software, hardware, and firmware. Processor 8 may comprise a general purpose processor that executes software to parse MIDI files and schedule MIDI events associated with the MIDI files. The scheduled events can be dispatched to a second general processor, which may be a digital signal processor (DSP) 10 in one example of device 4, in a time-synchronized manner and thereby serviced by DSP 10 in a synchronized manner, as specified by timing parameters in the MIDI files. DSP 10 processes the MIDI events according to the time-synchronized schedule created by processor 8 in order to generate MIDI parameters.
Device 4 may also implement an architecture in which the functionality ascribed to processor 8 and DSP 10 is combined into one processor, such as a multi-threaded DSP. In such an exemplary device, a first thread of the multi-threaded DSP may execute software to parse MIDI files and schedule MIDI events associated with the MIDI files. A second thread of the multi-threaded DSP may process the MIDI events according to the time-synchronized schedule created by the first thread of the multi-thread DSP. The first thread of the multi-threaded DSP may perform similarly to processor 8 as described herein. The second thread of the multi-threaded DSP may perform similarly to DSP 10 as described herein.
DSP 10 may also schedule subsequent processing of the MIDI synthesis parameters by MIDI hardware unit 12. MIDI hardware unit 12 generates audio samples based on the synthesis parameters. The second processor may be any type of processor capable of processing signals, and for illustration purposes, in one aspect of this disclosure, is DSP 10.
Processor 8 may comprise any of a wide variety of general purpose single- or multi-chip microprocessors. Processor 8 may implement a CISC (Complex instruction Set Computer) design or a RISC (Reduced Instruction Set Computer) design. Generally, processor 8 comprises a central processing unit (CPU) that executes software. Examples include 16-bit, 32-bit or 64-bit microprocessors from companies such as Intel Corporation, Apple Computer, Inc, Sun Microsystems Inc., Advanced Micro Devices (AMD) Inc., and the like. Other examples include Unix- or Linux-based microprocessors from companies such as International Business Machines (IBM) Corporation, RedHat Inc., and the like. The general purpose processor may comprise the ARM9, which is commercially available from ARM Inc., and the DSP may comprise the QDSP4 DSP developed by Qualcomm Inc.
After processor 8 reads a MIDI event, DSP 10 may convert the MIDI event to a set of MIDI parameters. Based on the MIDI event, processor 8 schedules MIDI events for processing by DSP 10, and dispatches the MIDI events to DSP 10 according to this scheduling. In particular, this scheduling by processor 8 may include synchronization of timing associated with MIDI events, which can be identified based on timing parameters specified in the MIDI files. MIDI instructions in the MIDI files may instruct a particular MIDI voice to start or stop. Other MIDI instructions may relate to aftertouch effects, breath control effects, program changes, pitch bend effects, control messages such as pan left of right, sustain pedal effects, main volume control, system messages such as timing parameters, MIDI control messages such as lighting effect cues, and/or other sound effects. After scheduling MIDI events, processor 8 may provide the scheduling to DSP 10 so that DSP 10 can process the events.
When DSP 10 receives scheduled MIDI events from processor 8, DSP 10 may process the MIDI events in order to generate MIDI parameters. The timing in which these MIDI events are serviced by DSP 10 is scheduled by processor 8, which creates efficiency by eliminating the need for DSP 10 to perform such scheduling tasks. Accordingly, DSP 10 can service the MIDI events for the first audio frame while processor 8 is scheduling MIDI events for the next audio frame. Audio frames may comprise blocks of time, e.g., 10 millisecond (ms) intervals, that may include several audio samples. The digital output, for example, may result in 480 samples per frame, which can be converted into an analog audio sample. Many events may correspond to one instance of time so that many notes or sounds can be included in one instance of time according to the MIDI format. Of course, the amount of time delegated to any audio frame, as well as the number of samples per frame may vary in different implementations.
After DSP 10 converts the MIDI event to a set of MIDI parameters, DSP 10 may transmit the parameters to memory 50 for data storing. Memory 50 may comprise multiple storage units within memory 50. Memory 50 may be any type of device capable of storing data. Memory 50 may also be any type of process for storing data, for example, using link lists or arrays to store data. For example, memory 50 may comprise storage units, which may be a registers, comprising memory locations configured to store a pointer pointing to each MIDI parameter value at a specific location.
Storage units within memory 50 may be partitioned into at least three regions. DSP 10 may divide the MIDI parameters into at least two sets and store each set into one of the three regions of storage units within memory 50. The storage units within memory 50 are described in detail below. Memory 50 may be integrated into one or more of the other devices in audio device 4, or may be a separate unit from MIDI hardware unit 12 and DSP 10.
DSP 10 may command MIDI hardware unit 12 to retrieve all the MIDI parameters stored in two of the regions of storage units within memory 50 for an individual MIDI frame in order to generate an audio sample. The data for one MIDI frame may be the MIDI parameters stored in every storage unit within memory 50. The audio samples may be a pulse-code modulation (PCM) signal. A PCM signal is a digital representation of an analog signal in which the analog signal is sampled at regular intervals. Each MIDI frame may correspond to approximately 10 milliseconds, or otherwise as specified in a header of the MIDI file. Upon receiving the MIDI parameters from the storage units within memory 50, DSP 10 may signal MIDI hardware unit 12 to process the parameters to generate the audio sample. MIDI hardware unit 12 may be any type of device capable of generating audio samples. MIDI hardware unit 12 may be integrated into one or more of the other devices in audio device 4, or may be a separate unit. Upon generating the audio samples, MIDI hardware unit 12 may output the audio samples to DSP 10 for any post processing.
The MIDI parameters stored in the storage units within memory 50 may be synthesis parameters and non-synthesis parameters. The synthesis parameters may be digital, and generally, the synthesis parameters define the wave shape of the analog audio sound. Synthesis parameters are all the necessary parameters to generate an audio sample. Some, but not all, synthesis parameters are modulation frequency, vibrato frequency, filter cut-off frequency, filter resonance, pitch envelope, and/or volume envelope. Table 1 is an exemplary list of synthesis parameters.
Non-synthesis parameters are those event parameters that are not used to generate an audio sample. In particular, non-synthesis parameters are parameters necessary for the general functionality of MIDI hardware unit 12, DSP 10, or memory 50, but do not define the wave shape. For example, non-synthesis parameters are usually the parameters that define the playing of the audio samples generated the MIDI hardware unit 12. Some, but not all, examples of non-synthesis parameters include voice number, voice time, voice program number, voice channel number, and/or voice key number. Non-synthesis parameters may be more than just parameters that define playing of audio samples. They may be any MIDI parameter necessary for the general functionality of the audio device 4. Table 2 is an exemplary list of non-synthesis parameters.
Tables 1 and 2 are exemplary synthesis and non-synthesis parameters. In accordance with this disclosure, MIDI parameters are grouped and stored based on their need to be accessed by DSP 10 and hardware unit 12. Synthesis and non-synthesis parameters that need to be accessed by both DSP 10 and hardware unit 12 may be stored in the first region of storage units within memory 50. Synthesis and non-synthesis parameters that need to be accessed only by DSP 10 may be stored in the second region of storage units within memory 50. Synthesis and non-synthesis parameters that need to be accessed only by hardware unit 12 may be stored in the third region of storage units within memory 50. Thus, the regions of storage units within memory 50 may store both synthesis and non-synthesis parameters.
MIDI hardware unit 12 may use the synthesis parameters in generating audio samples. After generation of the audio sample, MIDI hardware unit 12 may output the audio sample in 10 millisecond frames when signaled by DSP 10. MIDI hardware unit 12 may process the parameters at 48 kilohertz, but the process rate may vary in different implementations. The process of generating the audio samples inside MIDI hardware unit 12 is well known in the art.
After generating the audio samples, MIDI hardware unit 12 may send an interrupt to DSP 10 signaling completion of generating audio samples. MIDI hardware unit 12 may then output the audio samples to DSP 10 for any post processing. After DSP 10 post processes the audio samples, DSP 10 may output this audio sample to a Digital-to-Analog converter (DAC) 14. DAC 14 converts the audio samples into an analog signal and outputs the analog signal to a driver circuit 16. Drive circuit 16 may amplify the signal to drive one or more speakers 19A and 19B to create audible sound.
Storage units 18 may be any type of device capable of storing data. For example, storage units 18 may be registers comprising memory locations configured to store a pointer pointing to each MIDI parameter value at a specific location. Each of storage units 18A-18N may be partitioned into at least three different regions, a first region 20A through first region 20N, a second region 22A through second region 22N, and third region 24A through 24N, respectively. Therefore, storage unit 18A comprises first region 20A, second region 22A, and third region 24A, and storage unit 18N comprises first region 20N, second region 22N, and third region 24N. First regions 20A-20N are collectively referred to as “first regions 20.” Second regions 22A-22N are collectively referred to as “second regions 22.” Third regions 24A-24N are collectively referred to as “third regions 24.” Each region may be capable of storing both synthesis and non-synthesis parameters. Of course, there could be more than three regions within storage units 18 in a system.
In one example, first regions 20 of storage units 18 may be accessible by both DSP 10 and MIDI hardware unit 12. Second regions 22 of storage units 18 may be accessible by only DSP 10. Third regions 24 of storage units 18 may be accessible by MIDI hardware unit 12, and initialized by DSP 10. After initialization by DSP 10, third regions 24 may be inaccessible by DSP 10. That is, third regions 24 may be accessible by MIDI hardware unit 12 after third regions 24 are initialized by DSP 10, but is inaccessible to DSP 10 following initialization. “Accessible” is herein defined as being readable and writeable, while “initialize” is herein defined as to set the initial values by DSP 10. The initial values may be zero. Thus, first regions 20 may be readable and writeable by both DSP 10 and MIDI hardware unit 12. Second regions 22 may be readable and writeable by only DSP 10. And third regions 24 may be readable and writeable by MIDI hardware unit 12. DSP 10 can only write to third regions 24 when setting the initial value. After setting the initial value, DSP 10 cannot write to third regions 24, until a MIDI event specifies to initialize one of third regions 24. When the MIDI event specifies to initialize third regions 24, DSP 10 may only write the initial value to third regions 24. Between initializing MIDI events, DSP 10 may not write to third regions 24. DSP 10 may never read from third regions 24.
Storage units 18A-18N may be independent from each other. In that, one of storage units 18 may be initialized but not contain MIDI parameters, a second one of storage units 18 may already contain MIDI parameters, while a third one of storage units 18 may not be initialized. Therefore, first regions 20A-20N, second regions 22A-22N, and third regions 24A-24N may also be independent from each other.
Table 3 is an exemplary list of MIDI parameters that may be stored in first regions 20. The MIDI parameters in table 3 may be accessible by both DSP 10 and hardware unit 12. The list of MIDI parameters shown in Table 3 is merely exemplary.
Table 4 is an exemplary list of MIDI parameters that may be stored in second regions 22. The MIDI parameters in table 4 may be accessible by only DSP 10. The list of MIDI parameters shown in Table 4 is merely exemplary.
Table 5 is an exemplary list of MIDI parameters that may be stored in third regions 24. The MIDI parameters in table 5 may be accessible only by hardware unit 12 and initialized by DSP 10. The list of MIDI parameters shown in Table 5 is merely exemplary.
DSP 10 may receive MIDI events and convert them into MIDI parameters. The MIDI parameters may be synthesis and non-synthesis parameters. DSP 10 may group the MIDI parameters into parameters accessible only by DSP 10, and parameters accessible by both DSP 10 and MIDI hardware unit 12. DSP 10 may output the MIDI parameters accessible by both DSP 10 and MIDI hardware unit 12 to one or more of first regions 20. DSP 10 may output the MIDI parameters accessible only by DSP 10 to one or more of second regions 22. In the event of a note-on or new voice, DSP 10 may initialize one or more of third regions 24. In the event of an existing voice, DSP 10 may not initialize one or more of third regions 24, and the MIDI parameters stored in one or more of third regions 24 may be updated by MIDI hardware unit 12. The phrase “Note-on” is used herein to refer to a MIDI event that is the first instance of a note when no other note is being processed in the frame. The term “New voice” is used herein to refer to a MIDI event that defines a first instance of a note when at least one other note is being processed in the frame. The term “Existing voice” is used herein to refer to a MIDI event where there is at least one note being processed throughout the entire frame, and no new note is processed in the frame. The term “note” may refer to any set of MIDI parameters requiring processing such as, but not limited to, a musical note.
In a further example, DSP 10 may signal MIDI hardware unit 12 to process the data to generate audio samples. MIDI hardware unit 12 may need to access the synthesis parameters of one or more first regions 20 and third regions 24. After generating audio samples, MIDI hardware unit 12 may need to store some MIDI parameters for the next frame. These MIDI parameters may be stored in one or more of third regions 24. The functionality of MIDI hardware unit 12 is described in detail below.
In addition, MIDI hardware unit 12 may include a coordination module 32. Coordination module 32 coordinates data flows within MIDI hardware unit 12. When MIDI hardware unit 12 receives an instruction from DSP 10 (
At the direction of coordination module 32, synthesis parameters may be loaded from memory 50 (
The instructions loaded into program RAM unit 44A or 44N instruct the associated processing element 34A or 34N to synthesize one of the voices indicated in the list of synthesis parameters in VPS RAM unit 46A or 46N. There may be any number of processing elements 34A-34N (collectively “processing elements 34”), and each may comprise one or more ALUs that are capable of performing mathematical operations, as well as one or more units for reading and writing data. Only two processing elements 34A and 34N are illustrated for simplicity, but many more may be included in hardware unit 20. Processing elements 34 may synthesize voices in parallel with one another. In particular, the plurality of different processing elements 34 work in parallel to process different synthesis parameters. In this manner, a plurality of processing elements 34 within MIDI hardware unit 12 can accelerate and possibly improve the generation of audio samples.
When coordination module 32 instructs one of processing elements 34 to synthesize a voice, the respective processing element may execute one or more instructions associated with the synthesis parameters. Again, these instructions may be loaded into program RAM unit 44A or 44N. The instructions loaded into program RAM unit 44A or 44N cause the respective one of processing elements 34 to perform voice synthesis. For example, processing elements 34 may send requests to a waveform fetch unit (WFU) 36 for a waveform specified in the synthesis parameters. Each of processing elements 34 may use WFU 36. An arbitration scheme may be used to resolve any conflicts if two or more processing elements 34 request use of WFU 36 at the same time.
In response to a request from one of processing elements 34, WFU 36 returns one or more waveform samples to the requesting processing element. However, because a wave can be phase shifted within a sample, e.g., by up to one cycle of the wave, WFU 36 may return two samples in order to compensate for the phase shifting using interpolation. Furthermore, because a stereo signal may include two separate waves for the two stereophonic channels, WFU 36 may return separate samples for different channels, e.g., resulting in up to four separate samples for stereo output.
After WFU 36 returns audio samples to one of processing elements 34, the respective processing element may execute additional program instructions based on the synthesis parameters. In particular, instructions cause one of processing elements 34 to request an asymmetric triangular wave from a low frequency oscillator (LFO) 38 in MIDI hardware unit 12. By multiplying a waveform returned by WFU 36 with a triangular wave returned by LFO 38, the respective processing element may manipulate various sonic characteristics of the waveform to achieve a desired audio affect. For example, multiplying a waveform by a triangular wave may result in a waveform that sounds more like a desired musical instrument.
Other instructions executed based on the synthesis parameters may cause a respective one of processing elements 34 to loop the waveform a specific number of times, adjust the amplitude of the waveform, add reverberation, add a vibrato effect, or cause other effects. In this way, processing elements 34 can calculate a waveform for a voice that lasts one MIDI frame. Eventually, a respective processing element may encounter an exit instruction. When one of processing elements 34 encounters an exit instruction, that processing element signals the end of voice synthesis to coordination module 32. The calculated voice waveform can be provided to summing buffer 40 at the direction of another store instruction during the execution of the program instructions. This causes summing buffer 40 to store that calculated voice waveform.
When summing buffer 40 receives a calculated waveform from one of processing elements 34, summing buffer 40 adds the calculated waveform to the proper instance of time associated with an overall waveform for a MIDI frame. Thus, summing buffer 40 combines output of the plurality of processing elements 34. For example, summing buffer 40 may initially store a flat wave (i.e., a wave where all digital samples are zero.) When summing buffer 40 receives audio information such as a calculated waveform from one of processing elements 34, summing buffer 40 can add each digital sample of the calculated waveform to respective samples of the waveform stored in summing buffer 40. In this way, summing buffer 40 accumulates and stores an overall digital representation of a waveform for a full audio frame.
Summing buffer 40 essentially sums different audio information from different ones of processing elements 34. The different audio information is indicative of different instances of time associated with different generated voices. In this manner, summing buffer 40 creates audio samples representative of an overall audio compilation within a given audio frame.
Processing elements 34 may operate in parallel with one another, yet independently. That is to say, each of processing elements 34 may process a synthesis parameter, and then move on to the next synthesis parameter once the audio information generated for the first synthesis parameter is added to summing buffer 40. Thus, each of processing elements 34 performs its processing tasks for one synthesis parameter independently of the other processing elements 34, and when the processing for synthesis parameter is complete that respective processing element becomes immediately available for subsequent processing of another synthesis parameter.
Eventually, coordination module 32 may determine that processing elements 34 have completed synthesizing all of the voices required for the current audio frame and have provided those voices to summing buffer 40. At this point, summing buffer 40 contains digital samples indicative of a completed waveform for the current audio frame. When coordination module 32 makes this determination, coordination module 32 sends an interrupt to DSP 10 (
Cache memory 48, WFU/LFO memory 39 and linked list memory 42 are also shown in
In accordance with this disclosure, any number of processing elements 34 may be included in MIDI hardware unit 12 provided that a plurality of processing elements 34 operate simultaneously with respect to different synthesis parameters stored in memory 50 (
Processing elements 34 may process all of the synthesis parameters for an audio frame. After processing each respective synthesis parameter, the respective one of processing elements 34 adds its processed audio information in to the accumulation in summing buffer 40, and then moves on to the next synthesis parameter. In this way, processing elements 34 work collectively to process all of the synthesis parameters generated for one or more audio files of an audio frame. Then, after the audio frame is processed and the samples in summing buffer are sent to DSP 10 for post processing, processing elements 34 can begin processing the synthesis parameters for the audio files of the next audio frame.
Again, first audio processing element 34A processes a first audio synthesis parameter to generate first audio information while a second audio processing element 34N processes a second audio synthesis parameter to generate second audio information. At this point, first processing element 34A may process a third audio synthesis parameter to generate third audio information while a second audio processing element 34N processes a fourth audio synthesis parameter to generate fourth audio information. Summing buffer 40 can combine the first, second, third and fourth audio information in the creation of one or more audio samples.
On the other hand, if DSP 10 determines that the MIDI event is not an instruction to update a parameter of a MIDI voice (“NO” of 54), DSP 10 may determine whether MIDI hardware unit 12 is idle (58). MIDI hardware unit 12 may be idle before generating a digital waveform for a first MIDI frame of a MIDI file or after completing the generation of a digital waveform for a MIDI frame. If MIDI hardware unit 12 is not idle (“NO” of 58), DSP 10 may wait one or more clock cycles and then again determine whether MIDI hardware unit 12 is idle (58).
If MIDI hardware unit 12 is idle (“YES” of 58), DSP 10 may load a set of instructions into program RAM units 44 in MIDI hardware unit 12 (60). The instructions may be loaded from one of storage units 18 within memory 50. For example, DSP 10 may determine whether instructions have already been loaded into program RAM units 44. If instructions have not already been loaded into program RAM units 44, DSP 10 may transfer such instructions into program RAM units 44 using direct memory exchange. Alternatively, if instructions have already been loaded into program RAM units 44, DSP 10 may skip this step.
After DSP 10 has loaded the program instructions into program RAM units 44, DSP 10 may activate MIDI hardware unit 12 (62). For example, DSP 10 may activate MIDI hardware unit 12 by updating a register in MIDI hardware unit 12 or by sending a control signal to MIDI hardware unit 12. After activating MIDI hardware unit 12, DSP 10 may wait until DSP 10 receives an interrupt from MIDI hardware unit 12 (64). While waiting for the interrupt, DSP 10 may process and output a digital waveform for a previous MIDI frame to DAC 14. Upon receiving the interrupt, an interrupt service register in DSP 10 may set up a direct memory exchange request to transfer the digital waveform for a MIDI frame from summing buffer 40 in MIDI hardware unit 12 (66). In order to avoid long periods of hardware idling when the digital waveform in summing buffer 40 is being transferred, the direct memory exchange request may transfer the digital waveform from summing buffer 40 in thirty-two 32-bit word blocks. The data integrity of the digital waveform may be maintained by a locking mechanism in summing buffer 40 that prevents processing elements 34 from over-writing data in summing buffer 40. Because this locking mechanism may be released block-by-block, the direct memory exchange transfer may proceed in parallel to hardware execution. DSP 10 may perform any necessary post processing and output the data to DAC 14 (70).
After DAC 14 converts the digital waveform into an analog audio signal, DAC 14 may provide the analog audio signal to drive circuit 16 (102). Drive circuit 16 may use the analog signal to drive speakers 19 (104). Speakers 19 may be electromechanical transducers that converts the electrical analog signal into physical sound. When speakers 19 produce the sound, a user of audio device 4 may hear the sound and respond appropriately. For example, if audio device 4 is a mobile telephone, the user may answer a phone call when speakers 19 produce a ring tone sound.
In a further example of operation of audio device 4, DSP 10 may receive MIDI events and generate MIDI parameters in 10 millisecond frames, or otherwise as specified in a header of the MIDI event. MIDI hardware unit 12 may generate audio samples in 10 millisecond frames, or otherwise as specified in a header of a MIDI event. MIDI hardware unit 12 may generate audio samples at 48 kilohertz, but the process rate may be different in different implementations.
If the MIDI event is not a note-on (“NO” of 108), then if the MIDI event contains a new voice (“YES” of 112), then DSP 10 may update one of first regions 20 and second regions 22, and initialize one of third regions 24 (114) within one of storage units 18. DSP 10 may signal MIDI hardware unit 12 to generate audio samples (116). DSP 10 may schedule the processing of MIDI hardware unit 12 to generate audio samples, and may perform any post processing. MIDI hardware unit 12 may generate audio samples based on the MIDI parameters stored in one of first regions 20 and third regions 24 (118) within one of storage units 18. MIDI hardware unit 12 may update one of first regions 20 and third regions 24 (120) within one of storage units 18. DSP 10 then waits for a new MIDI event (106).
If the MIDI event does not contain a new voice (“NO” of 112), then the MIDI event may be an existing voice (124). DSP 10 may update one of first regions 20 and second regions 22 (126) within one of storage units 18. DSP 10 may signal MIDI hardware unit 12 to generate audio samples (128). MIDI hardware unit 12 may generate audio samples based on the MIDI parameters stored in one of first regions 20 and third regions 26 (130) within one of storage units 18. MIDI hardware unit 12 may update one of first regions 20 and third regions 24 (132) within one of storage units 18. DSP 10 may then wait for a new MIDI event (106).
In some examples, the techniques of this disclosure may be embodied on a computer-readable medium that stores data as described herein. In this case, this disclosure may be directed to a computer readable medium that stores Musical Instrument Digital Interface (MIDI) parameters, the computer readable medium comprising, a first region including first MIDI parameters accessible by a hardware unit and a processor, a second region including second MIDI parameters accessible by the processor, and a third region including third MIDI parameters accessible by the hardware unit and initialized by the processor.
Computer-readable medium includes computer storage media. A storage media may be any available media that can be accessed by a computer. By way of example, and not limitation, such computer-readable media may comprise volatile memory such as FLASH memory or various forms of random access memory (RAM) including dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), static random access memory (SRAM). Computer-readable media may also comprise a combination of volatile and non-volatile memory, where the computer may read from the non-volatile memory and read from and write to the volatile memory.
Some examples described in the disclosure may be used in devices such as cell phones to generate ringtones. There may be multiple other devices that may implement techniques described in this disclosure such devices may be a network telephone, a digital music player, a music synthesizer, a wireless mobile device, a direct two-way communication device (sometimes called a walkie-talkie), a personal computer, a desktop or laptop computer, a workstation, a satellite radio device, an intercom device, a radio broadcasting device, a hand-held gaming device, a circuit board installed in a device, a kiosk device, a video game console, various computerized toys for children, an on-board computer used in an automobile, watercraft or aircraft, or a wide variety of other devices.
Various examples have been described in the disclosure. The various systems, as described above, may reduce overall memory accesses and data bandwidth in a system while maintaining data integrity of each voice. The systems described above are more efficient because the MIDI system is not making copies of the MIDI parameters and instead allocating only one common memory for all the MIDI parameters. Further, the systems described above are more efficient because they generate audio samples by only accessing a few set of the MIDI parameters, instead of all the MIDI parameters. Also, some of the data needed for a subsequent frame is already stored, instead of being generated for every frame. The systems described above also increase efficiency by splitting the processing and generating steps for creating audio samples.
One or more aspects of the techniques described herein may be implemented in hardware, software, firmware, or combinations thereof. Any features described as modules or components may be implemented together in an integrated logic device or separately as discrete but interoperable logic devices. If implemented in software, one or more aspects of the techniques may be realized at least in part by a computer-readable medium comprising instructions that, when executed, performs one or more of the methods described above. Also, this disclosure contemplates a computer-readable medium that is partitioned in the manner described herein. The computer-readable data storage medium may form part of a computer program product, which may include packaging materials. The computer-readable medium may comprise random access memory (RAM) such as synchronous dynamic random access memory (SDRAM), read-only memory (ROM), non-volatile random access memory (NVRAM), electrically erasable programmable read-only memory (EEPROM), FLASH memory, magnetic or optical data storage media, and the like. The techniques additionally, or alternatively, may be realized at least in part by a computer-readable communication medium that carries or communicates code in the form of instructions or data structures and that can be accessed, read, and/or executed by a computer.
Code may be executed by one or more processors, such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry. Accordingly, the term “processor,” as used herein may refer to any of the foregoing structure or any other structure suitable for implementation of the techniques described herein. In addition, in some aspects, the functionality described herein may be provided within dedicated software modules or hardware modules configured or adapted to perform the techniques of this disclosure.
If implemented in hardware, one or more aspects of this disclosure may be directed to a circuit, such as an integrated circuit, chipset, ASIC, FPGA, logic, or various combinations thereof configured or adapted to perform one or more of the techniques described herein. The circuit may include both the processor and one or more hardware units, as described herein, in an integrated circuit or chipset.
It should also be noted that a person having ordinary skill in the art will recognize that a circuit may implement some or all of the functions described above. There may be one circuit that implements all the functions, or there may also be multiple sections of a circuit that implement the functions. With current mobile platform technologies, an integrated circuit may comprise at least one DSP, and at least one Advanced Reduced Instruction Set Computer (RISC) Machine (ARM) processor to control and/or communicate to DSP or DSPs. Furthermore, a circuit may be designed or implemented in several sections, and in some cases, sections may be re-used to perform the different functions described in this disclosure.
These and other examples are within the scope of the following claims.
The present application for patent claims priority to Provisional Application No. 60/896,404 entitled “MUSICAL INSTRUMENT DIGITAL INTERFACE PARAMETER STORAGE” filed Mar. 22, 2007, and assigned to the assignee hereof and hereby expressly incorporated by reference herein.
| Number | Date | Country | |
|---|---|---|---|
| 60896404 | Mar 2007 | US |
