The present invention relates to a signal processing device, method, and storage medium used in electronic musical instruments, etc., for adding acoustic effects to generated musical waveform data, for example, and to an electronic musical instrument equipped with the signal processing device.
In a digital signal processor (DSP) for adding acoustic effects to musical waveform data, the musical waveform data stored in an external memory are sequentially read out in each sampling time/period determined by a predetermined sampling cycle, and are transferred to internal memory capable of high-speed processing within the DSP.
The DSP executes effect applying processing on the musical waveform data transferred to the internal memory within the sampling time/period.
An advantage of the present disclosure is in that it provides a signal processing device, method, program and an electronic musical instrument, which ensure consistency between burst access and internal memory access by the signal processing device even when burst access is used for data transfer between the internal memory and the external memory of the signal processing device.
Additional features and advantages of the invention will be set forth in the descriptions that follow and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings.
To achieve these and other advantages and in accordance with the purpose of the present invention, as embodied and broadly described, in one aspect, the present disclosure provides a signal processing device, comprising: a first memory storing waveform data; a second memory to which the waveform data is being transferred; and a processor configured to execute a first program that imparts a sound effect to samples of the waveform data that have been transferred to the second memory, and a second program that performs burst access to a plurality of samples of the waveform data stored in the first memory so as to transfer the plurality of samples of the waveform data to the second memory, wherein the processor determines an execution schedule of the second program such that a process execution of the first program and a process execution of the second program are respectively completed once in every sampling period of the waveform data.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory, and are intended to provide further explanation of the invention as claimed.
The DSP and an external memory are connected to a common bus together with the sound source circuit and the CPU that controls the overall operation of the electronic musical instrument. In such a signal processing device, the access time to the external memory is generally slower than the access time to the internal memory.
If the DSP frequently accesses the external memory to read or write the musical waveform data, the latency (delay) in accessing the external memory and competition with bus access by other circuits may cause a decrease in overall processing speed, and may add noise to the musical sound produced by the electronic musical instrument.
The access time to the external memory can be shortened by using burst access, which is a batch access method between multiple addresses of the second memory, which is the internal memory of the DSP, and multiple addresses of the first memory, which is the external memory.
However, it is difficult to design for ensuring consistency (mutual execution order) between access to the first memory, which is the external memory, and access to the second memory, which is the internal memory of the DPS.
Hereinafter, embodiments of the present disclosure are explained in detail with reference to the drawings.
The electronic musical instrument 100 includes a CPU (Central Processing Unit) 102, a ROM (Read Only Memory) 103, a RAM (Random Access Memory) 104, a sound source 105, a DSP 101, a memory interface circuit (MEMIF: MEMory Interface) 106 to which an external memory (DRAM: Dynamic RAM) 107, which is a first memory, is connected, a key scanner 120 to which a keyboard 122 for performance (performance operators) and a switch panel 123 are connected, and an LED controller 121 for causing each key of the keyboard 122 to emit light. 121, all of which are interconnected by an internal bus 130.
The CPU 102 controls the overall operation of the electronic musical instrument. The ROM 103 stores a control program, which is read out to the RAM 104 and executed by the CPU 102, and its working data. This control program detects the playing state of the keyboard 122 and various operating states of the switch panel 123 via the key scanner 120, and, based on these playing state and operating states, executes operations providing instructions for sound generation and sound muting to the sound source 105.
MEMIF 106 controls access to the external memory (DRAM) 107.
The sound source 105 is a circuit that generates musical waveform data based on instructions from the control program executed by the CPU 102, and may be called a wave generator WG. The generated musical waveform data is transferred to the D-RAM 112 as a second memory within the DSP 101 via the internal bus 130, MEMIF 106, and D-RAM (Dynamic-RAM) 112 by burst access, which will be described later.
DSP 101 includes SPU (Signal Processing Unit) 108, MP-RAM (Main Program-RAM) 109, TRAM (Transfer RAM) 110, EMI (External Memory Interface) 111, internal memory (D-RAM) 112, W-RAM (Work RAM) 113, and P-RAM (Parameter RAM) 114.
The SPU 108 is a processor that controls the overall operation of the DSP 101.
The MP-RAM 109 is a memory that stores the main program (first program) of the DSP 101 executed by the SPU 108. The main program (first program) is a program for adding sound effects to the waveform data samples transferred to the D-RAM 112 as a second memory.
The TRAM 110 is a memory that stores a subprogram (second program) of the DSP 101 executed by the SPU 108. The subprogram (second program) performs burst access to a plurality of samples of waveform data stored in the external memory (DRAM: Dynamic RAM) 107 as a first memory and transfers them to the D-RAM 112 as a second memory. In this embodiment, the sub-program (second program) and the main program (first program) are developed to respective memories as separate programs, so that the execution schedule of each program can be controlled adequately.
EMI 111 is an interface circuit that controls access to external memory (DRAM) 107.
The internal memory (D-RAM) 112 as the second memory is an internal memory of the DSP 101 and is used as a mirror memory of the external memory (DRAM) 107 as the first memory.
W-RAM 113 is a working memory used in signal processing executed by SPU 108.
P-RAM 114 is a memory that stores parameter data used in signal processing executed by SPU 108.
The main program stored in the MP-RAM 109, the sub-program stored in the TRAM 110, the burst number in burst access, which will be described later, the parameter data stored in the P-RAM 114, etc., can be set, changed, and re-set by the CPU 102 via the internal bus 130.
The musical waveform data to which the sound effects have been applied by the DSP 101 is output from the SPU 108 to an external sound system 115 consisting of a D/A converter, an amplifier, a speaker, etc., which are not shown, so as to be sounded.
By executing the main program stored in the MP-RAM 109, the SPU 108 uses the parameter data stored in the P-RAM 114, uses the W-RAM 113 as a working memory, and executes operations for applying sound effects on the musical waveform data prefetched (described later) from the external memory (DRAM) 107 to the D-RAM 112. Independently from the main program, the SPU 108 executes the subprogram stored in the TRAM 110 to prefetch (described later) the musical waveform data stored in the external memory (DRAM) 107 into the internal memory (DRAM) 112 within the DSP 101, which is capable of high-speed processing. The SPU 108 executes signal processing such as sound effect imparting processing on the musical waveform data that have been transferred to the internal memory (D-RAM) 112.
With the main program, the SPU 108 does not directly access the external memory (DRAM) 107 when executing effect processing (mainly delay processing) using the external memory (DRAM) 107. In this case, the EMI 111 first accesses the internal bus 130 based on a subprogram which consists of commands describing access addresses read from the TRAM 110 and processing contents (read/write, burst access/single access, etc., which will be described later). If the access destination is the external memory (DRAM) 107, the external memory (DRAM) 107 is accessed via the MEMIF 106. However, the access destination may be the internal memory (D-RAM) 112.
With the main program, the SPU 108 outputs the musical waveform data in the internal memory (D-RAM) 112 or W-RAM 113 that have undergone sound effect imparting processing to the external sound system 115 (not shown).
The DSP 101 executes the series of processes described above in real time as the sampling time/period progresses. In this case, the DSP 101 executes prefetch/poststore control processing with respect to the external memory (DRAM) 107. Prefetch/poststore is a function of mirroring an external memory, which has a slow access speed, to a high-speed internal memory (D-RAM) 112 in the DSP 101, so to speak.
The prefetching is an operation of preparing the musical waveform data in the internal memory (D-RAM) 112, which is a fast memory within the DSP 101, by executing a read command to the external memory (DRAM) 107 before the SPU 108 of the DSP 101 executes the read command to the internal memory (D-RAM) 112.
The post-store is an operation of writing the musical waveform data, which have been temporarily stored in the high-speed internal memory (D-RAM) 112 by a write command issued to the internal memory (D-RAM) 112 by the DSP 101, onto the external memory (DRAM) 107 at a later timing. The prefetch and poststore operations described above are executed at timings different from the read/write operations for the internal memory (D-RAM) 112 within the DSP 101. Therefore, the SPU 108 of the DSP 101 executes internal read/write processing for the internal memory and accompanying sound effect imparting processing (signal processing) by the main program stored in the MP-RAM 109. On the other hand, the SPU 108 executes prefetch/poststore processing by the subprogram stored in the TRAM 110 in the DSP 101 separately from the main program.
The subprogram that executes prefetch/poststore control does not run at the same timing as the main program. Therefore, if the execution timing of the subprogram is not set correctly with respect to the execution of the main program, the prefetch may not be in time for the execution of the main program, and the main program may be executed even though the poststore has not been executed. Thereby, the sample data of the musical waveform data in the internal memory (D-RAM) 112 to be written in the external memory (D-RAM) 107 may be overwritten, causing a mismatch to occur, and the DSP 101 does not operate properly.
In the DSP 101 that executes the processing of the main program in each sampling time/period, unless prefetch/poststore with respect to the external memory (DRAM) 107 are completed within the sampling period, all the accesses cannot be completed. In order to efficiently perform many memory accesses, it is necessary to shorten the access time. Therefore, in this embodiment, the external memory (DRAM) 107 is constituted of an SDRAM (Synchronous DRAM), DDR-SDRAM (Double Data Rate-SDRAM) or the like, and burst access is used as data access between this external memory (DRAM) 107 and internal memory (D-RAM) 112 in the DSP 101 in order to reduce the access time.
Burst access is a function that allows batch access between multiple addresses in the internal memory (D-RAM) 112 of the DSP 101 and multiple addresses in the external memory (DRAM) 107 by issuing one instruction. The number of batch accesses is called the burst number. For example, if the burst number, that is, the number of samples of musical waveform data to be accessed at one time, is four, consecutive four addresses can be accessed at once at high speed. Therefore, in simple terms, the subprogram can be considered to execute one unit of processing in four sampling periods.
However, in this embodiment, the execution schedule of the second program is determined so that the execution of the first program and the second program is completed once every sampling period of the waveform data.
Here, the meaning of one processing unit is, for example, in the case of access from the external memory (DRAM) 107 to the internal memory (D-RAM) 112 for prefetching only samples of musical waveform data in the next four sampling periods, a unit in which prefetch burst read access only for the four sampling periods is executed.
On the other hand, as in delay effect processing, for example, when post-store burst write access of samples of musical waveform data for the current four sampling times/periods from the internal memory (D-RAM) 112 to the external memory (DRAM) 107, and prefetch burst read access of the delayed musical waveform data for the next four sampling timesperiods from the external memory (DRAM) 107 to the internal memory (D-RAM) 112 are to be executed, a unit in which the above-described post-store burst write access and the subsequent prefetch burst read access are executed as a set is one processing unit.
The thick solid line 201 indicates the execution state of the main program, and indicates that the processing steps of the main program linearly increase as the elapsed execution time increases from time t1 when the number of processing steps of the main program is 0. At time t3 when one sampling period has elapsed from t1, the processing step of the main program reaches the maximum value Mainmaxstep, and signal processing for one unit is completed. Thereafter, the main program repeats the same signal processing for each sampling period, such as t3→t5, t5→t7, t7→t9, t9→t11, t11→t13, t13→t15.
The thin solid line 202 indicates the execution state of the subprogram. The subprogram starts processing at time t2, which is, for example, half the sampling period after time t1 when the main program indicated by the thick solid line 201 started executing. As the elapsed execution time increases from this time t2, the processing steps of the subprogram also increase linearly, but the processing step of the subprogram reaches the maximum value Submaxstep at time t10 which is 4 sampling periods since the time t2 which is 0 step. That is, the thin solid line 202 indicates that the subprogram that executes one processing unit of prefetch/poststore is executed once in every four sampling periods. After time t10, the subprogram repeatedly executes the same burst access process every four sampling periods.
Here, with respect to the execution of the subprogram, at time points t4, t6, and t8 of the respective points surrounded by circle marks 204a, 204b, and 204c indicated by execution elapsed times t4, t6, and t8, the thick solid line 201 of the main program and the thin solid line 202 of the subprogram intersect. Furthermore, the execution state of the subprogram indicated by the thin solid line 202 can fluctuate, for example, within the range of thin dashed lines 203a to 203b depending on the load state of the internal bus 130.
Therefore, as shown in
On the other hand, as shown in
As described with reference to
Therefore, in this embodiment, in order to prevent inconsistency in access order with access to the internal memory (D-RAM) 112 (second memory) by execution of the main program (first program) by the SPU 108 (processor) of the DSP 101, the execution schedule of bust access, which instructs simultaneous access between a plurality of addresses of the internal memory (D-RAM) 112 and a plurality of addresses of the external memory (DRAM) 107, is predetermined in relation to the sampling period based on the burst number, which indicates the number of simultaneous access of the bust access.
The SPU 108 executes burst access between the internal memory (D-RAM) 112 and the external memory (DRAM) 107 by executing the subprogram according to the determined execution schedule. Further, the SPU 108 repeatedly executes the main program at each sampling period to perform signal processing, such as applying sound effects and the like, on the musical waveform data that has been stored in the internal memory (D-RAM) 112 as acquired from the external memory (DRAM) 107. The SPU 108 outputs the resulting musical waveform data to the external sound system 115, for example. Alternatively, for example, the musical waveform data obtained in the delay effect processing is written in the internal memory (D-RAM) 112 in order to be delayed in the external memory (DRAM) 107.
For example, when the burst number 4, as shown in
Then, each of the divided subprograms described above is each executed at successive sampling periods the number of which corresponds to the burst number.
For example, as shown in
At this time, the start timing of each divided subprogram at each sampling period is determined to be a predetermined period of time after the main program performs a processing step that is the same as the processing step the corresponding divided subprogram is going to perform.
For example, in
In the first embodiment of the method for determining the execution schedule shown in
As a result, the first embodiment of the method for determining the execution schedule shown in
In this way, the execution schedule of the second program is determined such that the start timing of each of the second programs that are divided and to be executed into a number corresponding to the burst number is set to a prescribed timing after the start timing of each execution of the corresponding first program. Further, the execution schedule of the second program is determined so that execution of the second program is completed once in each sampling period for the waveform data.
For example, when the number of bursts is 4, in the subprograms 502 (#0), 502 (#1), 502 (#2), and 502 (#3) to be executed in the respective sampling periods T (#0), T (#1), T (#3), and T (#4) shown in
In this example, the sampling period is changed from the value “0” to “bust number−1] as in 0, 1, 2, 3, 0, 1, 2, 3, etc., corresponding to the burst number (burst number=4 in the example in
In the example of
Similarly, in subprogram 502 (#1) executed in sampling period T (#1) where the sampling counter value is 1, the execution status of processing steps 1, 5, 9, 13, etc., i.e., the processing steps in which the remainder value obtained by dividing by the number of bursts=4 is 1, is set to “◯” and burst access is executed, and the execution status of other processing steps is set to “●” and burst access is not executed, thereby skipping those processing steps. In this case, in the processing steps 1, 5, 9, 13, etc., of subprogram 502 (#1), in which the remainder value obtained by dividing by the number of bursts=4 is 1, instructions for access (burst access or single access described later) consistent with the main program 501 (#1) to be executed in the sampling period T (#1) are stored in advance.
Similarly, in the subprogram 502 (#2) executed in the sampling period T (#2) where the sampling counter value is 2, the execution status of the processing steps, 2, 6, 10, 14, etc., i.e., the processing steps whose remainder value obtained by dividing by the number of bursts=4, is 2 is set to “◯” and burst access is executed, and the execution status of other processing steps is set to “●” and burst access is not executed, thereby skipping those processing steps. In this case, in the processing steps 2, 6, 10, 14, etc., of subprogram 502(#2), in which the remainder value obtained by dividing by the number of bursts=4 is 2, instructions for access (burst access or single access to be described later) consistent with the main program 501 (#2) to be executed in the sampling period T (#2) are stored in advance.
Similarly, in the subprogram 502 (#3) executed in the sampling period T (#3) where the sampling counter value is 3, the execution status of the processing steps, 3, 7, 11, 15, etc., i.e., the processing steps whose remainder value obtained by dividing by the number of bursts=4 is 3, is set to “◯” and burst access is executed, and the execution status of other processing steps is set to “●” and burst access is not executed, thereby skipping those processing steps. In this case, in the processing steps 3, 7, 11, 15, etc., of subprogram 502(#3), in which the remainder value obtained by dividing by the number of bursts=4 is 3, instructions for access (burst access or single access to be described later) consistent with the main program 501 (#3) to be executed in the sampling period T (#3) are stored in advance.
In the second embodiment of
For example, in
In the second embodiment of the method for determining the execution schedule shown in
As a result, as in the case of the first embodiment shown in
First, as described above, the TRAM 110 in
The operation start timing generation circuit 601 generates a reset signal “reset” that instructs the operation start timing of each subprogram at each sampling period. Specifically, the operation start timing generation circuit 601 generates the reset signal “reset” that resets the sub program counter 602 once every sampling period using a master counter mc inputted from a circuit not shown in the drawings.
More specifically, the operation start timing generation circuit 601 asserts the reset signal “reset” that is input to the program counter 602 when the value of the master counter mc, which changes from value 0 to value 4095 for each sampling period, reaches 2047, which corresponds to ½ sampling period, for example. As a result, in the example of
The sub program counter 602 (program counter circuit) outputs TRAM address, which is a program counter value that counts up from 0 at predetermined time intervals obtained by subdividing the sampling period, at the timing when the reset signal “reset” from the operation start timing generation circuit 601 is asserted in every sampling period. If the bus I/F circuit 604 (described later) asserts a busy signal indicating that the internal bus 130 in
A DFF (D-Flip Flop) circuit 605 temporarily holds the TRAM address output from the sub program counter 602.
A lower 2-bit comparator 603 (lower bit comparison circuit) compares values of the lower bits corresponding to the burst number (which, in the example of
As described above in the explanation of
The TRAM 110 outputs instruction data of the processing step that corresponds to the TRAM address outputted by the Sub Program counter 602 to the bus I/F circuit 604 (bus interface circuit).
As shown in
The bus I/F circuit 604, outputs Read/Write data, Address data and the instruction data instructing burst access to the internal bus 130 of
The instruction data for each processing step of the subprogram is read out sequentially based on the TRAM address output from the Sub Program counter 602, and an actual burst access is performed only when the Access Enable signal, which is generated once in every 4 processing steps by comparing the lower two bits of the sampling counter sc with the TRAM address, is asserted.
Under the control of the bus I/F circuit 604 described above, as described above with reference to
Similarly, with respect to the instruction data for burst access for the subprogram 502 (#1) to be executed in the sampling period T (#1) having the sampling counter sc of 1, in processing steps 1, 5, 9, 13, etc., where the lower two bits of TRAM address are 1, actual burst access is performed because the Access Enable signal and the Burst Enable signal are asserted. The burst access instruction data of the other processing steps are skipped because the Access Enable signal is negated although the Burst Enable signal is asserted.
Similarly, with respect to the instruction data for burst access for the subprogram 502 (#2) to be executed in the sampling period T (#2) having the sampling counter sc of 2, in processing steps 2, 6, 10, 14, etc., where the lower two bits of TRAM address are 2, actual burst access is performed because the Access Enable signal and the Burst Enable signal are asserted. The burst access instruction data of the other processing steps are skipped because the Access Enable signal is negated although the Burst Enable signal is asserted.
Similarly, with respect to the instruction data for burst access for the subprogram 502 (#3) to be executed in the sampling period T (#3) having the sampling counter sc of 3, in processing steps 3, 7, 11, 15, etc., where the lower two bits of TRAM address are 1, actual burst access is performed because the Access Enable signal and the Burst Enable signal are asserted. The burst access instruction data of the other processing steps are skipped because the Access Enable signal is negated although the Burst Enable signal is asserted.
As described above in the explanation of
Specifically, in the subprogram stored from the CPU 102 to the TRAM 110, in the processing steps=0, 4, 8, 12, etc. where the lower two bits of the “TRAM address” item value (
Similarly, in the processing steps=1, 5, 9, 13, etc., where the lower two bits of the “TRAM address” item value are 1, the access instructions consistent with the main program 501 (#1) to be executed in sampling period T (#1) are stored.
Similarly, in the processing steps=2, 6, 10, 14, etc., where the lower two bits of the “TRAM address” item value are 2, the access instructions consistent with the main program 501 (#2) to be executed in sampling period T (#2) are stored.
Similarly, in the processing steps=3, 7, 11, 15, etc., where the lower two bits of the “TRAM address” item value are 3, the access instructions consistent with the main program 501 (#3) to be executed in sampling period T (#3) are stored.
Here, when the instruction data output from TRAM 110 instructs a single access to the external memory (DRAM) 107 or the internal memory (D-RAM) 112, and the Burst Enable signal output from TRAM 110 is therefore negated, the bus I/F circuit 604 in
As described above, the signals read/written by accessing the internal bus 130 by the bus I/F circuit 604 are stored in or read out from the high-speed internal memory (D-RAM) 112 within the DSP 101.
According to the embodiments described above, it becomes possible to ensure a sufficient difference in execution time between the subprogram and the main program in each sampling period, and the execution timings of the two do not intersect. Therefore, data consistency is maintained during the prefetch/poststore burst access. As a result, it becomes possible to reduce the load on the bus shared with the CPU 102 and the sound source 105 to which the external memory (DRAM) 107 is connected, and it becomes possible to reduce the influence upon the CPU 102 and the sound source 105 due to an increase in access to the external memory (DRAM) 107 by the DSP 101. As a result, the DSP 101 can increase variations in sound effect applications.
In addition to improving processing performance with burst access, it can also support signal processing that requires single access (for example, when the reference destination address is not sequentially incremented but discontinuous, such as in a chorus effector).
Although the embodiments described above disclose examples in which the DSP 101 is used to impart acoustic effects in an electronic musical instrument, the present invention is not limited thereto, and can be applied to various uses by linking the signal processing device with an external memory.
According to the present invention, burst access is used for data transfer between an internal memory and an external memory of a signal processing device to reduce the frequency of external memory access, reduce competitions for bus access, and to ensure consistency with memory access between burst access and access to the internal memory by the signal processing device.
It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention. Thus, it is intended that the present invention cover modifications and variations that come within the scope of the appended claims and their equivalents. In particular, it is explicitly contemplated that any part or whole of any two or more of the embodiments and their modifications described above can be combined and regarded within the scope of the present invention.
Number | Date | Country | Kind |
---|---|---|---|
2022-192118 | Nov 2022 | JP | national |