This application is related to patent applications, application Ser. No. 10/184,609 filed Jun. 28, 2002 titled “INTERCONNECTION DEVICE WITH INTEGRATED STORAGE” now issued as U.S. Pat. No. 6,850,536 dated Jan. 12, 2005 and identified by, application Ser. No. 10/184,582 filed Jun. 28, 2002 titled “MEMORY INTERFACE WITH FRACTIONAL ADDRESSING” now issued as U.S. Pat. No. 6,799,261 dated Sep. 9, 2004 and identified by, application Ser. No. 10/184,583 filed Jun. 28, 2002 titled “RE-CONFIGURABLE STREAMING VECTOR PROCESSOR” and identified by, application Ser. No. 10/184,743 filed Jun. 28, 2002 titled “METHOD OF PROGRAMMING LINEAR GRAPHS FOR STREAMING VECTOR COMPUTATION” now issued as U.S. Pat. No. 6,934,938 dated Aug. 23, 2005 and identified by, which are filed on even day herewith and are hereby incorporated herein by reference.
This invention relates generally to the field of computer program schedulers. More particularly, this invention relates to a method and apparatus for efficient scheduling of program instructions for a streaming vector processor.
A computer compiler is used to produce object code from a source code program. An instruction scheduler is a portion of a computer compiler that is used to establish the order in which at least some of the instructions will be executed. Most computers use named storage, such as registers, to store intermediate values. Part of the job of the scheduler is the allocation of the named storage. Another part of the job of the scheduler is to rearrange the order of program instructions to make efficient use of available hardware resources.
In vector processing, it is common for a set of operations to be repeated for each element of a vector or other data structure. This set of operations is often described by a data-flow graph. For example, a data-flow graph may be used to describe all of the operations to be performed on elements of the data structure for a single iteration of a program loop. It may be necessary to execute these operations number of times during the processing of an entire stream of data (as in audio or video processing for example). Considerable effort has been applied to the efficient scheduling of program loops. Software pipelining is an approach in which the execution of one iteration of a loop is started before the previous iteration has been completed. I.e. the computation of the iterations is overlapped. This may be facilitated, for example, by moving the entry point of a loop. If the entry point is moved, one or more iterations of one or more instructions of the loop are moved outside of the main loop. This generates a prolog (instructions before the main loop) or epilog (instructions after the main loop).
In another loop optimization method, redundant speculative computations in the compilation of an inner loop are eliminated. A number of loop transformations (based on the loop inductive variable) are used in order to eliminate the redundant computations.
List scheduling is a simple way to reorder instructions with a basic block. Instructions are sorted according to priority order (e.g. instructions with the longest period first, or instructions on the critical path first). Next, every instruction is checked with respect to its data dependencies. If all predecessors are already scheduled, the candidate instruction is assigned to a cycle. Resource conflicts are then taken into account and an offset is computed to delay issue of the instructions until all required functional units are guaranteed to be free when needed. This process is iterated until all instructions are scheduled.
The novel features believed characteristic of the invention are set forth in the appended claims. The invention itself, however, as well as the preferred mode of use, and further objects and advantages thereof, will best be understood by reference to the following detailed description of an illustrative embodiment when read in conjunction with the accompanying drawing(s), wherein:
While this invention is susceptible of embodiment in many different forms, there is shown in the drawings and will herein be described in detail one or more specific embodiments, with the understanding that the present disclosure is to be considered as exemplary of the principles of the invention and not intended to limit the invention to the specific embodiments shown and described. In the description below, like reference numerals are used to describe the same, similar or corresponding parts in the several Views of the drawings.
The present invention relates to a scheduler for efficient scheduling of program instructions for a streaming vector processor. The scheduler is designed to maximize throughput in a streaming vector processor with consideration for limited interconnection between functional units (processing units) and unnamed storage between units. The scheduler manages routing and storage of intermediate results to achieve pipelined schedules.
One of the uses of the present scheduler is to reduce the size of the interconnect structure. This presents the scheduler with several choices: (a) use specific named storage to hold intermediate values, (b) route them through trampoline nodes (which can be otherwise idle functional units) until the consumer node can accept them or (c) use unnamed link-based storage. In the preferred embodiment, the use of specific named storage is avoided because it constrains the achievable throughput and breaks any clean data flow. Using the trampoline nodes helps maintain the stream-like flow of data through the computation, improving overall throughput and efficiency.
The source code may be provided in the form of a data-flow graph (DFG) representation, as described in co-pending application ‘Method of Programming Linear Graphs for Streaming Vector Processing’. This representation is particularly useful for programming a vector processor of the type described in co-pending application titled ‘Re-configurable Streaming Vector Processor’ and referred to as an ‘RSVP’. Prior scheduling techniques may not be used for loop optimization, since the RSVP architecture and the DFG programming model prohibit the movement of loop entry points. Further, loops may not be optimized by eliminating loop inductive variables, since loop inductive variables are not available in the RSVP architecture and the DFG programming model and no speculative instructions are performed.
A flow chart of one embodiment of the method of the present invention is shown in
At block 106 the DFG nodes are scheduled starting with the loop-period that was computed in at block 104. The primary objective of the scheduler is to maximize throughput and the secondary one is to minimize latency. During the process, the loop-period might be increased in order to accommodate resource constraints, order dependencies and iteration dependencies. The result of this step is a list of nodes that are bound to appropriate resources and are assigned specific firing cycles and partitions (iterations). The scheduler splits the input DFG into a number of partitions that are executed in parallel. The number of partitions represents the number of iterations that are executed in parallel once a steady state has been achieved. These iterations may begin in different cycles. The data and order dependencies are checked at each step, however, an additional verification of the correctness of the node list may be made at block 108.
The connections between functional units in an RSVP are not hard-wired, so once a valid schedule has been produced (so far, the connections between functional units inferred by the scheduler are virtual), the scheduler has to allocate the existing links to connect the producer and consumer functional units. These links are allocated on a cycle-by-cycle basis subject to a maximum number of simultaneous links per cycle. For some degenerate graphs, registers need to be allocated in order to store intermediate results. At block 110 the scheduler moves the flexible nodes (nodes that have scheduling slack) so as to minimize the number of simultaneous links that need to be made in a cycle. This process helps to reduce the number of links required and the number of registers required for storing intermediate results. Nodes are moved between their ASAP (as soon as possible) and ALAP (as late as possible) time slots in order to minimize the number of registers required in any cycle. This step is motivated by a “zero registers” goal.
At block 112 the cycles in which the number of links required to route the information from producers to consumers is greater than the number of available links is determined. These cycles are referred to as “hot spots”. Block 112 produces a list of nodes in an order that minimizes the contention for links.
At block 114, the tail-times of the graph's nodes (in the order in which they are passed) are computed. The tail-time is the number of cycles for which an output result is held before being used by another functional unit. The order in which the tail-times are computed has a large influence on the quality of the resulting schedule. The best order of computing the tail-times of the nodes is derived in block 112.
If more than one node is mapped onto a function unit resource at a given clock cycle, then the nodes have to be allocated to slices. At block 116 any node that is mapped onto a partitioned resource is assigned to one or more fixed slices within the resources. Since this is a complex problem, an approximation to the optimal solution is found that affects the tail-times of a limited number of nodes only. Allocation of slices in partitioned functional units may result in the formation of an irregular scheduling. In the RSVP vector processor, invalid data is indicated by a special bit pattern called a “slug”. In contrast with traditional software pipelining techniques, the schedule may be dependent upon the iteration number. In order to assure a correct execution of such a schedule slugs may be used to discard unnecessary results.
The allocation of slices within the functional units may affect the number of links required in a cycle. At block 118 the cycles in which the number of links required to route the information from producers to consumers is greater than the number of available links is determined again.
At block 120, the lead-times of the graph's nodes (in the order in which they are passed) are computed. The lead-time is the number of cycles for which a data value is stored at the functional unit input before being used by it.
In this embodiment, the tail-time and lead-time calculations are performed sequentially since these two problems are not independent in the RSVP. However, for some hardware platforms the two problems are independent.
At block 122 the scheduler assigns edge storage to either a producer functional unit, a consumer functional unit, or both. The stage allocates the links between the scheduled nodes. The scheduler exploits a capability of the functional units, which is that functional units maintain their results until overwritten. In addition, the input operands can be placed in input latches anytime before the functional unit fires. This contrasts with other data-flow designs in which the functional unit fires as soon as the input operands have all arrived. The RSVP has a switch-based, incomplete interconnection network rather than a completely interconnected shared-bus network. This adds complexity to the scheduler, which must now schedule both the use of the interconnection network as well as the use of functional units. The scheduler takes account of limited interconnection between Function Units (FU), but the scheduler may also be used when the functional units are fully interconnected. Trampoline nodes or linked-based storage may be inserted when direct connections are not possible (as opposed to using special or named registers; this option would also break the flow of the graph).
The exact cycle when the intermediate result is handed from the producer functional unit to the consumer functional unit determines the interconnection firing. The edges are scheduled based on a force-directed approach that tries to schedule the read and write operations in a way that minimizes the number of required links per cycle. Merges (one push followed by multiple pops, i.e. one link routes same value to different places at different times) are also performed where this is applicable (i.e. edges that share the same source node). Trampoline nodes might also be inserted in order to forward information from one node to another. The edges of the input DFG may be time-multiplexed onto the limited physical resources of the interconnect network.
The critical path is subject to pseudo-dynamic control, so that when a trampoline node is inserted the critical path is recalculated because the addition might change it. Certain types of nodes might also be duplicated in order to minimize the number of registers required in any given cycle. Intermediate results may be stored at the input or output of a functional unit. The functional unit fires independently of the input arrivals (in contrast to the Transport Triggered Architecture where the FU fires whenever a move that has the “trigger” input involved is being initiated). The Functional Unit (FU) characteristics make for a very large search space. Intermediate results are preferably stored at the inputs/outputs of the functional units onto which the graph's nodes have been mapped. If this is not possible, the results are stored at the inputs/outputs of free FU. If there are not enough free FUs, the results are stored in registers. By using the above-mentioned storing order the number of interconnections is minimized, thereby minimizing the amount of interconnection fabric required (since there are fewer ports to be supported).
Since the scheduler minimizes the interconnections required, it may be used in the design of application specific integrated circuits (ASIC). An integrated circuit that is to be used for a specific computation is designed so that the hardware is optimized for that computation. The hardware includes a set of functional units and the interconnection fabric to interconnect the functional units. The computation may be scheduled for a number of different functional unit configurations. For each configuration, the minimum interconnection fabric is determined by scheduling the computation using the scheduler of the present invention. In this manner, the amount of interconnection fabric is minimized, thereby minimizing the cost and complexity of the ASIC.
At decision block 124 a check is made to determine if the scheduling process has been completed. If not, as indicated by the negative branch from decision block 124, flow continues to decision block 126. If the process is completed, as indicated by the positive branch from decision block 124, the correctness is optionally verified again at block 128 and the process terminates at block 130.
At decision block 126, a check is made to determine if all of the lead-time and tail-time of a node is required during the edge allocation phase. If less than the maximum time is required, as indicated by the negative branch from decision block 126, another round of trials is started from block 112. If all of the lead-time and tail-time of a node is required during the edge allocation phase, as indicated by the positive branch from decision block 126, the loop-period is increased by one at block 132. This allows more flexibility in the register minimization process (longer ASAP-ALAP intervals) and allows migrations between partitions.
The output from the scheduler is typically a list of microcode instructions or VLIW (very long instruction word) instructions. These instructions specify the operation of the functional units and the interconnections between functional units and link-based or register storage.
The scheduler produces schedules that are optimized primarily for throughput and secondarily for latency. A trade-off between these two goals can be achieved by specifying the current loop latency preference. Such a feature is useful when the DFG is executed only a small number of times. There is a break-even point beyond which a schedule that is optimized for throughput makes more sense than one that is optimized for latency. Having two schedules and allowing the system to load one of them based on the iteration count, which is determined at run-time, can improve the performance of the system. However, this is at the expense of an increase in memory required for storing both schedules. Two or more schedules may be combined or compressed in order to reduce storage space.
In prior software pipelining techniques, the goal is to minimize the initiation interval (at the possible expense of increasing either or both of a loop prolog, or loop epilog). These epilogs may be removed if predicated execution is supported or if additional hardware is added. The Re-configurable Streaming Vector Processor (RSVP) of co-pending application titled “Re-configurable Streaming Vector Processor”, application Ser. No. 10/184,583 filed Jun. 28, 2002 provides a different mechanism for handling pipelining. This avoids the need for extra code to deal with scenarios that often generate epilogs in prior approaches. For example, in prior techniques, unrolling loops to run two iterations concurrently requires a check for an odd number of iterations to be included as part of an epilog. This is not required in the scheduler of the present invention.
In prior software pipelining, all the iterations are scheduled in the same way (i.e., a node is scheduled onto the same resource, in the same slice irrespective of the iteration count). In the scheduler of the present invention, iterations can be scheduled differently. For example, alternate iterations may use different resources instances or different slices of the same resource.
Sometimes the scheduling of adjacent iterations can be packed closer together if the two use different resource instances, even though they may have the same functional unit orderings. In particular, throughput is improved if the choice of functional unit instance alternates between different iterations. This gives rise to a resource map for adjacent iterations that has a slightly different shape (in contrast to the software pipeline techniques where each iteration has exactly the same shape and unit instance allocations), and can be more tightly packed, resulting in a higher throughput.
An example RSVP routine for performing a quantization operation is
TABLE 1 shows how three iterations of the quantization loop can be scheduled. The asterisks denote invalid data, which may be indicated by one of more bits in the data representation. The prefixed letters denote the iteration so, for example, b3 is the third node of iteration b. In this example up to three iterations are being processed at one time (in cycle 6 for example). The data-flow graph has been separated into three partitions. No epilog or prolog code is required. The process is pipelined so that a new result is obtained every 3 cycles, whereas a single iteration requires 7 cycles.
If storage is available for intermediate results, the schedule can be made more efficient, as shown in TABLE 2. In TABLE 2, Store0 and Store1 denote storage of some kind. Now as many as five iterations are processed concurrently (as in cycles 8 and 9). Hence there are five partitions in this schedule. Two new results are produced every four cycles, so the addition of storage has improved throughput.
If the Adder can be configured as a trampoline node, it may be used as Store1.
Notice that all iterations are scheduled in the same way, using the same resources. This is not always the case.
A data graph for a further exemplary computation is shown in
Three iterations are overlapped in this example. In
The present invention, as described in embodiments herein, is implemented using a programmed processor executing programming instructions that are broadly described above in flow chart form that can be stored on any suitable electronic storage medium. However, those skilled in the art will appreciate that the processes described above can be implemented in any number of variations and in many suitable programming languages without departing from the present invention. For example, the order of certain operations carried out can often be varied, additional operations can be added or operations can be deleted without departing from the invention. Error trapping can be added and/or enhanced and variations can be made in user interface and information presentation without departing from the present invention. Such variations are contemplated and considered equivalent.
While the invention has been described in conjunction with specific embodiments, it is evident that many alternatives, modifications, permutations and variations will become apparent to those of ordinary skill in the art in light of the foregoing description. Accordingly, it is intended that the present invention embrace all such alternatives, modifications and variations as fall within the scope of the appended claims.
Number | Name | Date | Kind |
---|---|---|---|
3718912 | Hasbrouck et al. | Feb 1973 | A |
4128880 | Cray, Jr. | Dec 1978 | A |
4744043 | Kloker | May 1988 | A |
4760545 | Inagami et al. | Jul 1988 | A |
4807183 | Kung et al. | Feb 1989 | A |
4825361 | Omoda et al. | Apr 1989 | A |
4918600 | Harper et al. | Apr 1990 | A |
5206822 | Taylor | Apr 1993 | A |
5317734 | Gupta | May 1994 | A |
5367494 | Shebanow et al. | Nov 1994 | A |
5390352 | Kinoshita | Feb 1995 | A |
5418953 | Hunt et al. | May 1995 | A |
5423040 | Epstein et al. | Jun 1995 | A |
5450607 | Kowalczyk et al. | Sep 1995 | A |
5495617 | Yamada | Feb 1996 | A |
5652909 | Kodosky | Jul 1997 | A |
5697788 | Ohta | Dec 1997 | A |
5717947 | Gallup et al. | Feb 1998 | A |
5719998 | Ku et al. | Feb 1998 | A |
5734863 | Kodosky et al. | Mar 1998 | A |
5742821 | Prasanna | Apr 1998 | A |
5764787 | Nickerson | Jun 1998 | A |
5790877 | Nishiyama et al. | Aug 1998 | A |
5805614 | Norris | Sep 1998 | A |
5821934 | Kodosky et al. | Oct 1998 | A |
5826080 | Dworzecki | Oct 1998 | A |
5881257 | Glass et al. | Mar 1999 | A |
5881263 | York et al. | Mar 1999 | A |
5887183 | Agarwal et al. | Mar 1999 | A |
5893143 | Tanaka et al. | Apr 1999 | A |
5936953 | Simmons | Aug 1999 | A |
5966528 | Wilkinson et al. | Oct 1999 | A |
5969975 | Glass et al. | Oct 1999 | A |
5999736 | Gupta et al. | Dec 1999 | A |
6052766 | Betker et al. | Apr 2000 | A |
6064819 | Franssen et al. | May 2000 | A |
6104962 | Sastry | Aug 2000 | A |
6112023 | Dave et al. | Aug 2000 | A |
6128775 | Chow et al. | Oct 2000 | A |
6173389 | Pechanek et al. | Jan 2001 | B1 |
6192384 | Daily et al. | Feb 2001 | B1 |
6202130 | Scales et al. | Mar 2001 | B1 |
6253372 | Komatsu et al. | Jun 2001 | B1 |
6370560 | Robertazzi et al. | Apr 2002 | B1 |
6381687 | Sandstrom et al. | Apr 2002 | B1 |
6430671 | Smith | Aug 2002 | B1 |
6437804 | Ibe et al. | Aug 2002 | B1 |
6442701 | Hurd | Aug 2002 | B1 |
6490612 | Jones et al. | Dec 2002 | B1 |
6513107 | Ansari | Jan 2003 | B1 |
6571016 | Mehrotra et al. | May 2003 | B1 |
6588009 | Guffens et al. | Jul 2003 | B1 |
6598221 | Pegatoquet et al. | Jul 2003 | B1 |
6629123 | Hunt | Sep 2003 | B1 |
6647546 | Hinker et al. | Nov 2003 | B1 |
6665749 | Ansari | Dec 2003 | B1 |
6732354 | Ebeling et al. | May 2004 | B1 |
6745160 | Ashar et al. | Jun 2004 | B1 |
6792445 | Jones et al. | Sep 2004 | B1 |
6898691 | Blomgren et al. | May 2005 | B1 |
7000232 | Jones et al. | Feb 2006 | B1 |
7010788 | Rehg et al. | Mar 2006 | B1 |
20020080795 | Van Wageningen et al. | Jun 2002 | A1 |
20020112228 | Granston et al. | Aug 2002 | A1 |
20020120923 | Granston et al. | Aug 2002 | A1 |
20030128712 | Moriwaki et al. | Jul 2003 | A1 |
Number | Date | Country | |
---|---|---|---|
20040003220 A1 | Jan 2004 | US |