The present invention relates generally to processing systems and, in particular, to the automatic configuration of processing systems.
In processing devices with decoupled architectures, memory access and computation are performed by separate (decoupled) hardware modules. For general purpose computing, the hardware may be further decoupled by introducing a control processing module in addition to the memory access and computation modules.
Streaming applications produce interim data; data that is produced and consumed by pairs of hardware accelerators. Prior approaches use a memory mapped buffer as a peripheral to store interim data that is generic to all interim data, and therefore not efficient for any access pattern. Alternative approaches include the use of external memory (DRAM) to store interim data. All of these approaches require extra bus ports, which lead to lower bus speeds and larger gate counts.
In one prior design approach for devices with a decoupled architecture, a data-flow-graph (DFG) is used to define the computation and a set of stream descriptors are used to define data access patterns. This approach has the ability to generate hardware automatically from the DFG and stream descriptors. In addition, some efforts have been made to develop tools that allow programs developed in high level languages, such as C/C++, to be converted into hardware (for example by programming the gates of an FPGA). The generated hardware tends to be inefficient unless the high level language includes features, such as memory access threads and computation threads, with the flexibility to describe both the computational task and data movement. These features allow streaming data access to memory and/or other hardware accelerators in a computation pipeline.
The use of high level languages for hardware programming aids software engineers who do not have system architecture or hardware expertise but may be familiar with high level languages (HLL's) such as C/C++ that are used to program embedded systems with DSPs or microcontrollers.
Stream descriptors have been used to access data in memory as streams and have also been used to generate stream data interface logic. In contrast, interim data storage of data being moved between computational modules has been handled by memory mapped buffers and/or first in first out (FIFO) buffers.
The accompanying figures, in which like reference numerals refer to identical or functionally similar elements throughout the separate views and which together with the detailed description below are incorporated in and form part of the specification, serve to further illustrate various embodiments and to explain various principles and advantages all in accordance with the present invention.
Skilled artisans will appreciate that elements in the figures are illustrated for simplicity and clarity and have not necessarily been drawn to scale. For example, the elements in the figures may be simplified to aid understanding of embodiments of the present invention.
Before describing in detail embodiments that are in accordance with the present invention, it should be observed that the embodiments reside primarily in combinations of method steps and apparatus components related to configuration of buffers for streaming data transfer. Accordingly, the apparatus components and method steps have been represented where appropriate by conventional symbols in the drawings, showing only those specific details that are pertinent to understanding the embodiments of the present invention so as not to obscure the disclosure with details that will be readily apparent to those of ordinary skill in the art having the benefit of the description herein.
In this document, relational terms such as first and second, top and bottom, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element preceded by “comprises . . . a” does not, without more constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.
It will be appreciated that some of the embodiments of the invention described herein may comprise one or more conventional processors and unique stored program instructions that control the one or more processors to implement, in conjunction with certain non-processor circuits, some, most, or all of the functions for automatic configuration of buffers for streaming data transfer described herein. The non-processor circuits may include, for example, data access modules such as user input devices and peripherals. As such, these functions may be interpreted as a method to configure buffers for streaming data transfer.
Further, it is expected that one of ordinary skill, notwithstanding possibly significant effort and many design choices motivated by, for example, available time, current technology, and economic considerations, when guided by the concepts and principles disclosed herein will be readily capable of generating such software instructions and programs and ICs with minimal experimentation.
One aspect of the present invention relates to a method to configure buffers for interim data storage between data source and destinations for streaming data transfers based on stream descriptor parameters. A buffer, also referred to as an interim data storage device, may be configured manually by a user, automatically by a computer, or by a user interaction with a computer.
The processing system in
The SMIF's 108 and 110 may be implemented using a reconfigurable logic circuit. Stream descriptors contain data access patterns (parameters that describe the shape and location of stored data) which can be used to configure an efficient buffer. Reconfigurable platforms, such as field programmable gate arrays (FPGA's) allow configuration of memory hierarchy for different applications. Some of this memory may be configured a buffer 202. Stream descriptors can be used to define stream data transfers between computational modules, which may also be implemented using reconfigurable logic.
Alternatively, the SMIF's 108 and 110, buffer 202 and computational modules may be implemented using one or more custom logic circuits, such an application specific integrated circuit (ASIC) or structured ASIC, this is configured only once.
The stream memory interfaces 108 and 110 may also be referred to as stream access modules, stream device or stream data interfaces, for example. The buffer 202 is also referred to as an interim data storage device.
In accordance with an embodiment of a further aspect of the invention, a specification of a configurable processor is generated automatically by generating (1) a specification of a first stream memory interface operable to access data in accordance with a first stream descriptor, (2) a specification of a second stream memory interface operable to access data in accordance with a second stream descriptor, and (3) a specification of an interim data storage device to be accessed by the first stream memory interface and the second stream memory interface and to be operable to receive data from a first computational module via the first stream memory interface and to transfer data to a second computational module via the second stream memory interface. The specification of the configurable processor, which comprises the specifications of the first stream memory interface, the second stream memory interface and the interim data storage device, may be output to a configurable or reconfigurable processor, a communication channel or to a computer readable storage device.
One embodiment of the present invention relates to the configuration of processing hardware from a C/C++ language description of a process. The C/C++ language provides a multi-threaded framework. Computation and communication are decoupled explicitly using the ability of the C/C++ language to describe multiple program threads. Computation and memory access are defined in separate threads that facilitate scheduling of the process in the hardware. Computation threads are implemented by computational modules while data access threads are executed by stream memory interfaces (SMIF's). The computational threads are also referred to as computation kernels or functions. Buffers and associated stream memory interfaces are used to communicate among computational modules and data access modules. The communication among computation threads, referred to as data channels, may be implemented, for example, with direct connections, in addition to the buffers.
In prior approaches computation and memory access are interleaved within the same program thread. In such approaches, a compiler has the more difficult task find the parallelism in between computation and data transfers in order to overlap the operations. The memory access patterns are less efficient because they are inferred by the compiler and may not match the intent of the programmer. The compiler applies a series of code transformations to eliminate dependencies, and then generates sequences of load/store instructions based on new access patterns of the transformed loop. This means that data transfer depends on the access pattern inferred by the compiler from the loop structure. The use of stream descriptors in accordance with the present invention enables complex access patterns that are not easily discernible from nested loop structures. Stream descriptors also decouple memory address generation from the actual computation allowing grouped data elements to better match the underlying memory hierarchy.
In one embodiment of the invention, processing hardware is configured automatically for an application defined by a plurality of programming instructions of a high level language that include stream descriptors, descriptive of data access locations, data access thread definitions, and computation thread definitions. The automatic configuration is achieved by compiling the programming instructions using a first compiler to produce a description of data flow between the data access threads and the computational threads. A second compiler is then used to generate a hardware description and executable code dependent upon the description of the data flow. Finally, a programmer is used configure the processing hardware in accordance with the hardware description. In particular, the programmer configures stream memory interfaces operable to access data in accordance with the stream descriptors, computational modules operable to process data in accordance with the computation thread definitions, and at least one buffer (such as a tile buffer) that is operable to pass interim data between the computational modules via the stream memory interfaces, in accordance with the description of the data flow.
In prior approaches, interim buffers have been accessed via a bus structure rather than via stream memory interfaces.
In a further embodiment of the invention, a system for automatic configuration of processing hardware includes an application program interface (API) tool that includes a data access thread class, a computation thread class and a stream descriptor data type. The API tool is operable to enable a programmer to produce an application program that defines data access threads, computation threads, stream descriptors and data movement between the threads. The system also includes a compiler that is operable to compile the application program to produce a description of data flow referencing the data access threads, the computation threads and stream descriptors of the application program, a means for generating a hardware description and executable code dependent upon the description of the data flow, and a means for configuring the processing hardware in accordance with the hardware description.
To configure the processing system, a programmer generates a set of programming instructions of a high level language to define the application. The set of programming instructions includes data access thread definitions dependent upon a software class template for a data access thread (each data access thread having a stream descriptor as a parameter, and, optionally, one of a data channel source and a data channel sink as a parameter), computation thread definitions dependent upon a software class template for a computation thread (each computation thread definition having a function pointer, a data channel source and a data channel sink as parameters); and stream descriptor definitions, descriptive of memory access locations. The set of programming instructions is compiled to produce a description of a data flow between the data access threads and the computational threads. Stream memory interfaces, operable to access a memory in accordance with the stream descriptor definitions, are configured in the processing system hardware, along with computational modules operable to process data in accordance with the computation thread definitions and one or more buffers operable to connect the computational modules in accordance with the description of the data flow.
In one embodiment, the processing hardware is a hardware accelerator that performs specific computations more efficiently than a general purpose main processor to which it is connected. The hardware accelerator includes a streaming memory interface (data access device) and a computational module. The streaming memory interface is used to prefetch, stage and align stream data elements, based upon a set of stream descriptors. For example, the stream descriptors may be START_ADDRESS, STRIDE, SKIP, SPAN, TYPE and COUNT values that define the location of data values in a memory. The computational module performs computations (adds, multiples, etc.) defined in the computation threads. In this example, the streaming memory interfaces, which control memory access, are decoupled from the computational modules, which perform computations.
Stream descriptors decouple memory address generation from the actual computation by relying on the programmer's knowledge of the algorithm. The programmer uses stream descriptors to express the shape and location of data in memory. The stream memory interfaces use these stream descriptors to fetch data from memory and present the aligned data in the order required by the computing platform. This decoupling allows the stream memory interface to take advantage of available memory bandwidth to prefetch data before it is needed. The system becomes dependent on average bandwidth of the memory subsystem with less sensitivity to the peak latency to access a particular data element. In addition, it benefits from having fewer stalls due to slow memory accesses, alleviating memory wall issues. In particular, the use of a buffer for interim data storage avoids performance limitations of the system bus and associated memory controller.
In addition, program threads offer a natural, well understood, programming framework to describe concurrently executing components. Threads can represent, for example, a function/loop or a cluster of functions/loops.
The stream descriptors may be included in a set of programming instructions of a high level language that define an application. The programming instructions further include computation thread definitions and memory access thread definitions. The programming instructions are compiled to produce a description of a data flow between the computational threads and the memory access threads to produce a control flow graph (CFG) that includes references to the computation threads, and to produce a symbol table with references to the data access thread, the computation threads and the stream descriptors. The specification of the interim data storage device may be generated with reference to the symbol table.
The hardware may be a reconfigurable vector processor, a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), for example.
In addition, microcontroller code 326 may be generated for a scalar processing core. This enables elements of the CFG that are not performed by the data path elements to be performed by a scalar core, such as a general purpose microcontroller core. The microcontroller code may be expressed in an executable and linkable format, for example, and stored in a computer readable medium.
In one embodiment of the invention, one or more dataflow graphs (DFG's) 320 are generated based on the set of operations in the computational threads in the CFG 314. A data flow graph is a directed graph that does not contain any conditional elements, such as branch points. In contrast, the CFG can contain conditional elements. The symbol table 312 is used to aggregate references to each thread name and each function pointer. For each function, a DFG 320 is created to describe the set of operations in graph notation. The DFG graph is used to generate a specification 322 of the stream data path for the processor. The specification 322 specifies how stream data path devices are to be implemented in the configurable hardware 318 and may be used to configure the hardware.
In accordance with one aspect of the present invention, the symbol table 312 is also used to generate a buffer specification 324. The buffer specification describes how a buffer for storing interim data transferred between computational modules is to be implemented in the hardware 318. The buffer may be a tile buffer or a FIFO (first in, first out) buffer, for example.
In the approach described above, the stream descriptors are included in a set of programming instructions of a high level language that define an application. The programming instructions may also include computation thread definitions and memory access threads. The programming instructions are compiled to produce a description of the data flow between the computational threads and the memory access threads together with a control flow graph (CFG) including references to the computation threads and a symbol table with references to the data access thread, the computation threads and the stream descriptors. The resulting specification of the interim data storage device (the buffer) is generated with reference the symbol table.
It will be apparent to those of ordinary skill in the art that buffer specification may be obtained by other means. However, in the present invention, buffer access is controlled in accordance with stream descriptors.
In SKIP=−3×640+1=−1919). If the data is written to a memory by a first computational module as shown in
An exemplary section of a symbol table is shown in
stream S0(START_ADDRESS, SKIP, stride, SPAN, TYPE, COUNT);
This instruction defines how data values for stream S0 are to be retrieved from memory. The parameters, START_ADDRESS, SKIP, stride, SPAN, TYPE, COUNT, etc., are stream descriptors that are used by a stream memory interface device to calculate the addresses in memory of successive data values. In some embodiments, a stream descriptor may be represented with a single parameter such as TYPE, or alternatively with a single parameter such as START_ADDRESS. In yet another embodiment, the parameters such as stride, SPAN, and SKIP are constants to represent a static shape in memory. The stream parameters are stored in a row of the symbol table for stream s1. In this example, the parameter values for stream S0 are given in row 704 of the table and the parameter values for stream S1 are given in row 706. The symbol table defines how data is routed between threads referenced in the CFG 314 and how the data is stored in the memory of the processor. In particular, for each stream 708 in the symbol table, the symbol table includes references 710 to the head and tail connection of each data channel in the computation threads and data access threads referenced in the CFG. It is noted that the terms ‘head’, ‘tail’, ‘sink’, ‘source’ and ‘ports’ are used to indicate connectivity and direction of data transfer for each data channel. In one embodiment, a compiler automatically determines the direction of data transfer from the CFG without explicitly definition by the programmer. These connections determine if a stream is an input or an output stream. In addition, the stream descriptors 712 are stored in the table. The symbol table 700 may include the attributes 714 of the memory. It will be apparent to those of ordinary skill in the art that various parameters may be used to describe the memory locations and access patterns of the data for input and/or output associated with memory access threads.
The symbol table 700 in
For the example CFG and Symbol Table entry in
It should be noted that the method 800 represents only an exemplary embodiment. For example, a buffer can still be used between accelerators even if the record displacement is not the same, if the programmer carefully maintains the state. In other embodiments, stream descriptor parameters may be generated or converted from existing parameters such that the location and shape are similar. Furthermore, at block 814, a threshold can be used in addition to the comparison of record shape, in order to set a different buffer size. The entire method 800 can be placed in an iterative loop such that different thread bandwidths (which are compared at block 818) can used to generate different FIFO buffers.
In the foregoing specification, specific embodiments of the present invention have been described. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the present invention as set forth in the claims below. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of the present invention. The benefits, advantages, solutions to problems, and any element(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential features or elements of any or all the claims. The invention is defined solely by the appended claims including any amendments made during the pendency of this application and all equivalents of those claims as issued.
Number | Name | Date | Kind |
---|---|---|---|
6941548 | Goodwin | Sep 2005 | B2 |
7305649 | Bellas et al. | Dec 2007 | B2 |
7603492 | Chai et al. | Oct 2009 | B2 |
20030172224 | Gulick et al. | Sep 2003 | A1 |
20040117595 | Norris et al. | Jun 2004 | A1 |
20050030980 | Debique et al. | Feb 2005 | A1 |
20060031791 | Moona et al. | Feb 2006 | A1 |
20060224786 | Mukaide et al. | Oct 2006 | A1 |
20060242617 | Bellas et al. | Oct 2006 | A1 |
20070067508 | Chai et al. | Mar 2007 | A1 |
20080120497 | Chai et al. | May 2008 | A1 |
Number | Date | Country | |
---|---|---|---|
20080244152 A1 | Oct 2008 | US |