Patent Application
20150019731
This disclosure relates to fair hierarchical arbitration of a shared resource.
A multiple-processor system generally offers relatively high performance; because each processor can operate independently of the other processors in the system with no centralized processor closely controlling every step of each processor. If there were such centralized control, the speed of the system would be determined by the speed of the central processor and its failure would cripple the entire system. Moreover, parallel processing potentially offers an increase in speed equal to the number of processors.
In a multi-processor system, the processors typically share some resources. The shared resource may be memory or a peripheral I/O device. The memory may need to be shared because the processors likely act upon a common pool of data. Moreover, the memory sharing may be either of the physical memory locations or of the contents of memory. For instance, each processor may have local memory containing information relating to the system as a whole, such as the state of interconnections through a common cross-point switch. This information is duplicated in the local memory of each processor. When these local memories are to be updated together, the processors must agree among themselves which processor is to update the common information in all the local memories. The I/O devices are generally shared because of the complexity and expense associated with separate I/O devices attached to each of the processors. An even more fundamental shared resource is a bus connecting the processors to the shared resources as welt as to each other. Two processors may not simultaneously use the same bus except in the unlikely occurrence that each processor is simultaneously requiring input of the same information.
The combination of independently operating multi-processors and shared resources means that a request for a shared resource may occur at unpredictable times and two processors may simultaneously need the same shared resource. If more than one request is made or is outstanding at any time for a particular shared resource, conflict resolution must be provided which will select the request of one processor and refuse the request of the others.
One aspect of the disclosure provides a method of arbitrating access to a shared resource. The method includes receiving requests from sources to access the shared resource. Each request has an associated traffic intensity of the respective source and an associated pendency of the request (e.g., age or waiting time). The method includes allocating access of the shared resource to each source in an order based on the associated traffic intensity and pendency of each request. The traffic intensity of a source may be the number of unacknowledged requests issued by that source at a time of generation of the associated request. The pendency of the request may be a difference between the generation time of the request and an arbitration cycle time.
Implementations of the disclosure may include one or more of the following features. In some implementations, the method includes allocating access of the shared resource to each source in an order based on an urgency associated with one or more requests. For example, a request to access from memory and/or execute instructions for an operating system may have greater urgency than a request to access from memory and/or execute instructions for an application that executes within the operating system. The method may include allocating access of the shared resource to each source in an order based on a queue depth associated with each request. The queue depth equals the number of requests outstanding for the shared resource at the generation time of the request. The queue depth may serve as a proxy of the number of requests outstanding. Specific notations (i.e., incrementing a counter of ‘outstanding requests’ when one is generated, and decrementing the same counter when it is satisfied) may be used to track the number of pending requests.
The method may include allocating access of the shared resource to a first request having an associated first traffic intensity before a second request having an associated second traffic intensity, where the first traffic intensity is greater than the second traffic intensity. Moreover, the method may include allocating access of the shared resource to a first request having an associated first pendency before a second request having an associated second pendency, where the first pendency is greater than the second pendency.
In some implementations, the method includes allocating access of the shared resource to a first request having a first attribute value before a second request having an associated second attribute value, where the first attribute value differs from the second attribute in some material way consistent with a figure of merit (e., is greater than the second attribute value). Each attribute value may equal a sum of the traffic intensity and the pendency of the respective request. In some examples, the attribute value equals a sum of the traffic intensity, the pendency, and an urgency of the respective request. The urgency may have a numerical value. In additional examples, the attribute value equals a sum of the traffic intensity, the pendency, and a queue depth of the respective request. The queue depth equals the number of requests outstanding for the shared resource at the generation time of the request.
The method may include reading a packet header of each request. The packet header has attributes that include the traffic intensity and the pendency. Moreover, the method may include updating the traffic intensity and/or the pendency in the packet header of each unselected request after each arbitration cycle.
Another aspect of the disclosure provides an arbiter that includes a receiver and an allocator in communication with the receiver. The receiver receives requests from sources to access at least one shared resource. Each request has an associated traffic intensity of the respective source and an associated pendency of the request. The allocator allocates access of at least one shared resource to each source in an order based on the associated traffic intensity and pendency of each request. The traffic intensity of a source may be the number of unacknowledged requests issued by that source at a time of generation of the associated request. The pendency of the request may be a difference between the generation time of the request and an arbitration cycle time.
In some implementations, the allocator allocates access of the shared resource to each source in an order based on an urgency associated with one or more requests. The urgency may be a weighting, such as a number, evaluated by the allocator. Moreover, the allocator may allocate access of the shared resource to each source in an order based on a queue depth associated with each request. The queue depth equals the number of requests outstanding for the shared resource at the generation time of the request.
The allocator may allocate access of the shared resource to a first request having an associated first traffic intensity before a second request having an associated second traffic intensity, where the first traffic intensity is greater than the second traffic intensity. Similarly, the allocator may allocate access of the shared resource to a first request having an associated first pendency before a second request having an associated second pendency, where the first pendency is greater than the second pendency. Additionally, or alternatively, the allocator may allocate access of the shared resource to a first request having a first attribute value before a second request having an associated second attribute value, where the first attribute value is greater than the second attribute value. Each attribute value may equal a sum of the traffic intensity and the pendency of the respective request. In some examples, the attribute value equals a sum of the traffic intensity, the pendency, and an urgency of the respective request. The urgency may have a numerical value. In additional examples, the attribute value equals a sum of the traffic intensity, the pendency, and a queue depth of the respective request. The queue depth equals the number of requests outstanding for the shared resource at the generation time of the request.
In some implementations, the receiver reads a packet header of each request. The packet header has attributes that include the traffic intensity and the pendency. The receiver may update the pendency in the packet header of each unselected request after each arbitration cycle.
Yet another aspect of the disclosure provides a computer program product encoded on a computer readable storage medium including instructions that when executed by a data processing apparatus cause the data processing apparatus to perform operations. The operations include receiving requests from sources to access the shared resource. Each request has an associated traffic intensity of the respective source and an associated pendency of the request (e.g., age or waiting time). The operations include allocating access of the shared resource to each source in an order based on the associated traffic intensity and pendency of each request. The traffic intensity of a source may be the number of unacknowledged requests issued by that source at a time of generation of the associated request. The pendency of the request may be a difference between the generation time of the request and an arbitration cycle time.
In some implementations, the operations include allocating access of the shared resource to each source in an order based on an urgency associated with one or more requests. For example, a request to access from memory and/or execute instructions for an operating system may have greater urgency than a request to access from memory and/or execute instructions for an application that executes within the operating system. The operations may include allocating access of the shared resource to each source in an order based on a queue depth associated with each request. The queue depth equals the number of requests outstanding for the shared resource at the generation time of the request.
The operations may include allocating access of the shared resource to a first request having an associated first traffic intensity before a second request having an associated second traffic intensity, where the first traffic intensity is greater than the second traffic intensity. Moreover, the operations may include allocating access of the shared resource to a first request having an associated first pendency before a second request having an associated second pendency, where the first pendency is greater than the second pendency.
In some implementations, the operations include allocating access of the shared resource to a first request having a first attribute value before a second request having an associated second attribute value, where the first attribute value greater than the second attribute value. Each attribute value may equal a sum of the traffic intensity and the pendency of the respective request. In some examples, the attribute value equals a sum of the traffic intensity, the pendency, and an urgency of the respective request. The urgency may have a numerical value. In additional examples, the attribute value equals a sum of the traffic intensity, the pendency, and a queue depth of the respective request. The queue depth equals the number of requests outstanding for the shared resource at the generation time of the request.
The operations may include reading a packet header of each request. The packet header has attributes that include the traffic intensity and the pendency. Moreover, the operations may include updating the pendency in the packet header of each unselected request after each arbitration cycle.
The details of one or more implementations of the disclosure are set forth in the accompanying drawings and the description below. Other aspects, features, and advantages will be apparent from the description and drawings, and from the claims.
Like reference symbols in the various drawings indicate like elements.
Referring to
In general, an arbitration process creates a linearization among sources Sn (i.e., sharers or requesters) representing a partial ordering of “events” (e.g., memory requests or network requests) that can be deemed “locally fair” among the available sources Sn. Fair arbitration among sources Sn sharing common resources 200 allows efficient operation of the sources Sn.
Referring again to
The traffic intensity λ of a source Sn is the number of unacknowledged requests Rn from that source Sn in the system 10 at any given point in time. Since the traffic intensity λ fluctuates with time, for example as a result of bursty tragic, the traffic intensity λ represents the number of unacknowledged requests at the time a request Rn is generated (i.e. when a request packet is created in a load-store unit, or network interface).
An arbitration scheme may be considered “fair” if, for an equal offered load, each of the n sources Sn receive 1/n of the aggregate bandwidth of the shard resource 200. Moreover, the arbitration scheme may be locally fair by considering the traffic intensity λn emitted by each source Sn. The arbitration scheme may use the traffic intensity λn as a weight of a request Rn and perform a weighted, round-robin arbitration. This may not result a sharing pattern that is temporally fair, since the selection process is biased toward requests Rn emitted by “busier” sources (those with a higher traffic intensity λn).
Referring to
In some examples, the allocator 120 allocates access of the shared resource 200 to a first request R1 having an associated first traffic intensity λ1 before a second request R2 having an associated second traffic intensity λ2, where the first traffic intensity λ1 is greater than the second traffic intensity λ2. Similarly, the allocator 120 may allocate access of the shared resource 200 to a first request R1 having an associated first pendency T1 before a second request R2 having an associated second pendency T2, where the first pendency T1 is greater than the second pendency T2.
In some implementations, the allocator 120 allocates access of the shared resource 200 to each source Sn in an order based on an urgency associated with one or more requests Rn. The urgency may be a weighting, such as a number, evaluated by the allocator 120. For example, a first request R1 may have a first urgency less than a second urgency of a corresponding second request R2. As s results, the allocator 120 may provide access to the shared resource 200 for the second request R2 before the first request R1. The second request R2 may be for accessing and/or executing instructions of an operating system, white the first request R1 may be for accessing and/or executing instructions of a software application that executes within the operating system.
The allocator 120 may allocate access of the shared resource 200 to each source Sn in an order based on a queue depth associated with each request Rn. The queue depth equals a number of requests Rn outstanding for the shared resource 200 at the generation time of the request Rn. The queue depth may serve as a proxy of the number of requests Rn outstanding. Specific notations (i.e., incrementing a counter of ‘outstanding requests’ when one is generated, and decrementing the same counter when it is satisfied) may be used to track the number of pending requests Rn.
Additionally, or alternatively, the allocator 120 may allocate access of the shared resource 200 to first request R1 having a first attribute value before a second request R1 having an associated second attribute value, where the first attribute value is greater than the second attribute value. Each attribute value may equal a sum of the traffic intensity λ and the pendency T of the respective request R. In some examples, the attribute value equals a sum of the traffic intensity λ, the pendency T, an urgency of the respective request, and/or a queue depth of the respective request R. The urgency may have a numerical and/or weighted value. The queue depth equals a number of requests Rn outstanding for the shared resource 200 at the generation time of the request R. In some implementations, the attribute value is expressed as a fraction of peak bandwidth, a value between 0 and 1, a time interval, or a value between 0 and a maximum number of requests Rn.
In some implementations, the method includes allocating 406 access of the shared resource 200 to each source Sn in an order based on an urgency associated with one or more requests Rn. For example, a request Rn to access from memory and/or execute instructions for an operating system may have greater urgency than a request to access from memory and/or execute instructions for an application that executes within the operating system. The method may include allocating 408 access of the shared resource 200 to each source Sn an order based on a queue depth associated with each request Rn. The queue depth equals a number of requests Rn outstanding for the shared resource 200 at the generation time of the request Rn.
The method may include allocating access of the shared resource 200 to a first request R1 having an associated first traffic intensity λ1 before a second request R2 having an associated second traffic intensity λ2, where the first traffic intensity λ1 is greater than the second traffic intensity λ2. Moreover, the method may include allocating access of the shared resource 200 to a first request R1 having an associated first pendency T1 before a second request R2 having an associated second pendency T2, where the first pendency T1 is greater than the second pendency T2.
In some implementations, the method includes allocating access of the shared resource 200 to a first request R1 having a first attribute value before a second request R2 having an associated second attribute value, where the first attribute value greater than the second attribute value. Each attribute value may equal a sum of the traffic intensity λ1, λ2 and the pendency T1, T2 of the respective request R1, R2. In some examples, the attribute value equals a sum of the traffic intensity λ1, λ2, the pendency T1, T2, an urgency, and/or a queue depth of the respective request R1, R2. The queue depth equals a number of requests Rn outstanding for the shared resource 200 at the generation time of the request Rn.
Each processing element in the network system 500 may participate in a network-wide protocol where every request Rn (e.g., memory load) has a corresponding reply Pn (e.g., data payload reply). This bifurcates all messages in the network system 500 by decomposing all communication into two components: request Rn and reply Pn.
The source Sn of the request Rn may maintain a count N of the number of simultaneously outstanding requests Rn pending in the network system 500. The count N can be incremented for every newly created request Rn by a processing element and decremented for every reply Pn received. At the time the request Rn is made, the source Sn forms a message of one or more packets 505, each having a packet header that carries information about the message and for routing the packet from the source Sn to its destination Dn. A field within the packet header conveys the traffic intensity λ as λ=N (initial value when the request is initiated), where N is a current count of unacknowledged requests Rn. Since requests Rn are not necessarily uniform in size, a finer granularity of traffic intensity λ as a count M of the number of message flits (flow control units) currently in the network system 500.
Referring to
The arbitration process may select at most one input that will be granted the output. The receiver 110 may update the pendency Tn in the packet header of each unselected request Rn after each arbitration cycle. For example, every unselected request Rn−1 may receive an updated pendency T as T=T+1. Moreover, in examples where the pendency T is added to the traffic intensity λ, the traffic intensity can be updated as λ=λ+1, representing the accumulation of time (measured in clock cycles) waiting at the front of the queue.
The arbitration scheme combines weighted round-robin with age-based arbitration in a way that provides starvation-free selection among an arbitrary set of inputs (e.g., requests Rn from sources Sn) of varying traffic intensity λ. The traffic intensity λ provides a connection between arbitration priority and offered load. Relating the traffic intensity λ to the number of outstanding, unacknowledged requests Rn in the network system 500 may smooth out transient load imbalance and temporarily provides preference to those sources Sn that are generating traffic but not getting serviced in a timely manner. The traffic intensity λ is relatively larger for a first source S1 communicating with a distant destination D1 than for a second sources S2 communicating with nearby destination D2, because the round-trip latency is larger and relatively more packets 505 are in-flight for the first source S1. As a result, the arbiter 100 may grant the first source S1 access to the output (to the shared resource 200) more often than the second source S2.
Various implementations of the systems and techniques described here can be realized in digital electronic circuitry, integrated circuitry, specially designed ASICs (application specific integrated circuits), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and/or interpretable on a programmable system including at least one programmable processor, which may be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.
These computer programs (also known as programs, software, software applications or code) include machine instructions for a programmable processor, and can be implemented in a high-level procedural and/or object-oriented programming language, and/or in assembly/machine language. As used herein, the terms “machine-readable medium” and “computer-readable medium” refer to any computer program product, apparatus and/or device (e.g., magnetic discs, optical disks, memory, Programmable Logic Devices (PLDs)) used to provide machine instructions and/or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term “machine-readable signal” refers to any signal used to provide machine instructions and/or data to a programmable processor.
Implementations of the subject matter and the functional operations described in this specification can be implemented in digital electronic circuitry, or in computer software, firmware, or hardware, including the structures disclosed in this specification and their structural equivalents, or in combinations of one or more of them. Moreover, subject matter described in this specification can be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by, or to control the operation of, data processing apparatus. The computer readable medium can be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter effecting a machine-readable propagated signal, or a combination of one or more of them. The terms “data processing apparatus”, “computing device” and “computing processor” encompass all apparatus, devices, and machines for processing data, including by way of example a programmable processor, a computer, or multiple processors or computers. The apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code that constitutes processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them. A propagated signal is an artificially generated signal, e.g., a machine-generated electrical, optical, or electromagnetic signal, that is generated to encode information for transmission to suitable receiver apparatus.
A computer program (also known as an application, program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted languages, and it can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in a portion of a file that holds other programs or data (e.g., one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub programs, or portions of code). A computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.
The processes and logic flows described in this specification can be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows can also be performed by, and apparatus can also be implemented as, special purpose logic circuitry, e.g., an FPGA programmable gate array) or an ASIC (application specific integrated circuit).
Processors suitable for the execution of a computer program include, by way of example, both general and special purpose microprocessors, and any one or more processors of any kind of digital computer. Generally, a processor will receive instructions and data from a read only memory or a random access memory or both. The essential elements of a computer are a processor for performing instructions and one or more memory devices for storing instructions and data. Generally, a computer will also include, or be operatively coupled to receive data from or transfer data to, or both, one or more mass storage devices for storing data, e.g., magnetic, magneto optical disks, or optical disks. However, a computer need not have such devices. Moreover, a computer can be embedded in another device, e.g., a mobile telephone, a personal digital assistant (PDA), a mobile audio player, a Global Positioning System (GPS) receiver, to name just a few. Computer readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media and memory devices, including by way of example semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices; magnetic disks, e.g., internal hard disks or removable disks; magneto optical disks; and CD ROM and DVD-ROM disks. The processor and the memory can be supplemented by, or incorporated in, special purpose logic circuitry.
To provide for interaction with a client, one or more aspects of the disclosure can be implemented on a computer having a display device, e.g., a CRT (cathode ray tube), LCD (liquid crystal display) monitor, or touch screen for displaying information to the client and optionally a keyboard and a pointing device, e.g., a mouse or a trackball, by which the client can provide input to the computer. Other kinds of devices can be used to provide interaction with a client as well; for example, feedback provided to the client can be any form of sensory feedback, e,g., visual feedback, auditory feedback, or tactile feedback; and input from the client can be received in any form, including acoustic, speech, or tactile input. In addition, a computer can interact with a client by sending documents to and receiving documents from a device that is used by the client; for example, by sending web pages to a web browser on a client's client device in response to requests received from the web browser.
One or more aspects of the disclosure can be implemented in a computing system that includes a backend component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a frontend component, e.g., a client computer having a graphical client interface or a Web browser through which a client can interact with an implementation of the subject matter described in this specification, or any combination of one or more such backend, middleware, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (“LAN”) and a wide area network (“WAN”), an inter-network (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks).
The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. In some implementations, a server transmits data (e.g., HTML page) to a client device (e.g., for purposes of displaying data to and receiving client input from a client interacting with the client device). Data generated at the client device (e.g., a result of the client interaction) can be received from the client device at the server.
While this specification contains many specifics, these should not be construed as limitations on the scope of the disclosure or of what may be claimed, but rather as descriptions of features specific to particular implementations of the disclosure. Certain features that are described in this specification in the context of separate implementations can also be implemented in combination in a single implementation. Conversely, various features that are described in the context of a single implementation can also be implemented in multiple implementations separately or in any suitable sub-combination. Moreover, although features may be described above as acting in certain combinations and even initially claimed as such, one or more features from a claimed combination can in some cases be excised from the combination, and the claimed combination may be directed to a sub-combination or variation of a sub-combination.
Similarly, while operations are depicted in the drawings in a particular order, this should not be understood as requiring that such operations be performed in the particular order shown or in sequential order, or that all illustrated operations be performed, to achieve desirable results. In certain circumstances, multi-tasking and parallel processing may be advantageous. Moreover, the separation of various system components in the embodiments described above should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.
A number of implementations have been described. Nevertheless, it will be understood that various modifications may be made without departing from the spirit and scope of the disclosure. Accordingly, other implementations are within the scope of the following claims. For example, the actions recited in the claims can be performed in a different order and still achieve desirable results.
