The present invention relates generally to device access fairness and, in particular embodiments, to device access fairness in a storage network environment.
One way to implement this hybrid topology is to transmit FC traffic wrapped in Internet packets over a combination of FC and network interface card (NIC) links. However, this solution is problematic. FC frames are typically much larger that NIC packets (2000 bytes v. 256 bytes on average). Under a conventional fair arbitration scheme, FC requestors and NIC requesters are serviced on an alternating basis. Because FC frames are substantially larger than NIC packets and, consequently, pose greater network demands, FC traffic will have more throughput than NIC traffic. This situation creates a FC-heavy network that compromises the NIC's 10 gigabyte link speed. In order to preserve the NIC link speed, a dynamically adjustable arbitration scheme needs to be developed that can guarantee bandwidth for both NIC and FC traffic.
Embodiments of the present invention are directed to a network arbitration scheme that manages device access fairness by selectively and dynamically increasing a requestor queue's likelihood of being serviced. A requestor queue increases its service priority by duplicating a request entry onto a set of priority rings maintained by arbitration hardware in a host bus adapter. Duplication occurs when (1) a requestor's queue fill count (the number of descriptors stored in the queue) exceeds a watermark level or (2) a requestor's queue timer times out. In the case of time-out, the requester in the lower priority ring will duplicate itself in the higher priority ring. Because the arbitration hardware services requestors using a round robin selection scheme, the likelihood of a requestor queue being serviced increases as the number of its duplicate request entries on a priority ring increases. Upon being serviced, the requestor is able to perform the requested action, such as retrieving data from the host memory and storing it in local memory for eventual transmission over a network.
a illustrates exemplary requester service for a normal priority ring when there are no request entry duplicates according to embodiments of the invention.
b illustrates requester service for a high priority ring when there are no request entry duplicates according to embodiments of the invention.
In the following description of preferred embodiments, reference is made to the accompanying drawings in which it is shown by way of illustration specific embodiments in which the invention can be practiced. It is to be understood that other embodiments can be used and structural changes can be made without departing from the scope of the embodiments of this invention.
Embodiments of the present invention are directed to a network arbitration scheme that manages device access fairness by selectively and dynamically increasing a requester queue's likelihood of being serviced. A requestor queue increases its service priority by duplicating a request entry onto a set of priority rings maintained by arbitration hardware in a host bus adapter. Duplication occurs when (1) a requestor's queue fill count (the number of descriptors stored in the queue) exceeds a watermark level or (2) a requestor's queue timer times out. In the case of time-out, the requester in the lower priority ring will duplicate itself in the higher priority ring. Because the arbitration hardware services requestors using a round robin selection scheme, the likelihood of a requestor queue being serviced increases as the number of its duplicate request entries on a priority ring increases. Upon being serviced, the requestor is able to perform the requested action, such as retrieving data from the host memory and storing it in local memory for eventual transmission over a network.
When a requestor queue requests service, it sends a request entry to the arbitration hardware (arbiter) in the HBA. Depending on the requestor's queue fill count, watermark level 308, and timer value 310, the request entry may be duplicated within the same priority ring or the next higher priority ring maintained by the arbitration hardware to increase the likelihood of being serviced. The queue, rather than the arbiter, decides whether to duplicate one or more request entries. Independent of the duplication process, arbitration hardware utilizes one or more priority rings to determine which requestor queue to service. Both request entry duplication and requester service are described below.
A requester queue can duplicate a request entry one or more times on a priority ring to increase the requestor queue's service priority. Duplication occurs when (1) the number of descriptors in a requestor queue (the queue fill count) exceeds the programmable watermark level; or (2) the requestor's queue timer times out in which case the duplicate entry is made in the higher priority ring. Both a requestor's queue watermark level and time out value are programmable. Request entry duplication is disabled when either the requestor's queue watermark level is set to zero or when the requestor's queue timer is set to zero. Both request entry duplication preconditions are checked by logic 314 in the requestor queue at the beginning of each arbitration cycle.
When a requester queue is not empty, it places a request entry into its prededicated slot in a priority ring. Depending on the priority of the descriptor, the requester queue may place a request entry into either its prededicated slot in the normal priority queue or the high priority queue. Because most arbitration issues are resolved either at the normal 406 or high 404 priority ring levels, the highest priority ring 402 may be rarely used. Instead, the highest priority ring 402 can be reserved for debugging purposes. When a requestor requires debugging, it bypasses the arbitration hardware and writes itself directly into the highest priority ring in the DMA engine.
Duplication occurs in the first vacant spot in the lowest priority ring available. In the high priority ring, each requestor is guaranteed only one duplicate request entry. Thus, each requestor can duplicate its request entry only once in the high priority ring. Duplication in the normal priority ring is not restricted.
When a requestor queue requests service, it sends a request to the arbitration hardware (arbiter). The arbiter determines which requestor to service. This process is independent of and happens concurrently with request entry duplication. Higher priority rings are serviced before lower priority rings. Within each ring, request entries are serviced on a round robin basis. The arbiter communicates its selection to the corresponding requester queue in the DMA transfer engine through a “grant”. Upon receiving the “grant” from the arbiter, the queue issues a “valid descriptor” to the DMA transmit engine, which in turn, executes the command specified by the descriptor in the queue as defined by the read pointer. The arbiter moves to a lower priority ring when all request entries in the higher priority ring are serviced. After a request entry is serviced, three events occur. First, the hardware read pointer is modified to point to the next descriptor in the queue. Second, if the queue's fill count falls below the water-mark, all additional instantiations of the duplicate request entry in any of the priority rings, if any, are eliminated by the queue. Third, if the queue has been granted by the arbiter, the timer associated with that particular queue is reset to the user specified value, if any, so long as there is at least one unserviced descriptor in the requester queue (as determined by the position of the read and write pointers). If there are no unserviced descriptors in the requestor queue, the timer is disabled.
a illustrates exemplary requestor service for a normal priority ring 700 when there are no request entry duplicates according to embodiments of the invention. In this scenario, each requestor queue can fill only one of the eight prededicated slots 702-716 with a request entry when it contains a descriptor. The arbiter services each requestor on a round robin basis. The arbiter services the request entry (if any) in slot 702, followed by the request entry (if any) in slot 704, and so on through slot 716. Because there are no duplicates to service, the arbiter circles back to slot 702 after servicing the request entry (if any) in slot 716.
b illustrates requestor service for a high priority ring 718 when there are no request entry duplicates according to embodiments of the invention. In this scenario, each requestor queue can fill only one of the eight prededicated slots 702-716 with a request entry when it contains a descriptor. The arbiter services each requestor on a round robin basis. The arbiter services the request entry (if any) in slot 702, followed by the request entry (if any) in slot 704, and so on through slot 716. Because there are no duplicates to service, the arbiter circles back to slot 702 after servicing the request entry (if any) in slot 716.
Although embodiments of this invention have been fully described with reference to the accompanying drawings, it is to be noted that various changes and modifications will become apparent to those skilled in the art. Such changes and modifications are to be understood as being included within the scope of embodiments of this invention as defined by the appended claims.