One or more embodiments of the invention relate generally to the field of integrated circuit and computer system design. More particularly, one or more of the embodiments of the invention relate to a method and apparatus for programmable completion tracking logic to support multiple virtual channels.
Communication between devices within a computer system is typically performed using one or more busses to interconnect the devices. These busses may be dedicated busses coupling two devices or non-dedicated multi-drop busses that are multiplexed by a number of units and devices (e.g., bus agents). In addition, these busses may be dedicated to transferring a specific type of information. For example, many microprocessor architectures include a three-bus system comprised of address, data and control busses for respectively transferring data, address and control signals.
A vast amount of research in architecture design is directed to increasing data throughput within computer systems. One area of focus is directed towards increasing the processing speed and bandwidth available to peripheral devices. During the past decade, peripheral component interconnect (PCI) has provided a very successful general purpose input/output (I/O) interconnection standard. PCI is a general purpose I/O interconnect standard that utilizes PCI signaling technology, including a multi-drop parallel bus implementation. Unfortunately, the decade of time since the introduction of PCI has resulted in the identification of the various shortcomings of PCI.
PCI operates according to a reflected wave signaling technique, where a half signal swing signal is propagated, such that once reflected, the signal achieves a full signal swing. However, as electrical loads on a PCI bus are increased, the set-up time required to achieve the full signal swing fails to meet timing requirements. Furthermore, the low clock speed, the use of delayed transactions and wait states, as well as other deficiencies, resulted in poor performance for PCI devices. As a result, system architects have devised PCI-X to target deficiencies in the PCI bus protocol.
Unfortunately, PCI-X retains many of the deficiencies attributed to the PCI bus protocol. For example, PCI-X retains the reflected wave signaling technique of PCI. Due to the use of the reflected wave signaling, it is a challenge to design a device or system that satisfies PCI-X signal timing requirements as additional device loads are added to a PCI-X bus. As a result, the number of loads on a PCI-X bus to one device is effectively reduced to meet PCI-X signal timing requirements. As a result, PCI Express (PCI_XP) is designed to overcome deficiencies of PCI and PCI-X.
A PCI-XP link provides a point-to-point link, as defined by PCI Express Base Specification 1.0a (Errata dated 7 Oct. 2003) to allow bi-directional communication between peripheral endpoint devices via a switch. Accordingly, the various peripheral endpoints coupled to a PCI-XP link are required to arbitrate for ownership of PCI-XP link to issue transactions. Such arbitration is required since endpoints are generally not allowed to simultaneously drive a PCI-XP link. PCI-XP encodes transactions using a packet-based protocol. Packets are transmitted and received serially and byte stripped across the available lanes of a PCI-XP link.
PCI-XP transactions can be divided into two categories. Those transactions that are non-posted and those that are posted. Non-posted transactions, such as memory reads, implement a split transaction communication model similar to the legacy PCI-X split transaction protocol. For example, a requester device transmits a non-posted type memory read request packet to a completer. The completer returns a completion packet with the read data to the requester. Posted transactions, such as memory writes, consist of a completer with no completion packet returned from completer to requester. Unexpected completions, as well as completion time-out, are two new error handling scenarios defined by PCI-XP.
The various embodiments of the present invention are illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which:
A method and apparatus for programmable completion tracking logic to support multiple virtual channels are described. In one embodiment, the apparatus includes a controller having programmable completion tracking logic for supporting multiple virtual channels. In one embodiment, a completion tracking queue is programmable to provide a predetermined number of entries to store upstream read request information corresponding to a virtual channel from a plurality of virtual channels supported by the controller. In one embodiment, the predetermined number of entries are shared among the plurality of virtual channels according to a minimum entry value and a maximum entry value defined for each respective virtual channel.
In one embodiment, control logic prohibits transmission of an upstream read request for a virtual channel unless an entry within the completion tracking queue is available for the virtual channel. Accordingly, in one embodiment, a common pool of entries within the completion tracking queue are made available to the multiple virtual channels provided by the controller to provide a varying quality of service (QoS) to a plurality of traffic classes defined for the controller.
In the following description, numerous specific details such as logic implementations, sizes and names of signals and buses, types and interrelationships of system components, and logic partitioning/integration choices are set forth to provide a more thorough understanding. It will be appreciated, however, by one skilled in the art that the invention may be practiced without such specific details. In other instances, control structures and gate level circuits have not been shown in detail to avoid obscuring the invention. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate logic circuits without undue experimentation.
In the following description, certain terminology is used to describe features of the invention. For example, the term “logic” is representative of hardware and/or software configured to perform one or more functions. For instance, examples of “hardware” include, but are not limited or restricted to, an integrated circuit, a finite state machine or even combinatorial logical. The integrated circuit may take the form of a processor such as a microprocessor, application specific integrated circuit, a digital signal processor, a micro-controller, or the like.
System
Representatively, chipset 110 may include memory controller hub 120 (MCH) coupled to graphics controller 140. In an alternative embodiment, graphics controller 140 is integrated into MCH, such that, in one embodiment, MCH 200 operates as an integrated graphics MCH (GMCH). Representatively, MCH 200 is also coupled to main memory 130. In one embodiment, main memory 130 may include, but is not limited to, random access memory (RAM), dynamic RAM (DRAM), static RAM (SRAM), synchronous DRAM (SDRAM), double data rate (DDR) SDRAM (DDR-SDRAM), Rambus DRAM (RDRAM) or any device capable of supporting high-speed buffering of data.
As further illustrated, chipset 110 includes an input/output (I/O) controller hub (ICH) 150. Representatively, ICH 150 may include a universal serial bus (USB) link or interconnect 154 to couple one or more USB slots 152 to ICH 150. In addition, an audio codec (AC) 190 may be coupled to ICH 150. Likewise, a serial advance technology attachment (SATA) 158 may couple hard disk drive devices (HDD) 156 to ICH 150. Likewise, ICH 150 may include peripheral component interconnect (PCI) PCI-Express (PCI-XP) root ports 160 (160-1, . . . 160-N) to couple switches 170 (170-1, . . . , 170-N) to ICH 150 via PCI-XP links 162 (162-1, . . . , 162-N) referred to herein as the “fabric”. In one embodiment, system BIOS 106 initializes computer system 100.
Although chipset 110 is illustrated as including a separate MCH 120 and ICH 150, in one embodiment, MCH 120 may be integrated within CPU 102. In an alternate embodiment, the functionality of MCH 120 and ICH 150 are integrated within chipset 110. In one embodiment, completion tracking logic 300 may be implemented within computer systems including an MCH integrated within a CPU, an MCH and ICH integrated within a chipset, as well as a system on-chip. Accordingly, those skilled in the art recognize that
Representatively, PCI-XP links 162 may provide a point-to-point link, such as defined by PCI Express Base Specification 1.0a (Errata dated 7 Oct. 2003) to allow bi-directional communication between peripheral endpoint devices 180 (180-1, . . . , 180-N) via switch 170. Accordingly, the various peripheral endpoints 180 are required to arbitrate for ownership of PCI-XP link 162 to issue transactions. Such arbitration is required since endpoints 180 are generally not allowed to simultaneously drive PCI-XP link 162. As described herein, an endpoint that is requesting data is referred to as a “requester”. Likewise, an endpoint from which data is requested is referred to as a “completer”.
PCI-XP encodes transactions using a packet-based protocol. Packets are transmitted and received serially and byte stripped across the available lanes of a PCI-XP link. In one embodiment, ICH backbone 200 of ICH 150 provides varying quality of service (QoS) for various traffic classes defined according to the packet-based communication provided by PCI-XP, which are routed via virtual channels (VCs). The QoS feature of PCI-XP refers to the capability of routing packets from different applications through the fabric with differentiated priorities and deterministic latencies and bandwidth.
Providing varying QoS to PCI-XP packets involves the definition of at least two traffic classes (TC). Once defined, packets associated with an application a TC are assigned the respective TC within a packet header. PCI-XP packets contain a TC number between 0 and 7 that is assigned by the device application or device driver. Packets with different TCs can move through the fabric with different priority, resulting in varying performance. These packets arrive through the fabric by utilizing virtual channel (VC) buffers implemented in switches, endpoints and root complex devices, such as ICH 150.
In one embodiment, as illustrated in
There may be three layers that build a transaction. The first layer may be the transaction layer, which begins the process of turning a request or completion data coming from a device core into packet data for a transaction. The second architectural build layer is called the data link layer; it ensures the packets going back and forth across the link are properly received (via techniques, such as error control coding). The third layer is called the physical layer. This layer is responsible for the actual transmitting and receiving of the packet across the link.
Furthermore, PCI-XP transactions can be divided into two categories. Those transactions that are non-posted and those that are posted. Non-posted transactions, such as memory reads, implement a split transaction communication model similar to the legacy PCI (PCI-X) split transaction protocol. For example, a requester device transmits a non-posted type memory read request packet to a completer. The completer returns a completion packet with the read data to the requester. Posted transactions, such as memory writes, consist of a completer with no completion packet returned from completer to requester.
Referring again to
Referring again to
In one embodiment, completion tracking logic supports error handling of unexpected completions, as well as completion time-out, which are two new error handling scenarios defined by the PCI-XP. For example, when a completion is returned from memory 130, completion tracking logic 300 compares the completion information with request information within completion tracking queue 360 (
In one embodiment, if a completion for an upstream NP request is not returned within a predetermined period of time, such as, for example, 20 milliseconds (ms), the transaction is considered to have timed out and an error is logged. In one embodiment, completion tracking logic 300 synthesizes a completion with error completion status. In one embodiment, the synthesized completion is provided to prevent legacy devices from exhibiting undesired or unintended behavior, which may hang computer system 100. In one embodiment, request information logic 310, as shown in
In one embodiment, as illustrated in
In addition, bandwidth requirements of read requests on the different VCs may vary quite significantly from one platform to another (e.g., between a desktop and a server), thus making it difficult to design an ideal amount of entries within completion tracking queue 360. Allocating more entries than necessary to a VC within completion tracking queue 360 may result in a higher cost, while constraining the number of entries too strictly may severely impact performance and thereby preclude any of the advantages from providing varying QoS for the traffic classes defined for ICH 150.
In one embodiment, completion tracking queue 360 is configured to enable assignment or definition of a minimum and maximum number of entries for each VC supported by chipset 110 (
In one embodiment, upstream request capture logic 320 detects upstream read requests directed to main memory 130. Representatively, request information 270 associated with the upstream NP read request is provided to available entry logic 340. In one embodiment, available entry logic 340 determines a virtual channel associated with the upstream NP read request. Once determined, available entry logic 340 determines whether an entry within completion tracking queue 360 is available for the associated virtual channel. When such is the case, request information 270 is stored within completion tracking queue. Otherwise, control logic 310 delays transmission of the upstream read request 262 until an entry for the associated virtual channel within completion tracking queue 360 becomes available.
As further illustrated in
In one embodiment, request information 270 is loaded into the lowest available entry 370 (370-1, . . . , 370-N) within completion tracking queue 360. In one embodiment, identification of the lowest available entry within completion tracking queue 360 is determined by available entry logic 340. In one embodiment, when a completion is available at the output of downstream queue 230 (
In one embodiment, length information 272 is taken from the lowest queue entry 370 that matches stream ID 270 and the length of the completion packet is subtracted from length information 272. If the difference is zero, it indicates that the entire completion has returned and the entry is removed. In one embodiment, this functionality is performed by downstream completion compare logic 330 in conjunction with VC allocation/removal logic 350, as directed by request information logic 310. In one embodiment, all entries greater than the queue entry containing the matching stream ID are shifted up by one entry. In other words, in one embodiment, removal of an entry from completion tracking queue 360 results in popping off of the entry as remaining queue entries below the matching entry are shifted up.
Referring again to
As illustrated in
In one embodiment, total non-posted upstream requests limit register (UNRL.T) 388 defines the total number of available entries to be provided by upstream request queue 360. As further illustrated, a non-posted upstream request limit (UNRL) register 386 is provided for each virtual channel (UNRL.V1, UNRL.Vp and UNRL.VO) to define the maximum number of entries available to the respective channel. Likewise, upstream minimum reserve (UMR) registers 384 define a minimum entry value to be allocated for each VC supported by ICH 150 (UMR.V1, UMR.Vp and UMR.V0). Finally, upstream request NP request limit disable (UNRL) registers 382 are set to disable a respective virtual channel supported by ICH 150 (UNRL.V1d, UNRL.Vpd and UNRL.V0d).
Referring again to
As further illustrated within
Operation
In any representation of the design, the data may be stored in any form of a machine readable medium. An optical or electrical wave 660 modulated or otherwise generated to transport such information, a memory 650 or a magnetic or optical storage 640, such as a disk, may be the machine readable medium. Any of these mediums may carry the design information. The term “carry” (e.g., a machine readable medium carrying information) thus covers information stored on a storage device or information encoded or modulated into or onto a carrier wave. The set of bits describing the design or a particular of the design are (when embodied in a machine readable medium, such as a carrier or storage medium) an article that may be sealed in and out of itself, or used by others for further design or fabrication.
It will be appreciated that, for other embodiments, a different system configuration may be used. For example, while the system 100 includes a single CPU 102, for other embodiments, a multiprocessor system (where one or more processors may be similar in configuration and operation to the CPU 102 described above) may benefit from the programmable completion tracking queue to support multiple virtual channels of various embodiments. Further different type of system or different type of computer system such as, for example, a server, a workstation, a desktop computer system, a gaming system, an embedded computer system, a blade server, a system on-chip, etc., may be used for other embodiments.
Having disclosed exemplary embodiments and the best mode, modifications and variations may be made to the disclosed embodiments while remaining within the scope of the embodiments of the invention as defined by the following claims.