The present disclosure relates generally to data interfaces between agents, and more specifically to point-to-point data interfaces requiring initialization before general operations of data transfer between the agents.
Microprocessor systems have until recently been interconnected via multi-point drop data buses. The processors, memory controllers, input-output controllers (which may generally be termed “agents”) would be able to exchange data over a common data bus structure. However, as data transmission rates become higher, limitations in the multi-point drop data buses are becoming a problem. The electrical loadings and reflections in a multi-point drop data bus system may limit the data transmission speed. In order to address these and other issues, newer systems are examining individual, dedicated point-to-point data interfaces between the agents of a system.
There will still exist variances among agents attempting to exchange data via the point-to-point interfaces. Source impedances, path impedances, and termination impedances may all vary due to process variations and other influences. Data skew among the various parallel data lines, and between the clock and data lines, may become more of a problem at higher data rates. For this reason, during an initialization process the two agents at the opposite ends of the point-to-point interface may exchange special data messages to support the initialization process. For example, pre-determined data messages may help initialize a set of deskewing buffers in a parallel interface. It would be possible to simply send a large number of such messages and presume that the two agents would successfully receive and act upon a sufficient number of them. However this may prove to be a time-consuming process. If the process consumes too much time, it may impact system performance if the initialization is needed not just on a relatively-rare system reset event, but also on commonly occurring events such as transitions between normal operating modes and low-power operating modes.
The present invention is illustrated by way of example, and not by way of limitation, in the figures of the accompanying drawings and in which like reference numerals refer to similar elements and in which:
The following description describes techniques for handshaking with acknowledgement to initialize a series of individual data lanes into data links is shown. In the following description, numerous specific details such as logic implementations, software module allocation, signaling techniques, and details of operation are set forth in order to provide a more thorough understanding of the present invention. It will be appreciated, however, by one skilled in the art that the invention may be practiced without such specific details. In other instances, control structures, gate level circuits and full software instruction sequences have not been shown in detail in order not to obscure the invention. Those of ordinary skill in the art, with the included descriptions, will be able to implement appropriate functionality without undue experimentation. In certain embodiments the invention is disclosed in the form of an interface for connecting together several Itanium® Processor Family (IPF) compatible processors, or for connecting together several Pentium® compatible processors, such as those produced by Intel® Corporation. However, the invention may be practiced for interconnecting other kinds of processors, such as an X-Scale® family compatible processor (but not limited to any family of processor), or for interconnecting other forms of agents, such as memory hubs or input/output device hubs or chipsets. The invention may also be practiced in the interfacing of mixed kinds of processors or other agents. Finally, the invention may be practiced in dedicated point-to-point interfaces, where either the sending and receiving of data occur on a pair of one-directional links, or where the sanding and receiving of data occur on simulataneous bi-directional links.
Referring now to
Agent A 100 may have series of lane transmitters T1102 through TN 110 and a clock transmitter TCLK 112 for sending data across link 1140. In other embodiments, the external clock transmitter may be omitted and the lane transmitters may send self-clocked data. Agent B 150 may have a series of lane receivers R1152 through RN 160 and a clock receiver RCLK 162 to receive the data and clock sent by agent A 100. Similarly, Agent B 150 may have series of lane transmitters T1170 through TN 1178 and a clock transmitter TCLK 180 for sending data across link 2190. In other embodiments, the external clock transmitter may be omitted and the lane transmitters may send self-clocked data. Agent A 100 may have a series of lane receivers R1120 through RN 128 and a clock receiver RCLK 130 to receive the data and clock sent by agent B 150.
The link 1140 and link 2190 are shown as including interconnecting lanes for physical transport between agents A 100 and B 150. The lanes in various embodiments may be unbalanced or balanced, differentially-driven. The number of lanes N may be any number. In some embodiments, when soft errors or hard errors in one or more lane are detected, those lanes may be ignored and the link may be configured for operation as a parallel interface with fewer than N lanes.
The signal path lengths and impedances of lanes 132, 134, 136, up to 142 may have significant differences, which may cause differing signal propagation times. This may cause unacceptable skews between lanes. For this reason, an initialization process may be undertaken upon power-on or other system reset activity. The initialization process may train the individual receivers R1152 through RN 160 to compensate for skew and other anomalies to permit efficient operation of link 1140 as a parallel data interface. Similarly the initialization process may train the individual receivers R1120 through RN 128 to compensate for skew and other anomalies to permit efficient operation of link 2190 as a parallel data interface.
In one embodiment, a sequence of numbered messages, called “training sequences”, may be sent over the individual lanes (132 through 142) temporarily acting during the initialization process as N individual serial interfaces clocked by TCLK 112. The use of the lanes as individual serial interfaces avoids the skew and other anomalies initially present when using link 1140 and link 2190 as parallel interfaces. There may be a first type of training sequence, a second type of training sequence, up to a final type of training sequence. Each of these numbered training sequences may pass parameter data for efficiently setting up the link 1140. An equivalent series of training sequences may be sent on link 2190. The outcome of the passing of training sequences back and forth across link 1140 and link 2190 may be to configure operational parallel interfaces using the lanes of link 1140 and link 2190. For example, the first training sequence may exhibit a known data pattern from which intra-lane skew timing may be determined. A second training sequence may pass parametric data about this timing and also about soft and hard data errors detected in the lanes.
Referring now to
The
In one embodiment, each agent agrees to begin sending its training sequences, modified to include an acknowledgement, after it correctly receives from the other agent two consecutive current training sequences. In other embodiments, the number of received current training sequences may be more or fewer than 2, and they need not be received consecutively. The selection of 2 consecutive received training sequences as a criteria may help reduce circuit complexity, as only one training sequence may need to be stored in order to compare with an incoming training sequence. The form of the acknowledgement may be a modification of a data pattern, a flag being set somewhere in the training sequence, or any other means of indicating an acknowledgment.
In keeping with this agreement, both agent A and agent B begin to determine whether they have correctly received two consecutive current training sequences from the other agent as soon as they begin transmitting the current training sequence themselves. At time B1, agent B begins transmitting at Tx @ B the current training sequence TS2, and examines the receipt of training sequence TS2 at Rx @ B. As agent A has been sending training sequences TS2 from time A1, at time B2 agent B will have successfully received TS2A_3 and TS2A_4. Therefore agent B has successfully received two of the current training sequences, and may then begin adding an acknowledgment indicator to subsequent transmissions of TS2, starting with TS2B_3. In one embodiment, agent B sends at least 4 of the TS2 messages including an acknowledgement. In other embodiments, few than 4 or more than 4 could be sent.
Because agent B began transmitting at a later time B1, agent A does not begin receiving training sequence TS2 until time A2. The Rx @ A successfully receives TS2B_1, but receives with an error TS2B_2. Only when Rx @ A receives TS2B_3 and TS2B_4, at time A5, does it receive 2 consecutive TS2 messages. Note that these TS2 messages do contain the acknowledgement, but this is permitted. (The second agent to correctly receive the 2 consecutive training sequences may generally be receiving one with an acknowledgement.) Since agent A has received 2 consecutive training sequences at time A5, Tx @ A may then transmit, at time A6, the TS2 messages including an acknowledgement, starting with TS2A_9.
Therefore at time A6 both agent A and agent B are currently transmitting TS2 messages including an acknowledgement. In one embodiment, each agent agrees to begin sending the next in the sequence of training sequences after (1) each agent has begun transmitting the current training sequence including an acknowledgement, and has transmitted at least 4 of these messages and (2) after beginning such transmissions, each agent correctly receives from the other agent two consecutive current training sequences, including an acknowledgement. In other embodiments, the number of received current training sequences including acknowledgement may be more or fewer than 2, and they need not be received consecutively. Additionally, in other embodiments the number of transmitted messages may be more or fewer than 4.
At time B4, agent B has transmitted at least 4 of TS2 including an acknowledgement (TS2B_3 through TS2B_7) and has also subsequently correctly received two consecutive TS2 messages including acknowledgement from agent A (TS2A_9 and TS2A_10). Therefore, agent B may then begin transmitting the next training sequence after TS2, namely TS3, at time B4. The first of these TS3 messages is transmitted from Tx @ B at time B4 (TS3B_1).
Similarly, at time A7, agent A has transmitted at least 4 of TS2 including an acknowledgement (TS2A_9 through TS2A_12) and has also (in this present example) subsequently correctly received two consecutive TS2 messages including acknowledgement from agent B (TS2B_6 and TS2A_7). Therefore, agent A may then begin transmitting the next training sequence after TS2, namely TS3, at time A7. The first of these TS3 messages is transmitted from Tx @ A at time A7 (TS3A_1). (It is noteworthy that the transmission of at least 4 of TS2 including an acknowledgement and correctly receiving two consecutive TS2 messages including acknowledgement from agent B may in fact occur in any order.)
The
Referring now to
At event E1, a first training sequence TS1B_1 may be sent from Tx @ B and received at Rx @ A. At event E2, a corresponding first training sequence TS1A_2 may be sent from Tx @ A and received at Rx @ B. At event E3, Rx @ B may receive with error a second TS1 from Tx @ A, and therefore the Rx @ B may discard any history (e.g. TS1A_2 and TS1A_3).
At event E4, the Rx @ A has correctly received 2 consecutive training sequences (TS1B_1 and TS1B_2), so starting with TS1A_4 the Tx @ A may begin transmitting at least 4 training sequences including acknowledgement. Then from event E5 to event E6, Rx @ B may correctly receive two consecutive training sequences (in this case including acknowledgement) TS1A_4 and TS1A_5. Therefore, at event E7, Tx @ B may begin transmitting at least 4 training sequences including acknowledgement. In other embodiments, the number of training sequences required to be received or transmitted may be different.
At event E8, Rx @ A has correctly received two consecutive TS1 including acknowledgement subsequent to Tx @ A beginning to transmit TS1 including acknowledgement. Therefore Tx @ A may begin sending the next in the sequence of training sequences, TS2, beginning at TS2A_1. At event E9, as agent A has advanced to transmitting TS2, Rx @ A may begin expecting to receive TS2 messages, but will simply ignore older TS1 messages (e.g. TS1B_10) as no longer relevant. Similarly, at event E10, agent B has advanced to transmitting TS2 so Rx @ B may begin expecting to receive TS2 message at that time. Here Rx @ B will simply ignore previously-received TS2 messages (e.g. TS2A_1 and TS2A_2) because they arrived before Rx @ B was ready for them.
Referring now to
Referring now to
When in decision block 518 it is determined that 2 consecutive TSM messages have been correctly received, then the process exits via the YES path and in block 522 TSM messages with acknowledgment may begin being sent. In one embodiment, 4 of these TSM messages with acknowledgement may be eventually sent. In other embodiments, fewer than 4 or more than 4 TSM messages with acknowledgement may be eventually sent. Then in decision block 526, it may be determined whether 2 consecutive TSM messages with acknowledgement have been correctly received and whether 4 TSM messages with acknowledgement have been sent. (Note that if any of the TSM messages sent in blocks 514 or 538 were TSM messages with acknowledgement, these should be included in the count determined in decision block 526.) In other embodiments, more than 2 or fewer than 2 received TSM messages with acknowledgement may be examined. If 2 consecutive TSM messages with acknowledgement have not been correctly received, or 4 TSM messages with acknowledgement have not been sent, then the process exits decision block 526 via the NO path, and in block 546 another TSM message including acknowledgement may be sent. Then in decision block 550 it may be determined whether the current time count exceeds a limit. If not, then the process exits via the NO path and re-enters decision block 526. If so, then the process exits via the YES path and enters block 552.
When either decision block 542 or decision block 550 exits via the YES path, the fact that the current time count exceeds a limit indicates a problem with one or more of the lanes. Therefore in block 552 some of the lanes, including lanes where the time limit was exceeded, may be removed from further consideration during the process. In one embodiment, standard subsets of the lanes may be removed and the interface may proceed with the remaining subsets of the lanes. For example, the subsets may be either one quarter or one half of all of the lanes, and the interface may proceed with either the remaining three quarters or one half, respectively, of the lanes. After block 552, the process then enters block 530.
When in decision block 526 it is determined that 2 consecutive TSM messages with acknowledgment have been correctly received, and that 4 TSM messages with acknowledgement have been sent, then the process exits via the YES path and in block 530 the training sequence message number M may be incremented. This may cause future sending of training sequences to advance from one training sequence to the next. In block 534 the time count may be reset and the process may repeat at block 514. The TSM now sent at block 514 will be advanced in sequence number.
Referring now to
The
In the
In the foregoing specification, the invention has been described with reference to specific exemplary embodiments thereof. It will, however, be evident that various modifications and changes may be made thereto without departing from the broader spirit and scope of the invention as set forth in the appended claims. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.
Number | Name | Date | Kind |
---|---|---|---|
5701416 | Thorson et al. | Dec 1997 | A |
5809331 | Staats et al. | Sep 1998 | A |
5870428 | Miller et al. | Feb 1999 | A |
6002882 | Garde | Dec 1999 | A |
6009488 | Kavipurapu | Dec 1999 | A |
6038606 | Brooks et al. | Mar 2000 | A |
6195702 | Cole et al. | Feb 2001 | B1 |
6356106 | Greeff et al. | Mar 2002 | B1 |
6442712 | Jeon | Aug 2002 | B1 |
6690757 | Bunton et al. | Feb 2004 | B1 |
6766464 | Collier | Jul 2004 | B2 |
6829641 | Schenkel et al. | Dec 2004 | B2 |
6880026 | Imming et al. | Apr 2005 | B2 |
6925077 | Acharya | Aug 2005 | B1 |
6985502 | Bunton | Jan 2006 | B2 |
6996643 | Wolfe et al. | Feb 2006 | B2 |
7010607 | Bunton | Mar 2006 | B1 |
7233996 | Kaniyur et al. | Jun 2007 | B2 |
7386626 | Kota et al. | Jun 2008 | B2 |
7395347 | Nemawarkar et al. | Jul 2008 | B2 |
7437643 | Khanna et al. | Oct 2008 | B2 |
7454514 | Lesartre et al. | Nov 2008 | B2 |
7512695 | Talaat et al. | Mar 2009 | B2 |
20020046379 | Miki et al. | Apr 2002 | A1 |
20020103995 | Owen et al. | Aug 2002 | A1 |
20020165973 | Ben-Yehezkel et al. | Nov 2002 | A1 |
20030226072 | Bruckman et al. | Dec 2003 | A1 |
20030236995 | Fretwell | Dec 2003 | A1 |
20040071250 | Bunton et al. | Apr 2004 | A1 |
20040236802 | Baratakke et al. | Nov 2004 | A1 |
20050013246 | Miyake et al. | Jan 2005 | A1 |
20050017756 | Priborsky | Jan 2005 | A1 |
20050024926 | Mitchell et al. | Feb 2005 | A1 |
20050027880 | Emmot | Feb 2005 | A1 |
20050132214 | Naftali | Jun 2005 | A1 |
20050152386 | Lesartre et al. | Jul 2005 | A1 |
20050154946 | Mitbander et al. | Jul 2005 | A1 |
20050225426 | Yoon et al. | Oct 2005 | A1 |
20050251595 | Lesartre | Nov 2005 | A1 |
20050270988 | DeHaemer | Dec 2005 | A1 |
20050286567 | Cherukuri et al. | Dec 2005 | A1 |
20060041696 | Cherukuri et al. | Feb 2006 | A1 |
Number | Date | Country |
---|---|---|
10145419 | May 1998 | JP |
2004350266 | Dec 2004 | JP |
2005182485 | Jul 2005 | JP |
2005236854 | Sep 2005 | JP |
2006115070 | Apr 2006 | JP |
WO 9738377 | Oct 1997 | WO |
Number | Date | Country | |
---|---|---|---|
20050262280 A1 | Nov 2005 | US |