Printed circuit board (PCB) materials exhibit temperature dependent characteristics, some of which result in greater signal attenuation at higher temperatures. The signal losses are also higher at higher data rates. Systems operating in wide temperature ranges, such as in outdoor environments can see PCB interconnect loss variation exceeding 6 dB at 25 Gbps signaling rates. In some implementations, such as Telecom, transceivers should be able to tolerate this variation error free, across the entire temperature range without requiring re-establishment or re-training of links over lengthy periods, such as several years.
In addition to PCB materials, other components along link signal paths may exhibit signal attenuation at higher temperature. For example, this may include cables and connectors.
The foregoing aspects and many of the attendant advantages of this invention will become more readily appreciated as the same becomes better understood by reference to the following detailed description, when taken in conjunction with the accompanying drawings, wherein like reference numerals refer to like parts throughout the various views unless otherwise specified:
Embodiments of methods and apparatus to compensate for post-training insertion loss variation are described herein. In the following description, numerous specific details are set forth to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that the invention can be practiced without one or more of the specific details, or with other methods, components, materials, etc. In other instances, well-known structures, materials, or operations are not shown or described in detail to avoid obscuring aspects of the invention.
Reference throughout this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present invention. Thus, the appearances of the phrases “in one embodiment” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.
For clarity, individual components in the Figures herein may also be referred to by their labels in the Figures, rather than by a particular reference number. Additionally, reference numbers referring to a particular type of component (as opposed to a particular component) may be shown with a reference number followed by “(typ)” meaning “typical.” It will be understood that the configuration of these components will be typical of similar components that may exist but are not shown in the drawing Figures for simplicity and clarity or otherwise similar components that are not labeled with separate reference numbers. Conversely, “(typ)” is not to be construed as meaning the component, element, etc. is typically used for its disclosed function, implement, purpose, etc.
Insertion loss of high-speed serial interconnect on PCBs varies proportionally to the electrical signaling rate. Further, the insertion loss also varies proportionally with platform temperature. These variations depend on the PCB material selection.
For example, suppose the link is initially established at a platform temperature of 90 C and operates error free with a healthy margin. As time passes, the platform temperature may decline such that the insertion loss of the operating channel is reduced. The receiver is expected to operate error free with its initial solution when the interconnect is more lossy, and equally so sometime later when the interconnect is less lossy. The receiver applies additional attenuation to the incoming signal to compensate for the reduced system loss. It should apply this attenuation gradually over time and without causing any errors or dropping the link. Conversely, a link initially established at a low interconnect temperature should add additional gain to the link solution to compensate for the additional loss incurred as the interconnect temperatures increases.
In accordance with aspects of the embodiments disclosed herein the current temperature is incorporated as an indicator for the current interconnect insertion loss such that the receiver's convergence reserves additional adaptation capability for either post-training attenuation or gain if the link is established towards either temperature extreme. This approach reduces the need for the receiver to be capable of both additional attenuation and gain at any condition that it initially established the link, which would require additional operating margin in the design.
During initial adaptation of the link, the receiver trains its equalizers to maximize the link's margin at that operating condition. During this time, bit errors are allowed on the link and the receiver starts from an initial configuration and converges to a suitable solution by adapting each of its several equalizers. In one embodiment shown in
Comparator 212 includes a threshold adjustment input 213 and performs a comparator threshold function to output a signal 214 that is feed into a DFE 216 including a DFE adjustment signal 217. DFE 216 then outputs an equalized signal 218. As further shown by feedback loops 220 and 222, signals 214 and 218 are fed back through an amplifier 224 and provided as second inputs to summing blocks 208 and 210.
In one embodiment the receiver's firmware samples the current die temperature as a proxy for the platform temperature, assuming that the die temperature correlates to the platform temperature and the interconnect loss. In other embodiments, one or more temperature sensors may be used to measure the platform temperature (e.g., a temperature sensor on a PCB). Based on the current temperature, the firmware programs the VGA gain. When the temperature is below a certain threshold, the VGA gain is set to a low value. When the convergence logic of the receiver takes this into account, the other equalizers will converge on a link solution that accommodates this lower VGA gain. For example, the CTLE will select a higher gain solution for the overall link because while the VGA gain is initialized at a lower value, the requirements and the method for the link solution remain the same. Once the link solution is determined, the link-up state is established and the link transitions from the link training phase to the continuous adaptation phase during which data is transmitted over the link.
In a block 304, the firmware gain for VGA 206 is set based on the current temperature. In one embodiment the VGA gain is set using a table of preset gain vs. temperature values. In another embodiment, a gain vs. temperature function is used.
In a block 306, CTLE 204 is adjusted to search for CDR (Clock and Data Recovery) lock/edges at comparator 212. In a block 308, VGA 206 is adjusted further to achieve comparator thresholds (e.g., Least Mean Square (LMS) of error). In a block 310, DFE 216 is adjusted to achieve DFE convergence. As depicted by a decision block 312, DFE convergence is performed until bit and word lock convergences is completed. As further depicted, the operations in blocks 306, 308, 310, and decision block 312 are performed during link training.
Following link training, the flow enters a “Link Up” start block 314 and begins to transmit data over the link. As depicted in a block 316, the VGA gain and DFE are continuously updated based on changes to the current temperature. As further shown by a decision block 318, the link is monitored to ensure it is operating within the link margins. If a link margin is exceeded or otherwise a link failure is detected, decision block 318 determines the link is down (or otherwise should be reset) and the logic returns to block 302 to reinitialize the link.
During the continuous adaptation phase bit errors are not permitted. The receiver continues to adjust some of the equalizer coefficients that can be adjusted without causing errors. This includes the VGA. Since it was initially targeted to a low gain solution (in one example), the VGA has reserved additional gain capability to continuously and gradually adjust to any potential increase in channel loss in the system as the temperature rises.
Similarly, if the current die temperature is in the mid-range, between a low threshold and a high threshold, the VGA gain initial target will be for a mid-gain value, such that through continuous adaptation the gain can be increased or decreased (attenuates) compensating for any changes in the interconnect loss.
In this example the VGA gain initial target is programmed based on the temperature. If necessary, the CTLE solution could also be bounded to further preserve margin for future link changes.
It this example, setting the initial VGA gain based on temperature means that when the CTLE is initially adapted, the current VGA setting influences the convergence solution of the CTLE, and then once the CTLE is tuned appropriately further adaptation of the VGA is performed. The sequence of the adaptation of the individual equalization tools is important to this solution, in this example. Other sequences are also applicable and do not change the nature of the temperature dependent methodology.
As illustrated in the following figures, the continuous receiver adaptation approach may be used for different use cases and environments. Under one example use case, the approach is used to compensate for post-training insertion loss variation in signals transmitted via traces in a PCB. As an extension of this approach, the compensation may also apply to the full transmission route between two link partners, including PCB trace portions of the signal path in PCBs in the two link partners as well as the portion of the signal path comprising a cable or the link connected between the (link interfaces of the) link partners.
In some embodiments, the approach is implemented in an Ethernet PHY. The Ethernet PHY may be implemented in a stand-alone component, such as a NIC (Network Interface Controller) chip, or may be implemented as a circuit block in a System on a Chip (SoC). An example of an SoC 400 including an integrated Ethernet PHY is shown in
SoC 400 includes multiple processors 402, each including a plurality of cores 404 and a Mid-Level Cache (MLC) 406. Processors 402 are coupled to a scalable coherent fabric 408 that includes multiple cache agents (CHAs) 410, each having a home agent 412, a snoop filter 414 and a Last-Level Cache (LLC) 416. In one embodiment each processor 402 is associated with a respective CHA 410. Other configuration may also be used, as will be recognized by those skilled in the art.
Various components are coupled to scalable coherent fabric 408 including a PCIe root port (RP) 418, a Direct Memory Access (DMA) controller 420, and an integrated memory controller 422 having one or more channels such as depicted by two channels Ch0 and Ch1. An Input/Output (I/O) fabric 424 is also connected to scalable coherent fabric 408, as is a network interface and scheduler 426 and an Intel® QuickAssist technology block 428.
Multiple I/O components are coupled to I/O fabric 424 including a pair of PCIe RPs 430, a pair of SATA (Serial ATA) controllers 432, and a USB (universal serial bus) block 434. PCIe RPs 430 and SATA controllers 432 are coupled to a Flexible I/O Adapter (FIA) and high-speed IO (HSIO) block 436, which is configurable to output various combinations of signals received from PCIe RPs 430, and SATA controllers 432.
Network interface and scheduler 426 and QuickAssist technology block 428 are coupled to a flexible packet processor and switch 438 that includes multiple 4-lane links coupled to an Ethernet multirate PHY 440. In the illustrated embodiment, there are four 25 Gigabits per second (25 G) and one 10 G link. However, this is merely exemplary and non-limiting, as other combinations of links may be used.
The various I/O components and interfaces are coupled to I/O pins or BGA balls 442 that are disposed beneath SoC 400. For example, SoC 400 may be packaged as a Pin Grid Array (PGA) in which case I/O pins are used or a Ball Grid Array (BGA) in which case solder balls are used. (It is further noted an SoC may be installed in a PGA or BGA chip carrier, as well).
In the Figures herein, a number above a slash, such as 8/represents the number of lanes for a given link. For single-ended signals there is an I/O pin or BGA ball for each lane (for multi-lane links) in each direction (2 wires per bi-directional lane or link). For differential signals there would be a pair of I/O pins or BGA balls for each lane, per each direction (4 wires per bi-directional lane). For an 8-lane link (four lanes each direction), there would be 16 I/O pins or BGA balls 442. Channels Ch0 and CH1 of integrated memory controller 422 also employ a multilane link or bus (not shown).
As shown in the cross-section view of
As shown above the components, the signal path includes a PCB.a segment, a cable & mated connectors segment, and a link partner segment. Depending on the environment and where the compute platforms are located, the continuous adaptive equalization approach may be applied to the PCB.a segment, or it may be applied to the entire signal path or a portion of the signal path in addition to the PCB.a segment.
The configuration shown in
Under the configuration shown in
A genericized PHY chip/block 700 that is illustrative of the basic circuitry that may be used in a variety of different types of PHYs is shown in
PHY chip/block 700 includes a transmit (Tx) PHY protocol stack 702 and a receive (Rx) PHY protocol stack 704. These Tx and Rx PHY protocol stacks will have one or more layers and/or sublayers for implementing a Tx and Rx PHY corresponding to a given protocol. For example, a high-speed Ethernet link such as a 100GBase, 200GBase, or 400GBase link may have a particular set of layers including a Physical Coding Sublayer (PCS) sublayer, a Physical Media Attachment (PMA) sublayer, a Physical Media Dependent (PMD) sublayer, an AN (analog) sublayer, etc. Some protocols may further use a Forward Error Correction (FEC) sublayer. For protocols such as PCIe and CXL, different vendors may implement different SERDES (Serialize/Deserialize) schemes in addition to standardized PCIe and CXL layers/sublayers.
PHY chip/block 700 is configured to support a link with N lanes per direction, where N is an integer (e.g., 1, 2, 4, 8, 16, etc.) Each of the N lanes employs an instance of the same circuit, where one instance of the circuitry is depicted in
PHY chip/block 700 further includes firmware 710 comprising a continuous adaptive equalization module 712 and a link training module 714. In the illustrated example, continuous adaptive equalization module 712 and a link training module 714 comprise firmware instructions that are executed a processing element (PE) 716. Optionally, all or a portion of the logic for implementing continuous adaptive equalization and link training may be implemented via other forms of embedded logic, such as programmable logic (e.g., a field programmable gate array (FPGA)) and/or fixed logic (ASIC, DSP, etc.). For embodiments employing firmware instruction executed on some type of processing element, there will generally be one set of firmware instructions regardless of the number of lanes. At the same time, it is noted that separate instruction threads may be executed for each receive lane. For embedded logic employing programmable logic, there may generally be a set of programmable logic per instance of receive lane.
As further shown in
As shown by the freezing and sun icons, the street cabinet is exposed to local weather conditions. Depending on the location, the temperature might range from −40° C. to 40° C. (−40° F. to 104° F.). In some locations, the high end of the temperature range might be even higher. Combine the external environment temperature conditions with heat generated by the one or more compute platforms in street cabinet 800, and the temperature range might extend up to 90° C.
Under a daily workload cycle, the workload might vary during the day, with low utilization at night. To accommodate for low utilization, the compute platforms(s) may be put into low-power states, such as by putting the SoC and/or its processors and other circuitry into a sleep state. Thus, overnight the temperature of PCB 502 might reach the ambient weather temperature. During high utilization, the compute platforms will generate heat that may raise the temperature within the street cabinet to much higher temperatures than the ambient weather temperature.
As a further consideration, a deployment in a telecom or other high-demand environment requiring 99.9999 uptime may necessitate that the platform and its links be operational for many years without being brought down. Thus, the temperature range while a given platform is operating (without being taken offline) could easily exceed 100° C.
Under the continuous adaptive equalization approach, equalization components in the receiver are adjusted to maintain adequate margins in view of increased or decreased insertion losses caused by temperature changes in the platform PCBs and (optionally) cabling. This approach enables the link to operate correctly (e.g., without bit errors) over wide temperature ranges.
Although some embodiments have been described in reference to particular implementations, other implementations are possible according to some embodiments. Additionally, the arrangement and/or order of elements or other features illustrated in the drawings and/or described herein need not be arranged in the particular way illustrated and described. Many other arrangements are possible according to some embodiments.
In each system shown in a figure, the elements in some cases may each have a same reference number or a different reference number to suggest that the elements represented could be different and/or similar. However, an element may be flexible enough to have different implementations and work with some or all of the systems shown or described herein. The various elements shown in the figures may be the same or different. Which one is referred to as a first element and which is called a second element is arbitrary.
In the description and claims, the terms “coupled” and “connected,” along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. Rather, in particular embodiments, “connected” may be used to indicate that two or more elements are in direct physical or electrical contact with each other. “Coupled” may mean that two or more elements are in direct physical or electrical contact. However, “coupled” may also mean that two or more elements are not in direct contact with each other, but yet still co-operate or interact with each other. Additionally, “communicatively coupled” means that two or more elements that may or may not be in direct contact with each other, are enabled to communicate with each other. For example, if component A is connected to component B, which in turn is connected to component C, component A may be communicatively coupled to component C using component B as an intermediary component.
An embodiment is an implementation or example of the inventions. Reference in the specification to “an embodiment,” “one embodiment,” “some embodiments,” or “other embodiments” means that a particular feature, structure, or characteristic described in connection with the embodiments is included in at least some embodiments, but not necessarily all embodiments, of the inventions. The various appearances “an embodiment,” “one embodiment,” or “some embodiments” are not necessarily all referring to the same embodiments.
Not all components, features, structures, characteristics, etc. described and illustrated herein need be included in a particular embodiment or embodiments. If the specification states a component, feature, structure, or characteristic “may”, “might”, “can” or “could” be included, for example, that particular component, feature, structure, or characteristic is not required to be included. If the specification or claim refers to “a” or “an” element, that does not mean there is only one of the element. If the specification or claims refer to “an additional” element, that does not preclude there being more than one of the additional element.
As discussed above, various aspects of the embodiments herein may be facilitated by corresponding software and/or firmware components and applications, such as software and/or firmware executed by an embedded processor or the like. Thus, embodiments of this invention may be used as or to support a software program, software modules, firmware, and/or distributed software executed upon some form of processor, processing core or embedded logic a virtual machine running on a processor or core or otherwise implemented or realized upon or within a non-transitory computer-readable or machine-readable storage medium. A non-transitory computer-readable or machine-readable storage medium includes any mechanism for storing or transmitting information in a form readable by a machine (e.g., a computer). For example, a non-transitory computer-readable or machine-readable storage medium includes any mechanism that provides (e.g., stores and/or transmits) information in a form accessible by a computer or computing machine (e.g., computing device, electronic system, etc.), such as recordable/non-recordable media (e.g., read only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.). The content may be directly executable (“object” or “executable” form), source code, or difference code (“delta” or “patch” code). A non-transitory computer-readable or machine-readable storage medium may also include a storage or database from which content can be downloaded. The non-transitory computer-readable or machine-readable storage medium may also include a device or product having content stored thereon at a time of sale or delivery. Thus, delivering a device with stored content, or offering content for download over a communication medium may be understood as providing an article of manufacture comprising a non-transitory computer-readable or machine-readable storage medium with such content described herein.
The operations and functions performed by various components described herein may be implemented by firmware running on a processing element, via embedded hardware or the like, or any combination of hardware and software/firmware. Such components may be implemented as software/firmware modules, hardware modules, special-purpose hardware (e.g., application specific hardware, ASICs, DSPs, etc.), embedded controllers, hardwired circuitry, hardware logic, etc. Software content (e.g., data, instructions, configuration information, etc.) may be provided via an article of manufacture including non-transitory computer-readable or machine-readable storage medium, which provides content that represents instructions that can be executed. The content may result in a computer performing various functions/operations described herein.
As used herein, a list of items joined by the term “at least one of” can mean any combination of the listed terms. For example, the phrase “at least one of A, B or C” can mean A; B; C; A and B; A and C; B and C; or A, B and C.
The above description of illustrated embodiments of the invention, including what is described in the Abstract, is not intended to be exhaustive or to limit the invention to the precise forms disclosed. While specific embodiments of, and examples for, the invention are described herein for illustrative purposes, various equivalent modifications are possible within the scope of the invention, as those skilled in the relevant art will recognize.
These modifications can be made to the invention in light of the above detailed description. The terms used in the following claims should not be construed to limit the invention to the specific embodiments disclosed in the specification and the drawings. Rather, the scope of the invention is to be determined entirely by the following claims, which are to be construed in accordance with established doctrines of claim interpretation.