1. Technical Field of the Invention
This disclosure relates generally to memory systems, components, and methods and more particularly to a method and apparatus for providing logic analyzer observability into a buffered memory channel, as well as providing selected logic analyzer functionalities.
2. Description of the Related Art
Digital processors, such as microprocessors, use a computer memory subsystem to store data and processor instructions. Some processors communicate directly with memory, and others use a dedicated controller chip, often part of a “chipset,” to communicate with memory.
Conventional computer memory subsystems are often implemented using memory modules. Referring to
Simulations have shown that for applications of 2 to 4 memory modules (in particular, dual inline memory modules, or DIMMs) per memory channel, similar multi-drop bus technology reaches a maximum bandwidth of 533-667 MT/s (mega-transactions/second), or 4.2-5.3 GB/s (gigabytes/second) for an eight byte wide DIMM. Achieving the next significant level, 800 megatransfers/second (MT/s) and beyond, will be difficult if not impossible with the multi-drop bus topology.
To overcome the obstacles for accommodating multiple memory modules at high transfer rates described above, a buffered memory module subsystem with a “point-to-point” topology has been proposed.
Referring to
Buffered memory module 140 is typical of the memory modules. A memory module buffer (MMB) 146 connects module 140 to a host-side (upstream) memory channel 132 and to a downstream memory channel 142. A number of Dynamic Random Access Memory chips 144, or DRAMs 144 connect to memory module buffer 146 through a memory device bus (not shown in
Each memory channel 112, 122, 132, and 142 in
Although the buffered memory module subsystem 100 of
Each of the memory channels 112, 122, 132, and 142 is composed of two uni-directional buses for data traffic in both directions. That is, commands and data can travel in the direction away from the host 110 and status and data can travel towards the host 110. For convenience, the movement of command and data through the memory channels in a direction away from the host 110 will henceforth be referred to as “southbound.” Likewise, movement of status and data through the memory channels in the direction toward the host 110 will be referred to “northbound.” It should be apparent that these terms have nothing to do with the actual geographic orientation of the memory channels.
The actual signal paths that make up the memory channels are implemented using high-speed serial differential signals. The number of differential signals in the southbound direction may be different than the number of signals in the northbound direction.
In normal mode of operation, host 110 accesses the memory space of module 140 by sending a commands and data, addressed to memory module 140, southbound on host memory channel 112. The MMB 146 of buffered memory module 120 receives the commands/data and resends it on memory channel 122 to the MMB 146 of buffered memory module 130. The MMB 146 of buffered memory module 130 next receives the command and resends it on memory channel 132 to MMB 146 of buffered memory module 140. On module 140, MMB 146 detects that the command is directed to it, decodes it, and transmits DRAM commands and signaling to the DRAMs (e.g., 144) controlled by that buffer. When a response is expected (such as when a read is requested), MMB 146 receives the data from the DRAMs, encodes/formats the data, and sends it backwards (northbound) along the memory channels 132, 122, and 112, repeated without modification by the MMBs 146 on modules 130 and 120, to host 110.
As in all memory subsystems, for validation and debug purposes it is extremely important to provide the ability to observe, record, and trigger on events in logical transactions across system interconnects. However, in the buffered memory module subsystem of
Specifically, this challenge arises because, unlike the memory subsystem of
To provide a capability for solving these and/or similar problems, some embodiments of the invention provide a Logic Analyzer Interface (LAI) interposer assembly 175, as illustrated in
In
A DIMM 120 is inserted in the upper connector 161 that is located on the LAI 175. The DIMM 120 may also be referred to as the interposed DIMM 120. The upper connector 161 that attaches the interposed DIMM 120 to the LAI 175 is illustrated as a standard vertical connector, but alternate connector styles such as angled or straddle mount might be used to minimize possible physical interference problems. An LAI interposer assembly 175 may be installed in the first DIMM socket 160, as shown in
Standard logic analyzer high density probes 170 are attached to the LAI 175 with the probe sockets 165. During operation of the LAI 175, the MMB 147 included on the interposer operates in an LAI mode, while the other MMBs 146 on the interposed DIMM 120 and the DIMMs 130, 140, 150 operate in normal mode.
Attaching an interposed DIMM 120 to the LAI 175 is advantageous because the same number of normally operating MMBs 146 are available as when the LAI 175 is not installed to instrument the buffered memory module subsystem. The LAI 175 will be made to appear transparent to normal traffic on the memory channel.
For clarity, the arrows in
Conversely, the northbound data being sent from DIMM 130 arrives at interposed DIMM 120, which retransmits the data to the MMB 147. In other words, for the configuration shown in
The northbound data in the MMB 147 operating in LAI mode is handled differently compared to the other MMBs 146 operating in normal (DRAM buffer) mode. In the normal mode the MMBs 146 capture, modify as necessary, and retransmit northbound data to the next component (MMB 146, MMB 147, repeater (not shown), or host 110) in the chain. However, the MMB 147 does not alter the content of the received northbound data before sending it northbound.
In addition to the operation described above, the MMB 147 operating in LAI mode may demultiplex and transmit southbound and northbound data at a reduced transfer rate to the probe sockets 165, where it is detected by the logic analyzer probes 170 for input to a logic analyzer mainframe.
The signals shown outside of the dashed box represent links whose pin assignments in LAI mode for MMB 147 compared to DRAM buffer mode for the MMBs 146 of
The signals shown inside the dashed box represent signaling usages that have different connections in LAI mode for the MMB 147 as compared to the normal (DRAM buffer) mode for the MMBs 146 of
In LAI mode for MMB 147 the southbound and northbound signals are demultiplexed and transmitted by cables 170 to the attached logic analyzer at the same reduced transfer rate (as in normal mode) using the signals S[59:00] and N[83:00], respectively. The dotted arrows shown inside the MMB 147 represent this demultiplexing process. The DRAM differential signal CLK[p,n], is sent as is to the logic analyzer from MMB 147.
Furthermore, to be effective in analyzing the collected link traces, the data provided to the logic analyzer includes not only a demuxed link traffic, but also several derived signals. This information is derived by the MMB 147 and is represented by eleven trigger signals TRIG[10:0], the FRAME signal, the EV[3:0] signals, and the QUAL signal. In cases where multiple interposers 175 are attached to the buffered memory module subsystems, there is also a derived set of four shared signals EV[3:0] between the MMB 147s in the form of cross triggers. The EV[3:0] signals provide cross-triggering information with finer timing granularity than the logic analyzer can achieve acting alone. The FRAME signal indicates the beginning of a frame, as defined by the high speed protocol. The QUAL signal indicates filtering (qualified storage) opportunities. The TRIG[10:0] signals sends trigger signals to the logic analyzer to indicate a debug event in the high speed link. These are just some examples, other embodiments of the invention may provide other derived information to the logic analyzer. The MMB 147 also produces the signal MODE to indicate the training status of the high speed channel 112, 122, 132, 142, i.e. the link has gone through the initialization state where the training of the link is complete.
This ability to provide derived information to the logic analyzer is particularly advantageous. Critical logic analysis may now occur on the MMB 175, which could not be implemented in the logic analyzer itself. Therefore, despite the inability of logic analyzer designs to scale in performance to link debug complexity, the features in the MMB 147 affords a high degree of overall logic analysis and debug capabilities. The type of derived information described above are just a few examples, other embodiments of the invention may provide other derived information to the logic analyzer. Likewise, other embodiments of the invention may share more or fewer signals EV across multiple MMBs 147 when multiple LAIs 175 are attached to a memory system.
In
The retiming circuit 234 of the southbound data path retransmits data destined for DIMMs to the south. The retiming/merge circuit 244 of the northbound data path performs a similar task for northbound data, but in the normal mode it must also interleave data from the DRAM devices into the northbound data stream. Both the southbound and northbound data path have a demux/deskew circuit 232 and 242, respectively. In normal mode and in LAI mode, these circuits select the appropriate channels of the northbound and southbound data streams. The northbound delay pipeline 240 and the southbound delay pipeline 230 also demultiplex the channel information to slow the data transfer rate for output to the attached logic analyzer during LAI mode.
In LAI mode, the control/status register 200 is accessed by the SMB bus to set modes and parameters for all MMB modes, including LAI features. In turn, the Events Selection & Response logic 210 and the Protocol Unwrapping & Pattern Recognition logic 220 are controlled by the control/status register 200. Logic 210 and logic 220 derive the TRIG, FRAME, EV, and QUAL signals that were discussed above with respect to
An important point illustrated by
Having described some embodiments of the invention, several contemplated examples of tasks that the LAI 175 could perform will be described below. This description does not limit embodiments of the invention to only those tasks, as it will be recognized by those of skill in the art that there are other tasks that embodiments of the invention are particularly well-suited for.
In a link protocol validation/debug scenario, the LAI 175 may capture traces of buffered memory module links that are performing simple to complex operations. The traces are used to debug low level link protocol/operations, such as link initialization, retraining, power transitions, resets, and error recovery. For this purpose the host 110 would usually initiate traffic as a result of diagnostic or focus test software execution, rather than operating system/application activity, since this would allow specific behaviors to be produced repeatedly and without significant interference due to other link activity. Analysis of the traces could be primarily manual, since it is expected to be fairly simple, although automated checkers could also be employed to check rigorously for protocol flaws.
In a system initialization/debug scenario, the captured traces would be used to debug/validate the BIOS (basic input/output system) configuration of MMB register write and read sequences as part of system initialization. Other software configurations could be debugged/validated as well.
In a host memory controller logic interaction with DRAM scenario, traces of either tests or normal execution would be captured to enable debug of DRAM control sightings where the memory controller and/or buffer exhibit unexpected or failing behavior during complex controller/buffer/DRAM interactions. MMB traffic would reveal the exact sequences and timing of interactions as basis for manual analysis in determining root cause of the faults.
In a system level debug scenario, traces of MMB traffic under normal operating system and application execution might reveal important clues about misbehavior of logic elsewhere in the system. In this case the interaction across the MMBs and interaction with the DRAMs all work flawlessly, but the content and/or sequencing of the traffic on the link can be used to provide indirect observability of system logic behavior.
In a logic dump mode, the host 110 would remove the port attached to the LAI 175 through the socket 160 from normal usage and instead pass selected internal logic values across the port specifically so they could be captured using the LAI. This would allow high bandwidth debug signals to be traced without adding dedicated debug bus pins to the memory control chip.
One of ordinary skill in the art will recognize that the concepts taught herein can be tailored to a particular application in many other advantageous ways. In particular, those skilled in the art will recognize that the illustrated embodiments are but one of many alternative implementations that will become apparent upon reading this disclosure. For instance, a wide variety of memory tests may be envisioned using the concepts disclosed herein, only a few of which are discussed specifically herein. The particular test sequence or sequences initiated using an embodiment is important from a testing viewpoint, but all such tests can be performed within the scope of the appended claims.
The preceding embodiments are exemplary. Although the specification may refer to “an”, “one”, “another”, or “some” embodiment(s) in several locations, this does not necessarily mean that each such reference is to the same embodiment(s), or that the feature only applies to a single embodiment.
Many of the specific features shown herein are design choices. Channel and bus widths, signaling frequencies, transfer rates, the number and type of memory modules, the number and type of memory chips on a memory module, control bus protocols, etc., are all merely presented as examples. For instance, memory modules can have multiple ranks of memory and/or multiple stacks of memory. Likewise, functionality shown embodied in a single integrated circuit or functional block may be implemented using multiple cooperating circuits or blocks, or vice versa. Such minor modifications are encompassed within the embodiments of the invention, and are intended to fall within the scope of the appended claims.