Memory error detection

Information

  • Patent Grant
  • 11928020
  • Patent Number
    11,928,020
  • Date Filed
    Tuesday, January 10, 2023
    a year ago
  • Date Issued
    Tuesday, March 12, 2024
    2 months ago
Abstract
Systems and methods are provided for detecting and correcting address errors in a memory system. In the memory system, a memory device generates an error-detection code based on an address transmitted via an address bus and transmits the error-detection code to a memory controller. The memory controller transmits an error indication to the memory device in response to the error-detection code. The error indication causes the memory device to remove the received address and prevent a memory operation.
Description
BACKGROUND

The present disclosure relates generally to memory systems. More particularly, the present disclosure relates to detection and correction of errors within a memory system.


As the data transfer rate in computer systems becomes progressively faster, modern memory systems are more prone to transmission errors. Noise, crosstalk, inter-symbol interference, among other factors, can significantly impair signal quality, especially as data rates approach multiple Gbps.





BRIEF DESCRIPTION OF THE FIGURES

This disclosure is illustrated by way of example, and not by way of limitation, in the accompanying drawings. Like reference numerals refer to similar elements.



FIG. 1 illustrates an exemplary memory system which facilitates error detection for both data and address, in accordance with one embodiment of the present invention.



FIG. 2A presents a block diagram illustrating the process of generating EDC codes based on both data and address, in accordance with one embodiment of the present invention.



FIG. 2B presents a block diagram illustrating the process of generating EDC codes based on both data and address, in accordance with one embodiment of the present invention.



FIG. 3 presents a flowchart illustrating the operation of a dynamic random-access memory (DRAM) which facilitates EDC for both address and data, in accordance with one embodiment of the present invention.



FIG. 4 presents a flowchart illustrating the operation of a memory controller which facilitates EDC for both address and data, in accordance with one embodiment of the present invention.



FIG. 5 presents an exemplary timing diagram illustrating the operation between a memory controller and a DRAM to facilitate address and data EDC, in accordance with one embodiment of the present invention.



FIG. 6 presents an exemplary timing diagram illustrating the operation between a memory controller and a DRAM to facilitate address and data EDC, in accordance with one embodiment of the present invention.


The following description is presented to enable any person skilled in the art to make and use embodiments disclosed herein, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of the present disclosure. Thus, the present disclosure is not limited to the embodiments shown, but is to be accorded the widest scope consistent with the disclosure.





DETAILED DESCRIPTION

In a memory system, address errors can be more detrimental than data errors, because a write operation with the wrong address can corrupt previously stored data. Embodiments of the present invention provide a mechanism for checking and correcting errors in both address and data transmission between the memory controller and DRAM. The DRAM computes an error-detection code (EDC) based on the address and/or data received from the controller, and transmits this EDC code back to the controller. The controller then determines whether the EDC code is consistent with the previously transmitted address and/or data. If an error is detected, and if the error corresponds to the address for a write operation, the controller can send an error indication to the DRAM, so that the DRAM can cancel the erroneous write command. In this way, the memory system can ensure both error-resistant data and address transmission.


Embodiments of the present invention also provide various configurations to optimize different characteristics of the memory system. For example, the DRAM can compute the address-based EDC code as soon as the address is received, thereby reducing the amount of latency incurred to the data due to buffering. The DRAM also subjects the received address (and optionally the received data) to additional delays, so that there is sufficient time for the controller to detect an error based the EDC code and issue an error indication. The total delay incurred to the address and optionally the data is sufficient such that the operation corresponding to an erroneous address can be precluded from being completed. This total delay may be more than the normal WRITE or READ latency. In further embodiments, the controller can withhold the data transmission until it receives and confirms the EDC code for the corresponding address, which reduces the amount of in-DRAM data buffering.


In the present disclosure, a “memory controller” or “controller” refers to a circuit, chip, or device on a computer's motherboard or CPU die which manages the flow of data going to and from the memory. A “DRAM” refers a dynamic random-access memory. Embodiments of the present invention are not limited to DRAM-specific applications. These embodiments can also be applied to various memory systems based on dynamic or static memories. The terms “data transmission” and “data” refer to the payload data transferred between the controller and DRAM. Although the address and command bits can also be broadly referred to as data, the term “data” in this disclosure, unless specified otherwise, refers to the payload bits which are separate from address and command bits. An “error indication” is a command or a signal transmitted by the controller to a DRAM, so the DRAM can prevent a memory operation corresponding to an erroneously received address from being completed. An error indication can be a command including one or more bits, transmitted via or outside of the command bus. The error indication can also be a separate signal transmitted via one or more separate pins.



FIG. 1 illustrates an exemplary memory system 100 which facilitates error checking and correction for both data and address, in accordance with one embodiment of the present invention. Memory system 100 includes a DRAM controller 110, a DRAM 140, a command bus 132, an address bus 134, a data bus (DQ bus) 136, and an error-detection code (EDC) bus 138. DRAM controller 110 includes command generation circuitry 113, address generation circuitry 112, write data circuitry 114, read data circuitry 116, a write data buffer 122, an address buffer 124, a read/write selector 120, and error detection circuitry 118. DRAM 140 includes command/error handling circuitry 142, an address buffer 146, a data buffer 147, a read/write selector 148, and a cyclic-redundancy-check (CRC) code generator 150.


During operation, command generation circuitry 113 generates a memory command, e.g., a read or write command, and places the command on command bus 132. At approximately the same time, address generation circuitry 112 places an address corresponding to the memory command on address bus 134. If the command is for a write operation, write data circuitry 114 places the data for the write operation in buffer 122. In general, buffer 122 is used to delay the data transmission after the write command and address have been sent to DRAM 140, due to the write latency present in DRAM 140, i.e., the delay between the time when DRAM 140 receives the command and address and the time when DRAM 140 is ready to accept data on DQ bus 136. If command generation circuitry 113 generates a read command, a certain amount of data buffering might also be necessary to accommodate the read latency in DRAM 140.


After command generation circuitry 113 places the command on command bus 132, command/error handling circuitry 142 in DRAM 140 receives the transmitted command. In addition, the corresponding address is received and placed in buffer 146. Typically, buffer 146 buffers the received address bits because the complete address (e.g., bank address, row address, and/or column address) may be transmitted over multiple clock cycles. Furthermore, buffer 146 ensures that the address information remains available until DRAM 140 is ready to receive the write data.


Subsequently, after a certain delay (e.g., the write latency), the corresponding data is received and buffered in buffer 147. Buffer 147 buffers the received data so that buffer 147 can be later frozen or the data can be discarded if DRAM controller 110 identifies an error in the transmitted address. At a given time, the data present on DQ bus 136 might not correspond to the address and/or command present on address bus 134 and command bus 132, due to the pipelined operation of memory system 100.


The received address is first buffered by buffer 146 and then sent to CRC generator 150. In addition, the write data is sent to CRC generator 150 via read/write selector 148. Read/write selector 148 is controlled by command/error handling circuitry 142 based on the received command. If the command indicates a read operation, read/write selector 148 allows the outgoing data from the DRAM core to be sent to CRC generator 150. If the command indicates a write operation (which is the case in the current example), read/write selector 148 allows the write data to be sent to CRC generator 150.


CRC generator 150 produces a CRC code based on both the address and write data. The CRC code is then transmitted via EDC bus 138 to DRAM controller 110. CRC generator 150 can concatenate the address with the data to produce an input to the CRC computation. Other methods of generating the CRC code based on both the address and data are also possible.


In this example, the EDC code generated by DRAM 140 is a CRC code. A CRC code (or “CRC”) is an error-detecting code. Its computation resembles a long division operation in which the quotient is discarded and the remainder becomes the result, with the distinction that the arithmetic used is the carry-less arithmetic of a finite field. The length of the remainder is less than or equal to the length of the divisor, which therefore determines how long the result can be. The definition of a particular CRC specifies the divisor to be used. The CRC code generated by CRC generation circuitry 150 can be based on the polynomial expression X8+X2+X1+1. In general, a respective EDC pin in EDC bus 138 is used to carry the CRC code for the data transmitted on a number (e.g., eight) of DQ pins within DQ bus 136. There is one EDC pin for every eight DQ pins, and the CRC code is transmitted via the EDC pin over eight data-clock periods, which correspond to eight data bits transmitted over each DQ pin. That is, an eight-bit serial CRC code is transmitted over one EDC pin for 64 bits of data transmitted via eight DQ pins over eight data-clock cycles. Other EDC code schemes can also be used.


After DRAM controller 110 receives the CRC code, error detection circuitry 118 compares the CRC code with a local CRC code generated based on the buffered write data and address. Alternatively, error detection circuitry 118 can checksum the buffered write data and address concatenated with the same CRC polynomial, and determines whether the checksum result is zero. The write address is buffered in buffer 124 to compensate for the delay between the time when the address is transmitted to DRAM 140 and the time when the CRC code is received and checked at DRAM controller 110. Error detection circuitry 118 also takes as input the write or read data which corresponds to the CRC received from DRAM 140. Controller 110 maintains a set of state information to keep track of the address and data (which may be for different memory operations) corresponding to the received CRC code. The state information may be maintained by command generation circuitry 113. Upon receiving the CRC code from DRAM 140, command generation circuitry 113 determines whether the received CRC code corresponds to write or read data, and sends a signal (denoted as “R/W”) to read/write selector 120. In response, read/write selector 120 selects the appropriate data (i.e., write or read) to send to error detection circuitry 118.


If the received CRC code is consistent with the selected data and address, controller 110 does not send any error indication to DRAM 140. As a result, after a predetermined period, DRAM 140 can automatically proceed with the write operation based on the address stored in buffer 146 and the corresponding write data.


Instead of transmitting an explicit error indication, controller 110 can transmit an explicit “commit” indication to DRAM 140 when the CRC code is consistent with the buffered data and address. This commit indication can be a command transmitted via or outside the command bus, or a signal transmitted over one or more separate pins. In response to the commit indication, DRAM 140 can proceed to commit the memory operation to the DRAM core. If the CRC code is inconsistent with the buffered data and address at controller 110, no commit indication is sent. As a result, after a certain time-out period, DRAM 140 can prevent the memory operation from being completed if no commit indication is received.


If error detection circuitry 118 detects an error based on the received CRC code, it can cause command generation circuitry 113 to generate and transmit an error indication to DRAM 140 via command bus 132. (The error indication can also be transmitted via one or more dedicated pins separate from command bus 132.) In response, command/error handling circuitry 142 can cause buffer 146 to remove the stored address corresponding to the error, and optionally cause buffer 147 to remove the corresponding write data. Buffer 146 can include a number of latches, and removing the stored address involves resetting these latches. In addition, in response to the error indication, command/error handling circuitry 142 can prevent the memory operation corresponding to the erroneously received address from being completed. For example, a commit of the memory operation to the memory core can be deliberately delayed sufficiently long using buffer 146 and/or buffer 147, such that controller 110 has time to detect errors and, if appropriate, abort the associated command. Subsequently, command generation circuitry 113 re-issues the corresponding memory commands. It is possible that the CRC code which identifies an error is derived from a write address and read data. In this case, the controller does not need to determine the source of the error (i.e., whether the error was incurred to the write address or read data) and can simply re-transmit all the transactions to which the erroneous CRC code corresponds.


Depending on the pipelined operations, DRAM 140 may discard a number of “in-flight” transactions (including command, address, and optionally data) received after the memory operation corresponding to the erroneous address. This approach can avoid read-after-write and write-after-write conflicts that could otherwise occur and also corrupt data in some way. In general, command/error handling circuitry 142 can discard the memory command associated with the error (or commands associated with the error if the error corresponds to more than one command due to pipelining). Command/error handling circuitry 142 can further discard any subsequently received command that can present a hazardous conflict with the command in error (for example, a read-after-write or write-after-read situation). Alternatively, command/error handling circuitry 142 can discard all the commands it has received and buffered after the erroneous command and before the first retransmitted command.


The CRC code could be generated based on address and data which belong to two or more memory operations due to the pipelined operation between controller 110 and DRAM 140. As a result, command generation circuitry 113 may need to re-issue two memory commands. Furthermore, if a write command is re-issued, the write data previously buffered in buffer 122 is sent to DRAM 140 again, in response to a signal from error detection circuitry 118 (denoted by an arrow from error detection circuitry 118 to buffer 122).


To minimize the delay between address transmission and the receiving of CRC code at controller 110, DRAM 140 can be configured to compute the CRC code as soon as the address is received, without first buffering the received address (denoted by a dotted arrow from the address flow to CRC generator 150). The CRC code can also be generated based on the address and data bits for the same memory operation. This configuration may involve additional buffering of the address, because the data transmission typically occurs after the address transmission. It is, nevertheless, not required to generate the CRC code based on the address and data corresponding to the same memory transaction, so long as both the controller and DRAM have an agreement on how to map a CRC code to the appropriate memory transaction(s).


Controller 110 can withhold the write data transmission until a CRC code corresponding to the address for the same operation is received and confirmed. This configuration may require additional buffering of the write data within controller 110. This buffering can be accomplished by buffer 122.


The example described above is described in the context of a write operation. In the case of a read operation, read/write selector 148 allows the data read from the DRAM core to be sent to CRC generator 150. In controller 110, read/write selector 120 also allows the received read data to be sent to error detection circuitry 118. If an error is detected for a read command, only the read command and the corresponding address is re-transmitted to DRAM 140, since errors in a read command, address, or data do not potentially result in irrecoverable data corruption in DRAM 140.


As noted above, there is usually write and read latency within DRAM 140. To improve system efficiency and utilization, the transfer of addresses and data between controller 110 and DRAM 140 is typically pipelined. That is, at a given moment, the bits carried on address bus 134 and the bits carried on data bus 136 may correspond to different operations. Hence, CRC generator 150 can generate CRC codes based on address and data belonging to different operations.



FIG. 2A presents a block diagram illustrating the process of generating a CRC code with pipelined write operations, in accordance with one embodiment of the present invention. This example shows a number of write operations, the address and data of which (denoted as “A” and “D” respectively) are indexed by a subscript which indicates their order. These write operations are pipelined in such a way that the write data for a prior operation can be present on the DQ bus when the address for the current operation is on the address bus. In FIG. 2A, the write data is assumed to lag behind the address by k memory commands. For example, when address An is on the address bus, write data Dn-k is on the DQ bus. As a result, the CRC generator within the DRAM can generate a CRC code, CRCn, based on An and Dn-k. k can be any integer, such as 0, 1, 2, 3, etc.


The value of k represent the about of lag of data behind the corresponding address. In addition to the write or read latency inherent in the DRAM, k can also include artificially injected buffering delay to the data to accommodate the time required to generate the CRC code, process the CRC code at the controller, and to issue an error indication by the controller. This way, the buffered memory command can be aborted before the data is committed to the DRAM core.


When k is zero, the data on the DQ bus is substantially aligned with the address on the address bus and they correspond to the same memory operation. However, in such a configuration, the DRAM may need to provide sufficient in-DRAM buffering for the write data to accommodate the write latency. One advantage of having the data and address aligned (i.e., corresponding to the same memory command) for generating the CRC code is that when an error is detected, the DRAM only needs to abort and/or re-issue the memory command corresponding to that error. When the data and address are not aligned (i.e., k is greater than zero), the controller might not be able to determine whether the error in the CRC corresponds to the data or address. As a result, the DRAM might need to abort multiple commands corresponding to the address and data, respectively. The controller might also need to re-issue these commands.


It is possible that sometimes the DQ bus is not always busy, and a given CRC code might be generated based only on an address, whereas the data portion of the input for the CRC is of a null value. In this case, the DRAM only needs to abort the command corresponding to the address if an error is detected in the CRC code.


In the example in FIG. 2B, a CRC code, CRCn, is generated based on An, Dn-k, and Dn-k+1. In other words, a particular CRC is generated based on not only the address and data present on the address bus and DQ bus, but also on the data corresponding to the previous memory command. This method can allow the controller to determine whether an error in CRCn is caused by an error in An, Dn-k, or Dn-k+1. For example, if CRCn is in error, but CRCn+1 and CRCn+1 are both correct, then An is in error. If CRCn and CRCn+1 are both in error but CRCn+1 and CRCn+1 are correct, then it is possible that Dn-k and/or An are in error.


Other methods are possible to allow the controller to determine whether a CRC error is caused by an address error or data error. For example, the DRAM can generate CRC codes for the data and address separately, and transmit both CRC codes to the controller.


The description below provides further explanation of the operations of the DRAM and controller, and exemplary timing diagrams of different embodiments.



FIG. 3 presents a flowchart illustrating the operation of a DRAM which facilitates EDC for both address and data, in accordance with one embodiment of the present invention. In this example, the DRAM first receives a command and the corresponding address (operation 302). The DRAM can also receive write data if the command is for a write operation, wherein the write data may or may not correspond to the write command and address due to pipelined operations. Subsequently, the DRAM computes a CRC code based on the received address and optionally the data (operation 304). The data could be write data (incoming) or read data (outgoing). The DRAM then transmits the CRC code to the memory controller (operation 306).


Next, the DRAM determines whether it has received an error indication from the controller before a given time (referred to as “retire time”) (operation 308). If the DRAM does not receive the error indication before the retire time, the DRAM proceeds with normal operations, such as write, read, active, refresh, and other register-related operations (operation 310). If an error indication is received, the DRAM purges the address corresponding to the error from the address buffer and optionally one or more pending commands following the command in error (operation 314). Subsequently, the DRAM receives a re-issued write or read command, the address, and optionally data (if the re-issued command is for a write operation) from the controller (operation 316). The DRAM then proceeds to handle the re-issued command (operation 304).



FIG. 4 presents a flowchart illustrating the operation of a memory controller which facilitates EDC for both address and data, in accordance with one embodiment of the present invention. In this example, the memory controller first transmits a command and address, and optionally write data to the DRAM (operation 402). Subsequently, the controller receives from the DRAM a CRC code based on the transmitted address and optionally write or read data (operation 404). The controller then performs the checksum computation on the buffered address (and optionally buffered data) concatenated with the received CRC code (operation 406). A correct CRC code would produce zero as the checksum result. Alternatively, the controller can generate its own CRC code and compare it with the received CRC code. The address and optionally the data and command used to generate the CRC code should be the same as the ones used to check the CRC code. However, they do not need to be for the same memory transaction.


Next, the controller determines whether received CRC code is correct (operation 408). If the CRC codes is correct, the controller proceeds to the next memory operation (operation 410). If the CRC code is incorrect, the controller further determines whether the address associated with the error corresponds to a read or write operation (operation 412).


If the address error corresponds to a write operation, the controller issues an error indication to the DRAM (operation 414). This error indication notifies the DRAM that it should discard or abort the commands for the previously stored erroneous address, and, if the corresponding write data has already been transmitted to the DRAM, discard the buffered write data as well. Subsequently, the controller re-issues the write command corresponding to the erroneous address (operation 416). Optionally, if the CRC code from the DRAM is based on address and data belonging to two different memory operations, the controller can re-issue the write or read command corresponding to the data portion (as opposed to the address portion) associated with the error (operation 420).


In a further embodiment, instead of sending an explicit error indication to the DRAM when the CRC is incorrect, the controller can send an explicit commit indication when the CRC is correct. This commit indication allows the DRAM to proceed with the buffered command and complete the memory operation. Correspondingly, if the CRC is incorrect, the controller does not transmit the commit indication, and the DRAM discards the command in error (and optionally additional pending commands received after the command in error) after a predetermined time-out period.


If the erroneous data portion is for a previously issued write command, it is not necessary for the controller to issue an error indication to the DRAM to discard the erroneous data, because the erroneous data will still be written to the correct address and will not corrupt good data stored elsewhere. The controller only needs to re-issue the write command to place the correct data in the same address. To illustrate this situation more clearly, take the example in FIG. 2. Suppose address An and data Dn-k are for two separate write commands. Also, suppose CRCn indicates an error. The controller then assumes that both write commands respectively corresponding to An and Dn-k had erroneous transmission. However, the controller only needs to issue an error indication for the write operation corresponding to An, so that the DRAM can purge the stored erroneous address An′. The controller does not have to issue an error indication for the write operation corresponding to Dn-k, because even if the erroneous data is written to the DRAM core, it can be corrected by a re-issued write command, since there is no address error associated with Dn-k. This is based on the assumption that the controller does not issue a read command immediately following the erroneous write command to the same address. The controller can ensure this does not happen by comparing the address of pending read operations with the erroneous write operation and holding the read operations until the correct write data is committed. Alternatively, the controller can return the correct read data based on the data buffered within the controller, without addressing the DRAM at all.


Now, with reference back to operation 412 in FIG. 4, if the controller determines that the address associated with the error corresponds to a read operation, the controller re-issues the read command corresponding to the erroneous address (operation 418). The controller can optionally re-issue the write or read command corresponding to the data portion associated with the error (operation 420). Subsequently, the controller proceeds to receive the next CRC code from the DRAM (operation 404).


Note that although the above example is directed to erroneous read and write operations, the same method can be applied to other types of operations, such as activate, refresh, and other register-related operations. In general, this method can be applied to any memory operation that involves transmission of an address.



FIG. 5 presents an exemplary timing diagram illustrating the operation between a memory controller and a DRAM to facilitate address and data EDC, in accordance with one embodiment of the present invention. In this example, the clock and inverted clock signals are denoted as “clk” and “clk” respectively. Each clock cycle is denoted as Ti. The command can be transmitted at every rising edge of clk, and address can be transmitted at every rising edge of clk and clk. In other words, the data rate of address transmission is twice that of command transmission. The data clock used in this example is twice as fast as the clock for command and address transmission. Here, the data clock and inverted data clock signals are denoted as “wclk” and “wclk” respectively. The data bits on the DQ bus and the CRC bits on the EDC bus are transmitted at every rising edge of wclk and wclk. Hence, the data rate of DQ and EDC is four times the data rate of the command bus and twice the data rate of the address bus.


In this example, at clock cycle T0, the controller issues a write command WR0, and places the corresponding addresses, AD01 and AD02, on the address bus. Assume that the DRAM has a write latency of two clock cycles (i.e., WL=2). The corresponding write data burst 512 is placed on the DQ bus at clock cycle T2. Assume that each write data burst is eight bits long. Hence, data burst 512 occupies clock cycles T2 and T3. Meanwhile, at T3, the controller issues the next write command WR1 and the corresponding addresses AD11 and AD12. AD11 and AD12 are received by the DRAM at approximately the same time as data burst 512 (which is the write data for WR0). The DRAM concatenates AD11 and AD12 with data burst 512, generates a CRC code 522, and places CRC code 522 on the EDC bus at T4.


While the controller receives CRC code 522, the controller continues to issue the next write command, WR2, and its corresponding address AD21 and AD22 at T5. In addition, at T5, the controller places data burst 514, which is the write data for WR1, on the DQ bus.


After receiving CRC code 522, the controller detects an error (indicated by a star), and issues an error indication 515 to the DRAM at T7. Error indication 515 corresponds to the write command WR1 and allows the DRAM to discard the erroneously received addresses AD11 and AD12. Subsequent to error indication 515, the controller re-issues WR0 and WR1 at T8 and T10, respectively. The controller does not need to issue an error indication for WR0, if the address of WR0 has been received correctly by the DRAM and there is no potential data hazard between WR0 and WR1. In response to error indication 515, the DRAM can discard the buffered address AD01, and AD02, and optionally buffered data burst 512.


Meanwhile, at T7, the controller places data burst 516, which is the write data corresponding to WR2, on the DQ bus. On the DRAM side, after receiving data burst 514, the DRAM concatenates AD21 and AD22 with data burst 514 and generates CRC code 524. Subsequent to data burst 514, the controller transmits data burst 516, which is the write data for WR2. At T10, the controller transmits data burst 518, which is the write data for the re-issued WR0.


Because data bursts 514 and 516 are received at the DRAM back-to-back, the DRAM also generates CRC code 526 based on AD01, AD02, and data burst 516. AD01 and AD02 at T8 correspond to the re-issued WR0. Since the controller does not find any error in CRC codes 524 and 526 (indicated by a check mark), the controller does not perform any additional operations with respect to these CRC codes.


In this example, there are seven clock cycles between the time when the address for WR0 (AD01 and AD 02) is placed on the address bus and the time when error indication 515 is issued. Correspondingly, these addresses are buffered in the DRAM for seven clock cycles in order for the DRAM to abort them in response to an error indication. This buffer delay is longer than the DRAM's normal write latency, which is two clock cycles. This buffering can be provided by buffer 146 in the system illustrated in FIG. 1. If the time required for the DRAM to compute the CRC and for the controller to check the CRC and issue an error indication is deterministic, the amount of buffering for the received address can also be determined. Furthermore, the DRAM can include a mechanism that determines this total latency required for the CRC check and configure the buffer automatically during a start-up sequence, such that the received addresses are buffered sufficiently long to allow the CRC check and error notification.



FIG. 6 presents an exemplary timing diagram illustrating the operation between a memory controller and a DRAM to facilitate address and data EDC, in accordance with one embodiment of the present invention. In this example, the controller is configured to transmit the write data after the corresponding write address is confirmed to have been received successfully at the DRAM. The write latency in this example is assumed to be three clock cycles (i.e., WL=3).


As illustrated in FIG. 6, at T0, the controller issues the first write command WR0 and the corresponding addresses, AD01 and AD02. Based on AD01 and AD02, the DRAM generates CRC code 622. Subsequently, the controller determines that CRC code 622 is error-free, and transmits data burst 614, which is the write data for WR0. Similarly, the DRAM generates CRC code 624 based on AD11 and AD12. Upon receiving and confirming CRC code 624, the controller transmits data burst 616, which is the write data corresponding to WR1.


Data burst 614 is transmitted at T3. Since data burst 614 lasts two full clock cycles (i.e., from T3 to T5), which coincide with the transmission of WR2, AD21, and AD22, the DRAM can first buffer AD21 and AD22 and then concatenate them with data burst 614 to produce CRC code 626. Upon receiving and confirming CRC code 626, the controller proceeds to transmit data burst 618, which is the write data corresponding to WR2.


In the example above, the DRAM generates a CRC code based on the received address as soon as the address is received, provided there is no data being received on the DQ bus. The controller would withhold the write data transmission until the CRC code corresponding to the write address is received and confirmed. However, when the DRAM receives addresses and data at the same time (for example, data burst 614, AD21, and AD22), the DRAM can first buffer the address and then generate the CRC code based on both the address and received data.


Although the description uses read and write operations as examples, the methods disclosed herein are not limited to these operations. In general, the present address error-detection mechanisms are applicable with any memory operation that involves an address transmission.


The components of the address error detection mechanism described above can include any collection of computing components and devices operating together. These components can also be components or subsystems in a larger computer system or network. Components of an address error detection mechanism can also be coupled among any number of components (not shown), for example, buses, controllers, memory devices, and data input/output (I/O) devices, in any number of combinations. Many of these system components may be situated on a common printed circuit board or integrated circuit, or may be integrated in a system that includes several printed circuit boards or ICs that are coupled together in a system, for example, using connector and socket interfaces such as those employed by personal computer motherboards and dual inline memory modules (“DIMM”). In other examples, complete systems may be integrated in a single package housing using a system in package (“SIP”) type of approach. Integrated circuit devices may be stacked on top of one another and utilize wire bond connections to effectuate communication between devices or may be integrated on a single planar substrate in the package housing.


Further, functions of an address error detection mechanism can be distributed among any number/combination of other processor-based components. The address error detection mechanism described above can be applied in, for example, various DRAM systems. As examples, the DRAM memory systems can include DDR systems like DDR SDRAM, as well as DDR2 SDRAM, DDR3 SDRAM, and other DDR SDRAM variants, such as Graphics DDR (“GDDR”) and further generations of these memory technologies, including GDDR2, GDDR3, GDDR4, and GDDR5, but are not limited to these memory systems.


Aspects of the address error detection mechanisms described herein may be implemented as functionality programmed into any of a variety of circuitry, including programmable logic devices (PLDs), such as field programmable gate arrays (FPGAs), programmable array logic (PAL) devices, electrically programmable logic and memory devices, and standard cell-based devices, as well as application specific integrated circuits (ASICs). The underlying device technologies may be provided in a variety of component types, e.g., metal-oxide semiconductor field-effect transistor (MOSFET) technologies like complementary metal-oxide semiconductor (CMOS), bipolar technologies like emitter-coupled logic (ECL), polymer technologies (e.g., silicon-conjugated polymer and metal-conjugated polymer-metal structures), mixed analog and digital, etc.


The foregoing descriptions of embodiments described herein have been presented only for purposes of illustration and description. They are not intended to be exhaustive or to limit the embodiments to the forms disclosed. Accordingly, many modifications and variations will be apparent to practitioners skilled in the art.


Moreover, some components are shown directly connected to one another, while others are shown connected via intermediate components. In each instance the method of interconnection, or ‘coupling,’ establishes some desired electrical communication between two or more circuit nodes, or terminals. Such coupling may often be accomplished using a number of circuit configurations, as will be understood by those of skill in the art. Therefore, the spirit and scope of the appended claims should not be limited to the foregoing description. Only those claims specifically reciting “means for” or “step for” should be construed in the manner required under the sixth paragraph of 35 U.S.C. § 112.

Claims
  • 1. A memory controller to control memory, the memory controller comprising: circuitry to transmit, to the memory, a write command and an associated address;circuitry to generate error detection information, dependent on the associated address;circuitry to detect, using the error detection information and additional information received from the memory, whether the associated address as received by the memory contains an error; andcircuitry to transmit to the memory, after a delay time with respect to the write command, a second communication, wherein the second communication upon receipt by the memory will cause the memory to store write data associated with the write command into a storage array of the memory.
  • 2. The memory controller of claim 1, embodied as an integrated circuit, wherein the memory controller further comprises: at least one first pin to transmit the write commands and associated address to the memory; andat least one second pin to receive the additional information from the memory.
  • 3. The memory controller of claim 1 wherein the additional information comprises a first cyclic redundancy code (CRC), wherein the memory controller comprises circuitry to calculate the error detection information in the form of a second CRC, dependent on the associated address as transmitted to the memory, and wherein the circuitry is to detect existence of the error dependent on whether the first CRC matches the second CRC.
  • 4. The memory controller of claim 3 wherein the first CRC represents a concatenation of the associated address with at least one second item of information from the group of: an address associated with another command; oran item of data.
  • 5. The memory controller of claim 1 wherein the second communication comprises the write data associated with the write command.
  • 6. The memory controller of claim 1 wherein the second communication comprises a commit command, the commit command operative, when received by the memory, to trigger the memory to store the write data in the storage array.
  • 7. The memory controller of claim 1 wherein the additional information comprises an error detection code that is also based on the write data.
  • 8. The memory controller of claim 1 wherein: the memory controller further comprises a buffer to hold the write data for at least a predetermined period of time; andthe memory controller further comprises circuitry to calculate and programmatically establish the predetermined period of time based on a nominal amount of time required for the circuitry to detect to determine whether, based on the additional information, the associated address as received by the memory contains the error; andthe second communication comprises the write data from the buffer.
  • 9. The memory controller of claim 1 wherein the memory comprises at least one dynamic random access memory (“DRAM”) integrated circuit and wherein the memory controller is embodied as a DRAM memory controller integrated circuit.
  • 10. A memory controller to control dynamic random access memory (DRAM), the memory controller comprising: circuitry to transmit, to the memory, a clock signal, a write command and, for each of a rising edge of the clock signal and a falling edge of a clock signal, respective bits of an associated address;circuitry to generate error detection information, dependent on the associated address;circuitry to detect, using the error detection information and additional information received from the memory, whether the associated address as received by the memory contains an error; andcircuitry to transmit to the memory, after a delay time with respect to the write command, a second communication, wherein the second communication upon receipt by the memory will cause the memory to store write data associated with the write command into a storage array of the memory.
  • 11. The memory controller of claim 10 wherein the circuitry to transmit the second communication is to transmit first bits of the second communication in association with a rising each of the clock signal and second bits of the second communication in association with a falling edge of the clock signal.
  • 12. The memory controller of claim 11 wherein the clock signal is a first clock signal, wherein the memory controller further comprises circuitry to transmit a second clock signal to the memory, the second clock signal having a frequency that is twice a frequency of the first clock signal, and wherein circuitry of the memory controller is to transmit first bits of the write data to the memory device in association with respective rising edges of the second clock signal and second bits of the write data in association with respective falling edges of the second clock signal.
  • 13. A method of operating a memory controller, to control memory, the method comprising: with circuitry, transmitting, to the memory, a write command and an associated address;with circuitry, generating error detection information, dependent on the associated address;with circuitry, detecting, using the error detection information and additional information received from the memory, whether the associated address as received by the memory contains an error; andwith circuitry, transmitting to the memory, after a delay time with respect to the write command, a second communication, wherein the second communication upon receipt by the memory will cause the memory to store write data associated with the write command into a storage array of the memory.
  • 14. The method of claim 13 wherein the error detection information comprises a first cyclic redundancy code (CRC), and wherein the method further comprises, with circuitry, calculating the error detection information as a second CRC, the second CRC dependent on the associated address as transmitted to the memory, and detecting existence of the error dependent on whether the first CRC matches the second CRC.
  • 15. The method of claim 14 wherein the first CRC represents a concatenation of the associated address with at least one second item of information from the group of: an address associated with another command; oran item of data.
  • 16. The method of claim 13 wherein the second communication comprises the write data associated with the write command.
  • 17. The method of claim 13 wherein the second communication comprises a commit command, the commit command operative, when received by the memory, to trigger the memory to store the write data in the storage array.
  • 18. The method of claim 13 wherein the additional information comprises an error detection code that is also based on the write data.
  • 19. The method of claim 13 wherein: the method further comprises calculating a duration of a period of time, according to which a buffer is hold at least one of the write command or the write data; andconfiguring buffer to store the least one for at least the duration.
  • 20. The method of claim 13 wherein the memory is dynamic random access memory (“DRAM”) and wherein the method further comprises transmitting a clock signal to the DRAM and transmitting a first bit of the associated address to the DRAM in association with a rising edge of the clock signal and a second bit of the associated address in association with a falling edge of the clock signal.
RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No. 17/481,246, filed for “Memory Error Detection” on behalf of inventors Ian Shaeffer and Craig E. Hampel on Sep. 21, 2021, which in turn is a continuation of U.S. patent application Ser. No. 16/678,159, filed for “Memory Error Detection” on behalf of inventors Ian Shaeffer and Craig E. Hampel on Nov. 8, 2019 (now U.S. patent Ser. No. 11/150,982), which in turn is a continuation of U.S. patent application Ser. No. 15/838,161, filed for “Memory Error Detection” on behalf of inventors Ian Shaeffer and Craig E. Hampel on Dec. 11, 2017 (now U.S. patent Ser. No. 10/558,520), which in turn is a continuation of U.S. patent application Ser. No. 14/864,500, filed for “Memory Error Detection” on behalf of inventors Ian Shaeffer and Craig E. Hampel on Sep. 24, 2015 (now U.S. Pat. No. 9,870,283), which in turn is a continuation of U.S. patent application Ser. No. 14/200,665, filed for “Memory Error Detection” on behalf of inventors Ian Shaeffer and Craig E. Hampel on Mar. 7, 2014 (now U.S. Pat. No. 9,170,894), which in turn is a continuation of U.S. patent application Ser. No. 14/020,755, filed for “Memory Error Detection” on behalf of inventors Ian Shaeffer and Craig E. Hampel on Sep. 6, 2013 (now U.S. Pat. No. 8,707,110), which in turn is a continuation of U.S. patent application Ser. No. 13/666,918, filed for “Memory Error Detection” on behalf of inventors Ian Shaeffer and Craig E. Hampel on Nov. 1, 2012 (now U.S. Pat. No. 8,555,116), which in turn is a continuation of U.S. patent application Ser. No. 12/424,094 (now U.S. Pat. No. 8,352,805), filed for “Memory Error Detection” on behalf of inventors Ian Shaeffer and Craig E. Hampel on Apr. 15, 2009, which in turn is a continuation-in-part application of U.S. patent application Ser. No. 12/035,022 (now U.S. Pat. No. 7,836,378), filed for “System to Detect and Identify Errors in Control Information, Read Data and/or Write Data” on behalf of inventors Ian Shaeffer, Craig Hampel, Yuanlong Wang, and Fred Ware on Feb. 21, 2008, which in turn is a continuation of U.S. patent application Ser. No. 11/436,284, filed for “System to Detect and Identify Errors in Control Information, Read Data and/or Write Data” on behalf of inventors Ian Shaeffer and Craig Hampel. U.S. patent application Ser. Nos. 16/678,159, 15/838,161, 14/200,665, 14/020,755, 13/666,918 and 12/424,094 are hereby incorporated by reference as though set forth herein.

US Referenced Citations (164)
Number Name Date Kind
3585378 Bouricius et al. May 1971 A
3893072 D'Antonio et al. Jul 1975 A
4054911 Fletcher et al. Oct 1977 A
4162536 Morley Jul 1979 A
4354225 Frieder et al. Oct 1982 A
4359771 Johnson et al. Nov 1982 A
4363125 Brewer et al. Dec 1982 A
4369510 Johnson et al. Jan 1983 A
4394763 Nagano et al. Jul 1983 A
4456993 Taniguchi et al. Jun 1984 A
4468731 Johnson et al. Aug 1984 A
4527237 Frieder et al. Jul 1985 A
4535455 Peterson Aug 1985 A
4543628 Pomfret Sep 1985 A
4584685 Gajar Apr 1986 A
4596014 Holeman Jun 1986 A
4597084 Dynneson et al. Jun 1986 A
4604750 Manton et al. Aug 1986 A
4648064 Morley Mar 1987 A
4792923 Nakase et al. Dec 1988 A
4888773 Arlington Dec 1989 A
4914657 Walter et al. Apr 1990 A
4920539 Albonesi Apr 1990 A
4924456 Maxwell et al. May 1990 A
4970714 Chen et al. Nov 1990 A
5065312 Bruckert et al. Nov 1991 A
5070474 Tuma et al. Dec 1991 A
5173905 Parkinson et al. Dec 1992 A
5218691 Tuma et al. Jun 1993 A
5313627 Amini et al. May 1994 A
5341251 Fincher et al. Aug 1994 A
5347643 Kondo et al. Sep 1994 A
5392302 Kemp et al. Feb 1995 A
5404361 Casorso et al. Apr 1995 A
5450609 Schultz et al. Sep 1995 A
5488691 Fuoco et al. Jan 1996 A
5490153 Gregg et al. Feb 1996 A
5502733 Kishi et al. Mar 1996 A
5550988 Sarangdhar et al. Aug 1996 A
5553231 Papenberg et al. Sep 1996 A
5559956 Sukegawa Sep 1996 A
5588112 Dearth et al. Dec 1996 A
5687183 Chesley Nov 1997 A
5729550 Nakajima et al. Mar 1998 A
5751932 Horst et al. May 1998 A
5751955 Sonnier et al. May 1998 A
5778419 Hansen et al. Jul 1998 A
5828677 Sayeed et al. Oct 1998 A
5841795 Olarig et al. Nov 1998 A
5987628 Von Bokern et al. Nov 1999 A
6003151 Chuang Dec 1999 A
6009542 Koller et al. Dec 1999 A
6012136 Brown Jan 2000 A
6038679 Hanson Mar 2000 A
6065146 Bosshart May 2000 A
6081145 Bandai et al. Jun 2000 A
6094732 Takano Jul 2000 A
6115763 Douskey Sep 2000 A
6125470 Hee et al. Sep 2000 A
6151689 Garcia et al. Nov 2000 A
6169710 Arai Jan 2001 B1
6189123 Anders Nystrom et al. Feb 2001 B1
6208663 Schramm et al. Mar 2001 B1
6212660 Joeressen et al. Apr 2001 B1
6249894 Lin et al. Jun 2001 B1
6308294 Ghosh et al. Oct 2001 B1
6314541 Seytter et al. Nov 2001 B1
6367048 McAuliffe et al. Apr 2002 B1
6373842 Coverdale et al. Apr 2002 B1
6393504 Leung et al. May 2002 B1
6397365 Brewer et al. May 2002 B1
6438723 Kalliojarvi Aug 2002 B1
6507928 Richardson Jan 2003 B1
6529561 Sipola Mar 2003 B2
6545994 Nelson et al. Apr 2003 B2
6553003 Chang Apr 2003 B1
6560725 Longwell et al. May 2003 B1
6606589 Tuma et al. Aug 2003 B1
6625749 Quach Sep 2003 B1
6646911 Hidaka Nov 2003 B2
6697986 Kim et al. Feb 2004 B2
6700867 Classon et al. Mar 2004 B2
6704898 Furuskar et al. Mar 2004 B1
6715116 Lester et al. Mar 2004 B2
6725414 Seyyedy Apr 2004 B2
6735726 Muranaka et al. May 2004 B2
6742159 Sakurai May 2004 B2
6745364 Bhatt et al. Jun 2004 B2
6754856 Cofler et al. Jun 2004 B2
6765740 Tsuchiya Jul 2004 B2
6779148 Tanaka Aug 2004 B2
6792501 Chen et al. Sep 2004 B2
6823424 Larson et al. Nov 2004 B2
6832340 Larson et al. Dec 2004 B2
6845472 Walker et al. Jan 2005 B2
6851081 Yamamoto Feb 2005 B2
6880103 Kim et al. Apr 2005 B2
6883130 Wilhelmsson et al. Apr 2005 B2
6909758 Ramesh et al. Jun 2005 B2
6912682 Aoki Jun 2005 B1
6941493 Phelps Sep 2005 B2
6977888 Frenger et al. Dec 2005 B1
6990604 Binger Jan 2006 B2
7027539 Yang et al. Apr 2006 B2
7047473 Hwang et al. May 2006 B2
7072307 Tong et al. Jul 2006 B2
7168023 Morgan et al. Jan 2007 B2
7200770 Hartwell et al. Apr 2007 B2
7231580 Shiota et al. Jun 2007 B2
7249289 Muranaka et al. Jul 2007 B2
7339759 Hashimoto Mar 2008 B2
7340641 Binger Mar 2008 B1
7355803 Yang Apr 2008 B2
7418436 Maeda et al. Aug 2008 B2
7421547 Matsui et al. Sep 2008 B2
7519894 WeiB et al. Apr 2009 B2
7523381 Eggleston et al. Apr 2009 B2
7529965 Ikeuchi et al. May 2009 B2
7548495 Kobayashi Jun 2009 B2
7562285 Wang et al. Jul 2009 B2
7624298 Kasahara et al. Nov 2009 B2
7657783 Levitan Feb 2010 B2
7836386 Schumacher et al. Oct 2010 B2
7831882 Tsern et al. Nov 2010 B2
7831888 Wang et al. Nov 2010 B2
7882423 Wang et al. Feb 2011 B2
7949931 Lastras-Montano May 2011 B2
8042023 Balb Oct 2011 B2
8132077 Wang et al. Mar 2012 B2
9471418 Ahn et al. Oct 2016 B1
11106799 Keromytis et al. Aug 2021 B2
20010056567 Sakurai Dec 2001 A1
20020053042 Brown May 2002 A1
20020144210 Borkenhagen et al. Oct 2002 A1
20030115417 Corrigan Jun 2003 A1
20040073649 Inami et al. Apr 2004 A1
20040088497 Deans et al. May 2004 A1
20040139310 Maeda et al. Jul 2004 A1
20040205433 Gower et al. Oct 2004 A1
20040209420 Ljungcrantz et al. Oct 2004 A1
20040237001 Schulz et al. Nov 2004 A1
20040264030 Yang Dec 2004 A1
20050055522 Yagi Mar 2005 A1
20050073899 Gallivan et al. Apr 2005 A1
20060075291 Takahashi Apr 2006 A1
20060077750 Pescatore Apr 2006 A1
20060098320 Koga et al. May 2006 A1
20060123483 Cohen Jun 2006 A1
20060126460 Kobayashi Jun 2006 A1
20060277434 Tsern et al. Dec 2006 A1
20070002482 Daikokuya et al. Jan 2007 A1
20070043917 Matsui et al. Feb 2007 A1
20070150872 Vohra Jun 2007 A1
20070162825 Wang et al. Jul 2007 A1
20080046802 Honda Feb 2008 A1
20080163007 Shaeffer et al. Jul 2008 A1
20080168325 Yoon Jul 2008 A1
20080195922 Lee Aug 2008 A1
20090006863 Mizuno Jan 2009 A1
20090204871 Eggleston et al. Aug 2009 A1
20100217915 O'Connor et al. Aug 2010 A1
20110246857 Bae et al. Oct 2011 A1
20120179866 Davis et al. Jul 2012 A1
20160179634 Alexander et al. Jun 2016 A1
Non-Patent Literature Citations (28)
Entry
Answers.com, “ECC Memory,” http://www.answers.com/topic/eec-memory, obtained Jun. 20, 2006. 2 pages.
Cardarilli et al., “Design of a Fault Tolerant Solid State Mass Memory,” IEEE Transactions of Reliability, vol. 52, No. 4, Dec. 2003, pp. 476-491. 16 pages.
Dell Computer Corporation et al., “Serial ATA II: Electrical Specification,” Rev. 1, May 26, 2004. 187 pages.
Digital Equipment Corporation, “ESE50 SDI Solid State Disk Service Guide,” Jun. 1993. 132 pages.
Digital Equipment Corporation, “Hardware Documentation—Machines DEC—VAX Hardware Reference,” printed from http://www.netbsd.org/documentation/hardware/Machines/DEC/vax on May 13, 2005. 91 pages.
Digital Equipment Corporation, “Software Product Description, Product Name: HSC High Performance Software, Ver. 8.6,” Jun. 1996. 6 pages.
Digital Equipment Corporation, “Software Product Description, Product Name: HSC Software, Ver. 6.5,” Jun. 1992. 4 pages.
EP Communication dated Jan. 2, 2013 in EP Application No. 07794938.6-2224. 3 pages.
Geisler, J., “Error Detection & Correction,” Taylor University, Computer and System Sciences Department, Sep. 16, 2005. 21 pages.
Grosspietsch et al., “A Memory Interface Chip Designed for Fault Tolerance,” VLSI System Design, Jun. 1987, pp. 112-118. 5 pages.
Grosspietsch et al., “The VLSI Implementation of a Fault-Tolerant Memory Interface—a Status Report,” VLSI '85, 1986, pp. 155-164. 10 pages.
Haas et al., “Advances in Server Memory Technology,” presented at Spring 2005 Intel Developers Forum, San Francisco, CA, Mar. 1, 2005. 31 pages.
Hanna, P., “Error Detection and Correction,” Lecture 9, Queen's University-Belfast, Jan. 3, 2005. 12 pages.
Hodgart et al., “A (16,8) Error Correcting Code (T=2) for Critical Memory Applications,” DASIA2000, Montreal, Canada, May 22-26, 2000. 4 pages.
JP Argument Response dated Sep. 12, 2012 to the Office Action dated Jun. 13, 2012 re JP Application No. 2009-511054. 6 pages.
JP Office Action dated Jun. 13, 2012 re JP Application No. 2009-511054. 5 pages.
JP Voluntary Amendment and Petition filed Jul. 16, 2010 in JP Patent Application No. 2009-511054. 8 pages.
Kilbuck et al., “Fully Buffered DIMM—Unleashing Server Capacity,” Micron Technology, Inc., Apr. 8, 2005. 199 pages.
May et al., “HiPER: A Compact Narrow Channel Router with Hop-by-Hop Error Correction,” IEEE Transactions on Parallel and Distributed Systems, vol. 13, No. 5, May 2002, pp. 485-498. 14 pages.
Notice of Allowance and Issue Fee(s) due for U.S. Appl. No. 11/145,429 dated Jul. 14, 2010. 4 Pages.
PCT International Preliminary Report on Patentability dated Aug. 12, 2008 in International Application No. PCT/US2007/011733. 22 pages.
PCT International Search Report and Written Opinion dated Jan. 11, 2008 in International Application No. PCT/US2007/011733. 13 pages.
PCT International Search Report and Written Opinion dated Mar. 6, 2007 in International Application No. PCT/US2006/020698. 16 pages.
Pearson Education, “Error Detection and Correction,” Logic and Computer Design Fundamentals, 3rd Ed., 2004, pp. 1-5. 5 pages.
Sidiropoulos et al., “A 700 Mb/s/pin CMOS Signaling Interface Using Current Integrating Receivers,” IEEE Journal of Solid-State Circuits, vol. 32, No. 5, pp. 681-690, May 1997. 10 pages.
Siewiorek, Dan, “20 Fault Tolerance & Memory Hierarchy,” Lecture Handout, Carnegie Mellon University, Nov. 23, 1998. 18 pages.
Wicker, Stephen B., “Error Control Systems for Digital Communications and Storage” Prentice-Hall, 1995, pp. 392-423. 17 pages.
Wikipedia, “Forward Error Correction,” http://en.wikipedia.org/wiki/forward_error_correction, Printed Jun. 20, 2006. 3 pages.
Related Publications (1)
Number Date Country
20230333927 A1 Oct 2023 US
Continuations (9)
Number Date Country
Parent 17481246 Sep 2021 US
Child 18095341 US
Parent 16678159 Nov 2019 US
Child 17481246 US
Parent 15838161 Dec 2017 US
Child 16678159 US
Parent 14864500 Sep 2015 US
Child 15838161 US
Parent 14200665 Mar 2014 US
Child 14864500 US
Parent 14020755 Sep 2013 US
Child 14200665 US
Parent 13666918 Nov 2012 US
Child 14020755 US
Parent 12424094 Apr 2009 US
Child 13666918 US
Parent 11436284 May 2006 US
Child 12035022 US
Continuation in Parts (1)
Number Date Country
Parent 12035022 Feb 2008 US
Child 12424094 US