Embodiments of the invention relate to memory devices, and, more particularly, in one or more embodiments to a memory device that can be operated in either a direct mode, in which conventional memory control signals are coupled to the memory devices, or an indirect mode, in which command packets are coupled to the memory devices.
As memory devices of all types have evolved, continuous strides have been made in improving their performance in a variety of respects. For example, the storage capacity of memory devices has continued to increase at geometric proportions. This increased capacity, coupled with the geometrically higher operating speeds of electronic systems containing memory devices, has made high memory device bandwidth ever more critical. One application in which memory devices, such as dynamic random access memory (“DRAM”) devices, require a higher bandwidth is their use as system memory in computer systems. As the operating speed of processors has increased, processors are able to read and write data at correspondingly higher speeds. Yet conventional DRAM devices often do not have the bandwidth to read and write data at these higher speeds, thereby slowing the performance of conventional computer systems. This problem is exacerbated by the trend toward multi-core processors and multiple processor computer systems. It is currently estimated that computer systems operating as high-end servers are idle as many as 3 out of every 4 clock cycles because of the limited data bandwidth of system memory devices. In fact, the limited bandwidth of DRAM devices operating as system memory can reduce the performance of computer systems to as low as 10% of the performance of which they would otherwise be capable.
Various attempts have been made to increase the data bandwidth of memory devices. For example, wider internal data buses have been used to transfer data to and from arrays with a higher bandwidth. However, doing so usually requires that write data be serialized and read data deserialized at the memory device interface. Another approach has been to simply scale up the size of memory devices or conversely shrink their feature sizes, but, for a variety of reasons, scaling has been incapable of keeping up with the geometric increase in the demand for higher data bandwidths.
More recently, proposals have also been made to stack several integrated circuit memory devices in the same package, but doing so threatens to create a large number of other problems to be overcome. These problems can be solved to a large extent by connecting the stack of interconnected memory devices to a logic die on which the memory devices are stacked. The logic die can then serve as a high-speed interface to the memory devices. However, taking advantage of the increased capabilities of this arrangement is more easily achieved if memory command and address signals are placed in a packet and coupled to the logic die through a high-speed bus. Yet many computer and other systems are designed to interface with memory devices using conventional memory command signals and conventional row and column address signals. Advanced memory systems formed by stacking memory devices on a logic die would therefore be unusable with such systems. However, memory device manufacturers generally desire to standardize their product offerings to the greatest extent possible to lessen the number of different memory devices that are manufactured, marketed, etc.
Therefore, a need exists for a method and system to allow advanced memory system formed by stacking interconnected memory device dice to be interfaced with systems by either using conventional memory commands and addresses or by using packets containing commands and addresses.
A computer system including a high-capacity, high bandwidth memory device 10 according to an embodiment of the invention is shown in
The DRAM die 20, 22, 24, 26 are connected to each other and to the logic die 30 by a relatively wide bus 34. The bus 34 may be implemented with through silicon vias (“TSVs”), which comprise a large number of conductors extending through the DRAM die 20, 22, 24, 26 at the same locations on the DRAM die and connect to respective conductors formed on the die 20, 22, 24, 26. In one embodiment, each of the DRAM die 20, 22, 24, 26 are divided into 16 autonomous partitions, each of which may contain 2 or 4 independent memory banks. In such case, the partitions of each die 20, 22, 24, 26 that are stacked on top of each other may be independently accessed for read and write operations. Each set of 16 stacked partitions may be referred to as a “vault.” Thus, the memory device 10 may contain 16 vaults.
As shown in
As explained in greater detail below, one of the functions performed by the logic die 30 can be to serialize the read data bits coupled from the DRAM die 20, 22, 24, 26 into a serial stream of 16 serial data bits coupled through 16 parallel bits of each upstream lane 42a-d of the bus 14. Similarly, the logic die 30 may perform the functions of deserializing 16 serial data bits coupled through one of the 16-bit downstream lanes 40a-d of the bus 14 to obtain 256 parallel data bits. The logic die 30 then couples these 256 bits through one of the 32-bit sub-buses 38a-p in a serial stream of 8 bits. However, other embodiments may use different numbers of lanes 40, 42 having different widths or different numbers of sub-buses 38a-p having different widths, and they may couple data bits having different structures. As will be appreciated by one skilled in the art, the stacking of multiple DRAM die results in a memory device having a very large capacity. Further, the use of a very wide bus connecting the DRAM die allows data to be coupled to and from the DRAM die with a very high bandwidth.
A logic die 30 according to an embodiment of the invention is shown in
Each of the link interfaces 50a-d applies its 256 parallel bits to a respective downstream target 60a-d, which decodes the command and address portions of the received packet (in the indirect mode) or the commands and addresses (in the direct mode) and buffers write data in the event a memory request is for a write operation. The downstream targets 60a-d output their respective commands, addresses and possibly write data to a switch 62. The switch 62 contains 16 multiplexers 64 each of which direct the command, addresses and any write data from any of the downstream targets 60a-d to its respective vault of the DRAM die 20, 22, 24, 26. Thus, each of the downstream targets 60a-d can access any of the 16 vaults in the DRAM die 20, 22, 24, 26. The multiplexers 64 use the address in the received memory requests to determine if its respective vault is the target of a memory request. Each of the multiplexers 64 apply the memory request to a respective one of 16 vault controllers 70a-p.
Each vault controller 70a-p includes a respective memory controller 80, each of which includes a write buffer 82, a read buffer 84 and a command pipeline 86. The commands and addresses in memory requests received from the switch 62 are loaded into the command pipeline 86, which subsequently outputs the received commands and corresponding addresses. Any write data in the memory requests are stored in the write buffer 82. The read buffer 84 may be used to store read data from the respective vault, as will be explained in greater detail below. The write data from the write buffer 82 are applied to a memory interface 88.
According to an embodiment of the invention, the commands and addresses from the command pipeline 86 are applied to a memory interface 88 through a command processing circuit, such as a command register 90. The command register 90 can be a free running interface register. In the direct mode, the commands and addresses from the command pipeline are applied to the memory interface 88. These commands and addressed may be applied to the memory interface 88 as they are received by the memory device 10. In the indirect mode, the command register 90 creates the commands and addresses and sends it to the memory interface 88. The command register 90 includes a sequencer (not shown) that transmits the commands and addresses to the memory interface in the proper order and at the proper times.
The memory interface 88 couples the received command and address signals from the command register 90 to the DRAM die 20, 22, 24, 26 through a command/address bus 92. The memory interface 88 also couples 32-bits of write data from the write buffer 82. In some embodiments, the memory interface 88 may include an ECC system (not shown), which uses ECC techniques to check and correct the data read from the DRAM die 20, 22, 24, 26. In such case, in addition to coupling write data to the DRAM die 20, 22, 24, 26, the memory interface 88 couples 4 bits of FCC from the ECC system to the DRAM die 20, 22, 24, 26 through a 36-bit data bus 94.
Although write data are loaded into the write buffer 82 as 256 parallel bits, they are output from the buffer 82 in two sets, each set being 128 parallel bits. These 128 bits may then be further serialized by the ECC system (not shown) to 4 sets of 32-bit data, which are coupled through the data bus 94. In the embodiment shown in
In the event a memory request is for a read operation, the command and address for the request are coupled to the DRAM die 20, 22, 24, 26 in the same manner as a write request, as explained above. In response to a read request, 32 bits of read data and 4 ECC bits are output from the DRAM die 20, 22, 24, 26 through the 36-bit data bus 94. The ECC bits are passed to the ECC system (not shown), which uses the ECC bits to check and correct the read data before passing the read data on to the read buffer 84. The ECC system also deserializes the 32 bits of read data into two sets of 128-bit read data. However, in some embodiments, the memory system does not include the ECC system.
After 2 sets of 128-bit read data have been stored in the read buffer 84, the read buffer transmits 256 bits to the switch 62. The switch includes 4 output multiplexers 104 coupled to respective upstream masters 110a-d. Each multiplexer 104 can couple 256 bits of parallel data from any one of the vault controllers 70a-p to its respective upstream master 110a-d. The upstream masters 110a-d format the 256 bits of read data into packet data (in the indirect mode) and couple the packet to respective upstream link interfaces 114a-d. In the direct mode, the read data are simply coupled to respective upstream link interfaces 114a-d. Each of the link interfaces 114a-d include a respective serializer 120 that converts the incoming 256 bits to a serial stream of 16 bits on each bit of a respective one of the 16-bit upstream links 42a-d.
The format of a downstream packet 150 that can be coupled to the memory system of
As shown in
Returning to
The downstream packet 150 also contains a third field 190, which includes a mask bit 192 that specifies whether a write will be masked, and 31 bits of write data 196. Following the third field 190 are one or more fields of write data 200. A final field contains a set of error checking bits 210, which may be cyclic redundancy check (“CRC”) bits, ECC bits or some other type of error checking bits. The error checking bits 210 correspond to the write data to allow the memory system to determine if there were any errors in the transmission of the write data. In the case where the error checking bits are ECC bits and the number of errors is not too great, the bits 210 may allow errors in the write data to be corrected.
Potential commands corresponding to the 4 command bits 156 in the first field 152 are shown in
With further reference to
As also shown in
The format of an upstream packet 250 is shown in
The commands corresponding to the Cmd bits 262, 272 in the upstream packet 250 are shown in
The commands corresponding to the Cmd1 bits 272 are also shown in
From the foregoing it will be appreciated that, although specific embodiments of the invention have been described herein for purposes of illustration, various modifications may be made without deviating from the spirit and scope of the invention. For example, although the embodiments of the invention are explained in the context of stacked DRAM die, it will be understood that the stacked die may be other types of memory device dice, such as flash memory device dice. Accordingly, the invention is not limited except as by the appended claims.
This application is a continuation of U.S. patent application Ser. No. 15/597,033, filed May 16, 2017, and issued as U.S. Pat. No. 10,109,343 on Oct. 23, 2018, which is a continuation of U.S. patent application Ser. No. 13/619,682, filed Sep. 14, 2012, and issued as U.S. Pat. No. 9,659,630 on May 23, 2017, which is a continuation of U.S. patent application Ser. No. 12/166,871, filed Jul. 2, 2008, issued as U.S. Pat. No. 8,289,760 on Oct. 16, 2012. The aforementioned applications and patents are incorporated by reference herein, in their entirety, and for all purposes.
Number | Name | Date | Kind |
---|---|---|---|
3755689 | Elmer et al. | Aug 1973 | A |
5179303 | Searles et al. | Jan 1993 | A |
5263032 | Porter et al. | Nov 1993 | A |
5726596 | Perez | Mar 1998 | A |
5748914 | Barth et al. | May 1998 | A |
5774475 | Qureshi | Jun 1998 | A |
5960008 | Osawa et al. | Sep 1999 | A |
5982684 | Schwartzlow et al. | Nov 1999 | A |
6052329 | Nishino et al. | Apr 2000 | A |
6078536 | Moon et al. | Jun 2000 | A |
6122688 | Barth et al. | Sep 2000 | A |
6177807 | Bertin et al. | Jan 2001 | B1 |
6181616 | Byrd | Jan 2001 | B1 |
6247138 | Tamura et al. | Jun 2001 | B1 |
6285211 | Sample | Sep 2001 | B1 |
6329859 | Wu | Dec 2001 | B1 |
6363017 | Polney | Mar 2002 | B2 |
6369627 | Tomita | Apr 2002 | B1 |
6380783 | Chao et al. | Apr 2002 | B1 |
6401213 | Jeddeloh | Jun 2002 | B1 |
6418068 | Raynham | Jul 2002 | B1 |
6519194 | Tsujino et al. | Feb 2003 | B2 |
6570425 | Yamaguchi | May 2003 | B2 |
6574626 | Regelman et al. | Jun 2003 | B1 |
6650157 | Amick et al. | Nov 2003 | B2 |
6658523 | Janzen et al. | Dec 2003 | B2 |
6727831 | Iwata | Apr 2004 | B2 |
6795369 | Choi et al. | Sep 2004 | B2 |
6839260 | Ishii | Jan 2005 | B2 |
6882304 | Winter et al. | Apr 2005 | B2 |
6889334 | Magro et al. | May 2005 | B1 |
6907555 | Nomura et al. | Jun 2005 | B1 |
7058865 | Mori et al. | Jun 2006 | B2 |
7107424 | Avakian et al. | Sep 2006 | B1 |
7135905 | Teo et al. | Nov 2006 | B2 |
7149134 | Streif et al. | Dec 2006 | B2 |
7168005 | Adams et al. | Jan 2007 | B2 |
7171596 | Boehler | Jan 2007 | B2 |
7184916 | Resnick et al. | Feb 2007 | B2 |
7197101 | Glenn et al. | Mar 2007 | B2 |
7203259 | Glenn et al. | Apr 2007 | B2 |
7205811 | Freyman et al. | Apr 2007 | B2 |
7243469 | Miller et al. | Jul 2007 | B2 |
7269094 | Lin et al. | Sep 2007 | B2 |
7323917 | Cho et al. | Jan 2008 | B2 |
7389375 | Gower et al. | Jun 2008 | B2 |
7423469 | Pickering et al. | Sep 2008 | B2 |
7464241 | Pete | Dec 2008 | B2 |
7466179 | Huang et al. | Dec 2008 | B2 |
7489743 | Koh et al. | Feb 2009 | B2 |
7567476 | Ishikawa | Jul 2009 | B2 |
7697369 | Koshizuka | Apr 2010 | B2 |
7710144 | Dreps et al. | May 2010 | B2 |
7764564 | Saito et al. | Jul 2010 | B2 |
7772907 | Kim et al. | Aug 2010 | B2 |
7853836 | Takada | Dec 2010 | B2 |
7855931 | LaBerge et al. | Dec 2010 | B2 |
7978721 | Jeddeloh et al. | Jul 2011 | B2 |
7979757 | Jeddeloh | Jul 2011 | B2 |
8010866 | LaBerge | Aug 2011 | B2 |
8127204 | Hargan | Feb 2012 | B2 |
8134876 | Choi et al. | Mar 2012 | B2 |
8248138 | Liu | Aug 2012 | B2 |
8289760 | Jeddeloh | Oct 2012 | B2 |
8315347 | Canagasaby et al. | Nov 2012 | B2 |
8356138 | Kulkami et al. | Jan 2013 | B1 |
8400808 | King | Mar 2013 | B2 |
8533416 | LaBerge et al. | Sep 2013 | B2 |
8570881 | Talbot et al. | Oct 2013 | B2 |
8793460 | LaBerge et al. | Jul 2014 | B2 |
8806131 | Jeddeloh et al. | Aug 2014 | B2 |
9146811 | LaBerge et al. | Sep 2015 | B2 |
9171597 | Eckel | Oct 2015 | B2 |
9275698 | Laberge et al. | Mar 2016 | B2 |
9411538 | LaBerge et al. | Aug 2016 | B2 |
9437263 | Eckel | Sep 2016 | B2 |
9524254 | Jeddeloh et al. | Dec 2016 | B2 |
9602080 | King et al. | Mar 2017 | B2 |
9659630 | Jeddeloh | May 2017 | B2 |
9899994 | King | Feb 2018 | B2 |
10109343 | Jeddeloh | Oct 2018 | B2 |
20010033030 | Leedy | Oct 2001 | A1 |
20020004893 | Chang | Jan 2002 | A1 |
20020054516 | Taruishi et al. | May 2002 | A1 |
20020097613 | Raynham | Jul 2002 | A1 |
20020125933 | Tamura et al. | Sep 2002 | A1 |
20020130687 | Duesman | Sep 2002 | A1 |
20020133666 | Janzen et al. | Sep 2002 | A1 |
20020138688 | Hsu et al. | Sep 2002 | A1 |
20030041299 | Kanazawa et al. | Feb 2003 | A1 |
20030132790 | Amick et al. | Jul 2003 | A1 |
20040073767 | Johnson et al. | Apr 2004 | A1 |
20040098545 | Pline et al. | May 2004 | A1 |
20040160833 | Suzuki | Aug 2004 | A1 |
20040168101 | Kubo | Aug 2004 | A1 |
20040199840 | Takeoka et al. | Oct 2004 | A1 |
20040206982 | Lee et al. | Oct 2004 | A1 |
20040225856 | Braun et al. | Nov 2004 | A1 |
20040237023 | Takahashi et al. | Nov 2004 | A1 |
20040246026 | Wang et al. | Dec 2004 | A1 |
20040252689 | Park | Dec 2004 | A1 |
20050005230 | Koga et al. | Jan 2005 | A1 |
20050071707 | Hampel | Mar 2005 | A1 |
20050091471 | Conner et al. | Apr 2005 | A1 |
20050144546 | Igeta et al. | Jun 2005 | A1 |
20050157560 | Hosono et al. | Jul 2005 | A1 |
20050174877 | Cho et al. | Aug 2005 | A1 |
20050278490 | Murayama | Dec 2005 | A1 |
20050289435 | Mulla et al. | Dec 2005 | A1 |
20060028864 | Rinerson et al. | Feb 2006 | A1 |
20060036827 | Dell et al. | Feb 2006 | A1 |
20060041799 | Sato | Feb 2006 | A1 |
20060056247 | Satoh | Mar 2006 | A1 |
20060059406 | Micheloni et al. | Mar 2006 | A1 |
20060123320 | Vogt | Jun 2006 | A1 |
20060126369 | Raghuram | Jun 2006 | A1 |
20060168101 | Mikhailov et al. | Jul 2006 | A1 |
20060223012 | Sekiguchi et al. | Oct 2006 | A1 |
20060233012 | Sekiguchi et al. | Oct 2006 | A1 |
20060245291 | Sakaitani | Nov 2006 | A1 |
20060253723 | Wu et al. | Nov 2006 | A1 |
20060262587 | Matsui et al. | Nov 2006 | A1 |
20060273455 | Williams et al. | Dec 2006 | A1 |
20060282578 | Lee | Dec 2006 | A1 |
20070058410 | Rajan | Mar 2007 | A1 |
20070070669 | Tsern | Mar 2007 | A1 |
20070074093 | Lasser | Mar 2007 | A1 |
20070096875 | Waterhouse et al. | May 2007 | A1 |
20070136645 | Hsueh et al. | Jun 2007 | A1 |
20070153951 | Lim et al. | Jul 2007 | A1 |
20070182471 | Kim | Aug 2007 | A1 |
20070210841 | Kim | Sep 2007 | A1 |
20070271424 | Lee et al. | Nov 2007 | A1 |
20070288707 | Tremaine | Dec 2007 | A1 |
20080080261 | Shaeffer et al. | Apr 2008 | A1 |
20080147897 | Talbot | Jun 2008 | A1 |
20080150088 | Reed et al. | Jun 2008 | A1 |
20080201548 | Przybylski et al. | Aug 2008 | A1 |
20080250292 | Djordjevic | Oct 2008 | A1 |
20080270842 | Ho et al. | Oct 2008 | A1 |
20090006775 | Bartley et al. | Jan 2009 | A1 |
20090016130 | Menke et al. | Jan 2009 | A1 |
20090021992 | Oh | Jan 2009 | A1 |
20090091968 | Dietrich et al. | Apr 2009 | A1 |
20090196093 | Happ | Aug 2009 | A1 |
20090244997 | Searles et al. | Oct 2009 | A1 |
20090251189 | Hsieh | Oct 2009 | A1 |
20090296867 | Do et al. | Dec 2009 | A1 |
20090300314 | Laberge et al. | Dec 2009 | A1 |
20090300444 | Jeddeloh | Dec 2009 | A1 |
20100005217 | Jeddeloh | Jan 2010 | A1 |
20100005376 | Laberge et al. | Jan 2010 | A1 |
20100014364 | Laberge et al. | Jan 2010 | A1 |
20100031129 | Hargan | Feb 2010 | A1 |
20100042889 | Hargan | Feb 2010 | A1 |
20100070696 | Blankenship | Mar 2010 | A1 |
20100079180 | Kim et al. | Apr 2010 | A1 |
20100091537 | Best et al. | Apr 2010 | A1 |
20100110748 | Best | May 2010 | A1 |
20100156488 | Kim et al. | Jun 2010 | A1 |
20100176893 | Nose | Jul 2010 | A1 |
20100271092 | Zerbe et al. | Oct 2010 | A1 |
20100308880 | Nose et al. | Dec 2010 | A1 |
20110075497 | Laberge | Mar 2011 | A1 |
20110148486 | Mosalikanti et al. | Jun 2011 | A1 |
20110271158 | Jeddeloh | Nov 2011 | A1 |
20110296227 | LaBerge et al. | Dec 2011 | A1 |
20120144276 | Hargan | Jun 2012 | A1 |
20120155142 | King | Jun 2012 | A1 |
20130208549 | King | Aug 2013 | A1 |
20130318298 | LaBerge et al. | Nov 2013 | A1 |
20130346722 | LaBerge et al. | Dec 2013 | A1 |
20140050034 | Lee | Feb 2014 | A1 |
20140053040 | Hargan | Feb 2014 | A1 |
20140258666 | LaBerge | Sep 2014 | A1 |
20140298119 | LaBerge et al. | Oct 2014 | A1 |
20140337570 | LaBerge et al. | Nov 2014 | A1 |
20150028928 | King | Jan 2015 | A1 |
20150063043 | Eckel | Mar 2015 | A1 |
20150364172 | Shim et al. | Dec 2015 | A1 |
20160027486 | Eckel | Jan 2016 | A1 |
20160079971 | Singh et al. | Mar 2016 | A1 |
20160133336 | Lim | May 2016 | A1 |
20160182063 | Seo et al. | Jun 2016 | A1 |
20160211011 | Qian | Jul 2016 | A1 |
20170163251 | King | Jun 2017 | A1 |
20170249984 | Jeddeloh | Aug 2017 | A1 |
Number | Date | Country |
---|---|---|
101036131 | Sep 2007 | CN |
05-265872 | Oct 1993 | JP |
0774620 | Mar 1995 | JP |
11-102599 | Apr 1999 | JP |
11-513830 | Nov 1999 | JP |
2003-303139 | Oct 2003 | JP |
2004-327474 | Nov 2004 | JP |
2005-4947 | Jan 2005 | JP |
2007-507794 | Mar 2007 | JP |
2007-140948 | Jun 2007 | JP |
2007-226876 | Sep 2007 | JP |
2007-328636 | Dec 2007 | JP |
2008-112503 | May 2008 | JP |
2008-140220 | Jun 2008 | JP |
2010-514080 | Apr 2010 | JP |
9714289 | Apr 1997 | WO |
2005033958 | Apr 2005 | WO |
2007028109 | Mar 2007 | WO |
2007038225 | Apr 2007 | WO |
2007095080 | Aug 2007 | WO |
2008054696 | May 2008 | WO |
2008076790 | Jun 2008 | WO |
2009148863 | Dec 2009 | WO |
2010002561 | Jan 2010 | WO |
2010011503 | Jan 2010 | WO |
2012060097 | May 2012 | WO |
2012082338 | Jun 2012 | WO |
Entry |
---|
Notice of Preliminary Rejection dated Mar. 21, 2013 for KR Appln No. 10-2011-7002671. |
Office action recieved for Chinese application No. 200980125792.2 dated Nov. 2, 2012. |
Received Office Action dated Nov. 6, 2012 for JP Application No. 2011-516419. |
Related U.S. Appl. No. 12/970,086, filed Dec. 16, 2010, entitled Phase Interpolators and Push-Pull Buffers. |
2nd Office Action for CN Application 200980125792.2 dated Aug. 29, 2013. |
Decision of Rejection for Appl No. 2011-516419 dated Jul. 16, 2013, Jul. 16, 2013. |
Extended Search Report for Appl No. 13157772.8 dated Jun. 28, 2013. |
International Search Report and Written Opinion dated Feb. 18, 2010 for PCT/US2009/046898. |
Notice of Preliminary Rejection for KR Appl# 1020117002671, dated Sep. 24, 2012. |
Preliminary Rejection for KR Appl No. 10-2011-7002671 dated Nov. 28, 2013. |
TW 1st Office Action for Appl No. 096120886 dated Apr. 24, 2013. |
Extended European Search Report dated Aug. 29, 2011 for European Application No. 09774012.0. |
Number | Date | Country | |
---|---|---|---|
20180374530 A1 | Dec 2018 | US |
Number | Date | Country | |
---|---|---|---|
Parent | 15597033 | May 2017 | US |
Child | 16116751 | US | |
Parent | 13619682 | Sep 2012 | US |
Child | 15597033 | US | |
Parent | 12166871 | Jul 2008 | US |
Child | 13619682 | US |