Patent Application
20240274170
A disclosure is presented that relates to circuitry usable to perform in-memory computation, such as multiply-and-accumulate or other sum-of-products like operations.
In neuromorphic computing systems, machine learning systems and circuitry used for some types of computations based on linear algebra, the multiply-and-accumulate or sum-of-products functions can be important components. Such functions can be expressed as follows:
In this expression, each product term is a product of a variable input Xi and a weight Wi. The weight Wi can vary among the terms, corresponding for example to coefficients of the variable inputs Xi.
The sum-of-products function can be realized as a circuit operation using cross-point array architectures in which the electrical characteristics of cells of the array effectuate the function. One problem associated with large computations of this type arises because of the complexity of the data flow among memory locations used in the computations which can involve large tensors of input variables and large numbers of weights.
It is desirable to provide structures for sum-of-products operations suitable for implementation in-memory, to reduce the number of data movement operations required.
A circuit is described supporting compute-in-memory CIM operations using signed bits. The circuit comprises an array of memory cells arranged in columns and rows, memory cells in columns connected to corresponding bit lines, and memory cells in rows connected to corresponding word lines. Sensing circuits are configured to sense differences between first and second currents on respective bit lines in selected pairs of bit lines and to produce outputs for the selected pairs of bit line as a function of the difference. The array is programmable to store signed weights in sets of memory cells, the sets being operatively coupled with a corresponding pair of bit lines and a corresponding pair of word lines. Word line drivers can be configured to drive voltages representing signed inputs on respective word lines in selected pairs of word lines. The outputs produced by the sensing circuits can be signed outputs.
A configuration of memory cells is described for storing a signed bit, usable as a coefficient in a CIM operation. The configuration includes a set of memory cells, the set including first and second memory cells connected to a first word line in a corresponding pair of word lines, and third and fourth memory cells connected to a second word line in the corresponding pair of word lines. The first and third memory cells are on the first bit in a corresponding pair of bit lines and the second and fourth memory cells are on a second bit line in the corresponding pair of bit lines.
Sensing circuits are described which include a sensing module connectable to a pair of bit lines. The sensing module includes a current mirror circuit having a first leg and a second leg operatively connectable to first and second bit lines in the pair of bit lines, and an adjustable reference current source, and responsive to control signals to set a first configuration to adjust current on the first leg using the adjustable reference current source, and to set a second configuration to adjust current on the second leg using the adjustable reference current source. A comparator is included to compare a voltage on the first leg to a voltage on the second leg.
The array of memory cells can be a NOR or AND architecture flash memory array. Other embodiments can use an array in a NAND architecture flash memory. The memory cells in the array of memory cells can be charge trapping memory cells.
A method for storing a signed bit in a memory array including word lines and bit lines is described. The method includes writing respective threshold levels VT1, VT2, VT3 and VT4 in first, second, third and fourth memory cells, wherein the first memory cell is on a first bit line and a first word line, the second memory cell is on a second bit line and the first word line, the third memory cell is on the first bit line and a second word line, and the fourth memory cell is on the second bit line and the second word line; including for a signed bit of −1, VT1 is a high threshold, VT2 is a low threshold, VT3 is a low threshold and VT4 is a high threshold; for a signed bit of +1, VT1 is a low threshold, VT2 is a high threshold, VT3 is a high threshold and VT4 is a low threshold; for a signed bit of 0, VT1 is a high threshold, VT2 is a high threshold, VT3 is a high threshold and VT4 is a high threshold.
A method for multiplying a signed input bit by a signed coefficient bit in a memory array including word lines and bit lines is described. The method includes writing respective threshold levels VT1, VT2, VT3 and VT4 in first, second, third and fourth memory cells to represent the signed coefficient bit, as described above, and applying respective word line voltages VWL0, VWL1 to the first and second word lines to represent the signed input bit, including when the signed input bit is −1, VWL0 is low and VWL1 is high; when the signed input bit is +1, VWL0 is high and VWL1 is low; and when the signed input bit is 0, VWL0 is low and VWL1 is low. Also, the method includes sensing a difference in respective currents IBL0 and IBL1 on the first and second bit lines.
Other aspects and advantages of the technology can be seen on review of the drawings, the detailed description and the claims, which follow.
The device 100 includes input/output circuits 105 for communication of control signals, data, addresses and commands with other data processing resources, such as a CPU or memory controller.
Input/output data is applied on bus 191 to a controller 110, and to cache 190. Also, addresses are applied on bus 193 to a decoder 142, and to the controller 110. Also, the bus 191 and bus 193 can be operably connected to data sources internal to the integrated circuit device 100, such as a general purpose processor or special purpose application circuitry, or a combination of modules providing for example, system-on-a-chip functionality.
The memory array 160 can include an array of memory cells in a NOR architecture or in an AND architecture, such that memory cells are arranged in columns along bit lines and in rows along word lines, and the memory cells in a given column are connected in parallel between a bit line and a source reference. The source reference can comprise a ground terminal or a source line connected to source side biasing resources. The memory cells can comprise charge trapping transistors cells, arranged in a 3D structure.
The bit lines can be connected by block select circuits to global bit lines 165, configured for selectable connection to a page buffer 180, and to CIM sense circuits 170.
The page buffer 180 in the illustrated embodiment is connected by bus 185 to the cache 190. The page buffer 180 includes storage elements and sensing circuits for memory operations, including read and write operations. For flash memory including dielectric charge trapping memory and floating gate charge trapping memory, write operations include program and erase operations.
A driver circuit 140 is coupled to word lines 145 in the array 160, and applies word line voltages to selected word lines in response to a decoder 142 which decodes addresses on line 193, or in a computation operation, in response to input data stored in input buffer 141.
The controller 110 is coupled to the cache 190 and the memory array 160, and to other peripheral circuits used in memory access and in memory computation operations.
Controller 110, using a for example a state machine, controls the application of supply voltages and currents generated or provided through the voltage supply or current sources in block 120, for memory operations and for CIM operations.
The controller 110 includes control and status registers, and control logic which can be implemented using special-purpose logic circuitry including state machines and combinational logic as known in the art. In alternative embodiments, the control logic comprises a general-purpose processor, which can be implemented on the same integrated circuit, which executes a computer program to control the operations of the device. In yet other embodiments, a combination of special-purpose logic circuitry and a general-purpose processor can be utilized for implementation of the control logic.
The array 160 includes memory cells arranged in columns and rows, where memory cells in columns are connected to corresponding bit lines, and memory cells in rows are connected to corresponding word lines. The array 160 is programmable to store signed coefficients (weights Wi) in sets of memory cells. An example of a set of memory cells storing a signed weight is described with reference to
In a CIM mode, the word line driver circuit 140 includes drivers configured to drive signed inputs Xi by a select mode of voltages on selected word lines and unselected word lines from the input buffer 141. The CIM sense circuits 170 are configured to sense differences between first and second currents on respective bit lines in selected pairs of bit lines and to produce outputs for the selected pairs of bit lines as a function of the difference. The outputs can be applied to storage elements in the page buffer 180 and to the cache 190.
An implementation of a memory array can be based on charge trapping memory cells, such as floating gate memory cells which can include polysilicon charge trapping layers, or dielectric charge trapping memory cells which can include silicon nitride charge trapping layers. Other types of memory technology can be applied in various embodiments of the technology described herein.
An example of read performance for a memory transistor 200, like that of
Sets of memory cells with a behavior like that of
The set of memory cells 300 in
To store a signed bit in the set of memory cells 300, respective threshold levels VT1, VT2, VT3 and VT4 in are written in the first, second, third and fourth memory cells 300-1 to 300-4.
In this embodiment, when the signed coefficient bit is −1, VT1 is a high threshold, VT2 is a low threshold, VT3 is a low threshold and VT4 is a high threshold. When the signed coefficient bit is +1, VT1 is a low threshold, VT2 is a high threshold, VT3 is a high threshold and VT4 is a low threshold. When the signed coefficient bit is 0, VT1 is a high threshold, VT2 is a low threshold, VT3 is a low threshold and VT4 is a high threshold. The terms “low” and “high” are used in this context to refer to values below and above, respectively, a read voltage by amounts suitable for operations as described herein.
Table 1 illustrates the example threshold states for the set of memory cells 300 using the values described with reference to
To execute a multiply operation, respective word line voltages VWL0, VWL1 are applied to the first and second word lines to represent the signed input bit Xi.
In this embodiment, when the signed input bit is −1, VWL0 is low and VWL1 is high. When the signed input bit is +1, VWL0 is high and VWL1 is low. When the signed input bit is 0, VWL0 is low and VWL1 is low. The terms “low” and “high” are used in this context to refer to values below and above, respectively, a level to turn on an erase state transistor by amounts suitable for operations as described herein.
Table 2 illustrates example word line voltages applied for signed input values Xi.
When the word line voltages are applied, currents IBL0 and IBL1 are induced on the first and second bit lines. Sensing a difference in respective currents IBL0 and IBL1 on the first and second bit lines can yield a result representing a product P=(Xi)×(Wi), an example of which is shown in Table 3.
In order to multiply a signed input vector Xi[0:M] by a signed weight vector Wi[1:N], a plurality of sets of memory cells like that of
In
The pair of bit lines BL01 and BL11 is connected to sensing circuitry includes a circuit 404 generate a difference in the first and second currents IBL0 and IBL1, the output of which is applied to analog-to-digital converter 406. The output of the analog-to-digital converter 406 is a signed sum of products B1[P:0], where the bit B1[P] can be a sign bit.
Depending on the stored bits and the input bits, the current on the bit lines and the difference in currents on the two bit lines can be represented as shown in Table 4.
The difference in the currents IBL0 and IBL1 from each cell represents an inner product of the sum of products:
The combination of currents from all the cells 400-1 to 400-N represents the sum of the inner products.
Sensing circuitry and control logic for CIM operations is illustrated in
The sense circuitry can be implemented as illustrated in
The sensing circuits (e.g. 404, 406) can be implemented as shown in
The counter 502 receives an enable signal from the timing control logic 501, and a reset input from other control circuits on the device. In this example, the counter is a two bit counter generating output COUNT[1:0] on bus 512 which is applied to a reference current circuit 503, and each of the sense circuitry modules SA1 to SAQ. The reference current circuit 503 produces a Icell control signal on bus 513 to set the reference current Icell for each of the sense circuitry modules SA1 to SAQ. The signed outputs B1[2:0] to BQ[2:0] of the sense circuitry modules SA1 to SAQ are applied on lines 521-1 through 521-Q to storage elements in a page buffer (e.g. page buffer 180 of
Voltage V0 at node N0 and voltage V1 at node N1 are applied as negative and positive inputs, respectively, of comparator 630. Output of comparator 630 is applied to a pass-through terminal of pass gate 635, having a control terminal EN1 receiving a signal from the counter or control logic. For example, the output of comparator 630 is applied to a control logic 501. When the comparator flips in a given sense circuitry module, then the corresponding enable signals (EN11/EN21 to EN1Q to EN2Q) are asserted for the corresponding sense module. The pass-through terminal of the pass gate 636 is connected to the output COUNT[P−1:0] (e.g., P=2) of the counter as discussed above. Output of the pass gate 635 is the sign bit B1[P]. The combination of the outputs of pass gate 635 and pass gate 636 is the signed output B1[P:0] of the sense circuitry module. A two-bit counter is used in this example. Higher resolution counters (e.g. 3 bits or more) can be used with corresponding circuit changes, such as changes in Icell generator step sizes, to sense finer degrees of current differences.
In operation, the current mirror circuit and the control signals CK1 and CK2 are controlled to obtain voltages V1 and V0, in a first step used to determine whether the difference between the current on bit lines BL0 and BL1 is positive or negative to obtain bit B1[P]. In a second step, or a sequence of steps, the current mirror circuit, the control signals and the Icell generator are used to determine the magnitude of the difference in currents to obtain bits B1[P−1:0].
In operation, V0 will decrease as the current on BL0 increases. Likewise, V1 decreases as current on BL1 increases.
In the first step, CK1 and CK2 are set so one of nodes N0 and N1 is connected to node 650 to establish a primary current leg for the current mirror circuit. In this first step, if V1 is greater than V0 (current on BL1 is lower) the output of the comparator 630 sets B1[P] to “1”, and the sign is positive. If V1 is less than V0 (current on BL1 is higher) the output of the comparator 630 sets B1[P] to “0”, and the sign is negative.
In the next steps (the number of steps depends on the size of the counter), the magnitude of the current difference is determined. The setting of the control signals CK1 and CK2 depends on the sign.
More generally, the control circuits are configured to execute a method including sensing a difference in respective currents IBL0 and IBL1 on the first and second bit lines, including determining a sign of the difference by comparing one of currents IBL0 and IBL1 to a reference current, and determining a magnitude of the difference including selecting one of currents IBL0 and IBL1 in dependence on the sign and comparing the selected one of the currents to a sequence of reference currents.
Table 5 illustrates representative outputs for sense circuitry module “i” for a case in which P=2.
A sequence of operations can include the following for a condition in which the current on the first bit line BL0 is about 0 μA, and the current on the second bit line BL1 is about 3 μA:
A sequence of operations can include the following for a condition in which the current on the first bit line BL0 is about 3 HA, and the current on the second bit line BL1 is about 0 μA:
Other implementations of the method described in this section can include a non-transitory computer readable storage medium storing instructions executable by a processor to perform any of the methods described above. Yet another implementation of the method described in this section can include a system including memory and one or more processors operable to execute instructions, stored in the memory, to perform any of the methods described above.
A number of flowcharts illustrating logic executed by a memory controller or by a memory device are described herein. The logic can be implemented using processors programmed using computer programs stored in memory accessible to the computer systems and executable by the processors, by dedicated logic hardware, including field programmable integrated circuits, and by combinations of dedicated logic hardware and computer programs. With all flowcharts herein, it will be appreciated that many of the steps can be combined, performed in parallel or performed in a different sequence without affecting the functions achieved. In some cases, as the reader will appreciate, a rearrangement of steps will achieve the same results only if certain other changes are made as well. In other cases, as the reader will appreciate, a rearrangement of steps will achieve the same results only if certain conditions are satisfied. Furthermore, it will be appreciated that the flow charts herein show only steps that are pertinent to an understanding of the invention, and it will be understood that numerous additional steps for accomplishing other functions can be performed before, after and between those shown.
Embodiments illustrated in
While the present invention is disclosed by reference to the preferred embodiments and examples detailed above, it is to be understood that these examples are intended in an illustrative rather than in a limiting sense. It is contemplated that modifications and combinations will readily occur to those skilled in the art, which modifications and combinations will be within the spirit of the invention and the scope of the following claims.
