Patent Application
20210343343
This disclosure relates generally to a compute-in-memory (CIM) architecture, and, more particularly, to the inputs for the analog-to-digital converters (ADCs) used therein.
Artificial neural network (ANN) applications are increasing in number and use. One particularly common type of ANN is a convolutional neural network (CNN), which is an ANN that includes one or more convolutional layers. A convolutional layer uses one or more kernels (also known as filters), which are arrays of weights, to process one or arrays of an input tensor and generate an output tensor. Running inferences with a CNN generally requires numerous computations to perform the many requisite tensor operations. In other words, using CNNs requires performing a vast number of matrix multiplications involving multiplying arrays of weights by corresponding arrays of inputs (also called activations). The computational and power resources of any device are, however, limited. The resources of some electronic devices, such as, for example, mobile devices and Internet of Things (IoT) devices, are particularly limited in comparison to desktop or server devices. One newly emerging architecture that allows for a significant reduction in power used, albeit at a cost of some reduction in accuracy, is the compute-in-memory (CIM) architecture using modified static random-access memory (SRAM) cells.
An exemplary electronic computing device may contain multiple CIM arrays. A CIM array comprises an array of modified SRAM cells programmable to store weights of a corresponding CNN. While the electronic computing devices comprises various digital-domain-computing components, the CIM array combines digital-domain components, analog-domain components, and converter components to convert between the two. The conversions may introduce noise or other inaccuracies into the computations of the electronic computing device. Accordingly, systems and methods to assist in mitigating deleterious effects of conversions between digital-domain components and analog-domain components would be desirable.
Certain aspects of the present disclosure are directed to an electronic device comprising a compute-in-memory (CIM) array comprising a plurality of columns. Each column comprises a plurality of CIM cells connected to a read bitline, a plurality of offset cells configured to provide a programmable offset value for the column, and an analog-to-digital converter (ADC) having the read bitline as a first input and configured to receive the programmable offset value. Each CIM cell is configured to store a corresponding weight
Certain aspects of the present disclosure are directed to a method for an electronic device wherein the electronic device comprises a compute-in-memory (CIM) array, the CIM array comprises a plurality of columns, and each column comprises a plurality of CIM cells connected to a read bitline, a plurality of offset cells, and an analog-to-digital converter (ADC) having the bitline as a first input. The method comprises: storing a corresponding weight in each CIM cell, and providing, to the ADC, by the plurality of offset values, a programmable offset value for the column.
Additional aspects, advantages, and features of the present disclosure may become apparent after review of the entire application.
The disclosed embodiments will hereinafter be described in conjunction with the appended drawings, provided to illustrate and not to limit the disclosed embodiments, wherein like designations denote like elements, and in which:
With reference to the Figures, several exemplary aspects of the present disclosure are described. The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any aspect described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other aspects.
The input module 106 routes input from digital components of the device 100, such as, for example, the processor 102 or the memory 103, for use by the cells of the CIM array 105. In some embodiments, the input module 106 includes one or more digital-to-analog converters (DACs) to convert digital input signals into analog signals for use by cells of the CIM array 105. The ADC module 107 converts analog outputs of the CIM array 105 into digital outputs for provision to digital components of the device 100, such as the processor 102 or the memory 103. The device 100 may be, for example, in the form of a system on a chip (SoC). The device 100 may include additional components and/or chips (not shown) and itself be a mobile computing device or an IoT device. The CIM array 105 may be used by the device 100 to perform convolution operations such as, for example, those used in neural-network inferencing. It should be noted that, while multiply-and-accumulate (MAC) is typically used to refer to a digital-domain operation, as used herein, it may also refer to a corresponding operation performed in a mixed analog and digital domains devices, such as the CIM modules 101, which will be explained in greater detail below.
Each filter, e.g., exemplary filter 212, of the 64 filters of tensor 214, has dimensions of one by five by two (1×5×2) (together, the filters of tensor 214 make up one 64×5×2 tensor). Each 1×5×2 filter, e.g., filter 212, is convolved with the input 202 to generate one 1×34×10 layer of an output 204 (together, the layers of the output 204 make up a tensor of dimensions 64×34×10). Each 1×34×10 layer of the output 204 represents three thousand four hundred (3400) MAC operations for 340 5×2 receptive fields of the input 202—such as exemplary receptive field 206—and the corresponding filter—such as exemplary filter 212. Note that the layer of system 200 uses a stride of one for the convolution of the input 202; alternative implementations may use different strides resulting in a different number of receptive fields and a differently sized output tensor. Each value of the output tensor 204 represents ten MAC operations for a 5×2 filter and a corresponding 5×2 receptive field.
The six hundred and forty (640) weights of the filter tensor 214 (64×5×2) may be stored in 640 corresponding cells of the CIM array 201, which has at least 10 rows and 64 columns. Note that each cell of the CIM array 201 is both a storage unit and a processing unit. Each column of the CIM array 201 may store the ten values of a corresponding filter 212, essentially transforming the 2-dimensional filter 212 into a one-dimensional vector using any suitable mapping. Note that, for simplicity, the CIM array 201 is illustrated with only a few lines for the input and the output representing, respectively, the rows and columns of the CIM array 201.
Over three hundred and forty (340) operational cycles, the ten (10) values of each of the 340 5×2 receptive fields (e.g., receptive field 206) of the input 202 are input to the CIM array 201 using the wordlines, e.g., 216. Note that each 2-dimensional receptive field is essentially transformed into a one-dimensional vector using the same mapping that is used for the filter. The values of the receptive field are multiplied by the corresponding weights, with the respective products of each of the 64 columns summed to produce a 1×1×64 output tensor (e.g., output tensor 210), which forms a part of the output tensor 204. The output tensor 204 corresponds to an aggreggation of the 340 1×1×64 output tensors for the 340 corresponding receptive fields (e.g., receptive field 206) of the input 202.
The CIM array 201 includes wordlines 216 (e.g., wordlines corresponding to rows of the compute-in-memory array 201) through which the CIM array 201 receives the values of the receptive fields (e.g., receptive field 206) as well as read bitlines 218 (corresponding to columns of the CIM array 201). Note that the CIM array 201 includes additional wordlines and bitlines (not shown in
The results of the multiplications by the CIM cells 314(j)(1)-314(j)(r) of each column j are output to a corresponding read bitline 312(j), such as, for example, read bitline 312(1) for column 1. The outputs for the CIM cells 314(j)(1)-314(j)(r) of a column j are summed by the corresponding bitline 312(j) and provided as an input to a corresponding ADC 308(j) in the ADC module 304. Each ADC 308 also receives a reference voltage VREF for use in converting the analog value on the bitline 312(j) into a corresponding digital value. An ADC 308 comprises one or more comparators. An ADC 308 may be any suitable type of ADC, such as, for example, a flash ADC. In some implementations, each ADC 308 is simply a comparator that compares the value on the corresponding bitline 312 to VREF and provides a 1-bit corresponding output based on the comparison. The outputs of the ADCs 308 are provided to digital processing module 313 for further processing, where the further processing may include operations for multi-bit implementations of the CIM module 300.
Multiplication occurs in each CIM cell 314 and accumulation (summation) by column of the CIM cell 314 multiplication results occurs using the corresponding read bitlines 312. The multiplication in each CIM cell 314 may be in the form of an AND operation of the corresponding activation and weight. In other implementations, other operations may be used to simulate the multiplication phase of a MAC operation. The resultant product is then stored as a charge on a corresponding capacitor 413. Note that, for improved readability, only exemplary capacitor 413(2)(1) of CIM cell 314(2)(1) is labeled in
For example, the read bitline 312(1), according to this aspect, accumulates charges of all the CIM cells in column one. For the accumulation phase, the RWLs 411(1)-411(3) are set to high to turn on the NMOS FETs 417 (such as, for example, the labeled exemplary NMOS 417(3)(2) of CIM cell 314(3)(2)), which are used as read-wordline transistors, and pass the charges on the capacitors 413 to the corresponding bitlines 314. The summed charge of the bitline 312(1) is input to ADC 308(1) where, based on the reference voltage VREF, the summed charge is converted to a digital output (e.g., a digital signal/number), which may be an input to a next layer of the neural network. If, given the exemplary weight values shown in the SRAM cells 419, the activations on the input lines 310(1)-310(3) were, respectively, for example, 1, 0, 1, then the sums of the bitlines 312(1)-312(3) would be, following the multiplications, respectively, 0+0+1=1, 1+0+0=1, and 1+0+1=2.
Note that, as used herein, input-line transistor refers generally to an active component of the CIM cell—such as, for example, PMOS 415—that is controlled by the input line 310 to selectively connect CIM cell components—such as, for example, the SRAM cell 419 and the capacitor 413. Similarly, read-wordline transistor refers generally to an active component of the CIM cell—such as, for example, NMOS 417—that is controlled by the read wordline to selectively connect CIM cell components—such as, for example, the capacitor 413 and the output to the read bitline 312.
Storing of weights in the SRAM cells 419 may be performed once for a multitude of activations, using programming bitlines 520 and 521 and programming wordline 516. For example, in operation, the SRAM cell 419 receives only one bit of information at nodes 517 and 519 via a wordline 516. For example, when the information received at node 517 is a “1” (e.g., high voltage), the other node 519 does not receive information or the information received at node 519 is a “0” (e.g., low voltage). Similarly, when the information received at node 517 is a “0,” the information received at node 519 is a “1.” The storing of weights may be followed by an an activation input and multiplication step to charge the capacitors 413 in accordance with the corresponding products. The transistor 415 is activated by an activation signal through the input line 310 and the transistor 417 is activated by a signal through the RWL 411. Each RWL 411 is provided to all the CIM cells 314 in a row of a CIM array and the RWL 411 is turned ON to accumulate the values of the column of CIM cells 314 along the corresponding read bitlines 312.
If node 517 is a “0,” (e.g., when the stored weight value is “0”), then the capacitor 413 will not be charged, even if a LOW input line 310 indicates an activation of “1” at the gate of the transistor 415. Accordingly, no corresponding charge will be provided to the corresponding read bitline 312. However, if node 517, which corresponds to the weight value, is a “1,” and the input line 310 is set to low (e.g., when the activation value is “1”), which turns on the PMOS transistor 415, then the capacitor 413 will get charged. After the capacitor 413 is charged, the transistor 415 is turned off so the charge remains stored in the capacitor 413. To move the charge from the capacitor 413 to the read bitline 312, the NMOS transistor 417 is turned on by the RWL 411, causing the NMOS transistor 417 to act as a short between the capacitor 413 and the corresponding read bitline 312. CIM cell 314 may contain additional circuitry such as, for example, reset circuitry (not shown) to reset the charge on the capacitor 413 as needed.
Table 1 illustrates an exemplary operation of the CIM module 314 of
The CIM array 705 comprises a c×r array of CIM cells 314—namely, CIM cells 314(1)(1)-314 (c)(r). Note that, for improved readability, only a few exemplary CIM cells 314 are labeled in
Offset cells 731 may be modified versions of CIM cells 314. In one implementation, offset cells 731 are identical to CIM cells 314 but disconnected from the input lines 310 and with transistor 415 connected to be always on. In one implementation of offset cells 731, instead of a transistor 415, there is a conductive short connection between the SRAM cell 419 and the capacitor 413. These modifications represent a couple of exemplary ways to prevent the input lines 310 from altering the outputs of the offset cells 731 from their programmed values.
The CIM array 705 may have equal numbers of CIM cells 314 and offset cells 731 symmetrically arranged in each column 730 for greater design, routing, and/or manufacturing convenience. In addition, having an equal number of offset cells 731 allows for relatively large voltage-reference offset values. Note, however, that a symmetric equal-number arrangement is not required, and alternative implementations may have unequal numbers of CIM cells 314 and offset cells 731, asymmetrically arranged, in each column 730.
In operation, the offset cells 731 are programmed together with the CIM cells 314. In other words, the offset values, which may be predetermined for each column 730, are stored in the offset cells 731 at substantially the same time as the weights are stored in the CIM cells 314. After the weights and offsets are loaded into the CIM array 705, operation proceeds substantially as described above in reference to
The CIM array 805 comprises a c×r array of CIM cells 314, namely, CIM cells 314(1)(1)-314(c)(r), as well as a c×o array of offset cells 844, namely, offset cells 844(1)(1)-844(c)(o). Note that, for improved readability, only a few exemplary CIM cells 314 and offset cells 844 are labeled in
The offset cells 844 may be identical to, or modified versions of, the CIM cells 314. In one implementation, the offset cells 844 are identical to the CIM cells 314, where the offset input lines 841 for the offset cells 844 activate the corresponding transistors of the offset cells 844 every time new input data is provided to the CIM cells 314 via the input lines 310; in other words, the inputs on lines 841 always correspond to activations of “1.”
In one implementation of the CIM module 800, the CIM array 805 comprises a c×(r+o) array of flexible cells whose function as a CIM cell 314 or an offset cell 844 is dynamically determined during operation. In other words, for any set of activations, a first number of rows may be designated as, and be operated to function as, rows of CIM cells 314 and a second number of rows may be designated as, and be operated to function as, rows of offset cells 844. Note that although
In one alternative implementation, the transistors 415 of the offset cells 844 are connected to be always on and either the offset cells 844 are disconnected from the offset input lines 841 or the offset input lines 841 are absent. In one implementation of offset cells 844, instead of a transistor 415, there is a conductive short connection between the SRAM cell 419 and the capacitor 413.
The CIM array 805 may have CIM cells 314 and offset cells 844 sharing internal and external design elements within each column 830 for greater design, routing, and/or manufacturing convenience. Typically, each column 830 would have many more rows of CIM cells 314 than offset cells 844, which would be sufficient to provide voltage offset capabilities for most purposes.
In operation, the offset cells 844 are programmed together with the CIM cells 314. In other words, the offset values, which may be predetermined for each column 830, are stored in the offset cells 844 at substantially the same time as the weights are stored in the CIM cells 314. After the weights and offsets are loaded into the CIM array 805, operation proceeds substantially as described above in reference to
In one alternative implementation, the offset values correspond to the inputs instead of the weights and are dynamically programmed via the offset input lines 841 with each new set of input values provided to the CIM array 805 via the input lines 310. In this implementation, the SRAM cells of the offset cells 844 are preprogrammed to all have “weight” values of “1” so that when the “weights” of the offset cells are multiplied by the inputs on the offset input lines 841, the offset cells output the corresponding offset input value to the to the corresponding read bitlines 833.
The above detailed description set forth above in connection with the appended drawings describes examples and does not represent the only examples that may be implemented or that are within the scope of the claims. The terms “example” or “exemplary,” when used in this description, means “serving as an example, instance, or illustration,” and not “preferred” or “advantageous over other examples.” The detailed description includes specific details for the purpose of providing an understanding of the described techniques. These techniques, however, may be practiced without these specific details. In some instances, well-known structures and apparatuses are shown in block diagram form in order to avoid obscuring the concepts of the described examples.
It is also noted that the operational steps described in any of the exemplary aspects herein are described to provide examples. The operations described may be performed in numerous different sequences other than the illustrated sequences. Furthermore, operations described in a single operational step may actually be performed in a number of different steps. Additionally, one or more operational steps discussed in the exemplary aspects may be combined. It is to be understood that the operational steps illustrated in the flow diagrams may be subject to numerous different modifications as will be readily apparent to one of skill in the art. Those of skill in the art will also understand that information and signals may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips that may be referenced throughout the above description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.
The various illustrative circuits described in connection with aspects described herein may be implemented in or with an integrated circuit (IC), such as a processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic device. A processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.
As used herein, a phrase referring to “at least one of” a list of items refers to any combination of those items, including single members. As an example, “at least one of: a, b, or c” is intended to cover a, b, c, a-b, a-c, b-c, and a-b-c, as well as any combination with multiples of the same element (e.g., a-a, a-a-a, a-a-b, a-a-c, a-b-b, a-c-c, b-b, b-b-b, b-b-c, c-c, and c-c-c or any other ordering of a, b, and c).
The present disclosure is provided to enable any person skilled in the art to make or use aspects of the disclosure. Various modifications to the disclosure will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
