This disclosure relates generally resistive elements such as those used in certain analog computing applications. This disclosure more specifically relates to a network of resistive elements with improved dynamic range and linearity.
In certain analog computers, which find application in, among other things, artificial intelligence, networks of resistive elements (or units having respective resistance elements) are used to control current levels for computing. The range of conductance of the resistive elements significantly affects the performance of the analog computers. Efforts therefore are ongoing in resistive element design.
Aspects of the present disclosure are best understood from the following detailed description when read with the accompanying figures. It is noted that, in accordance with the standard practice in the industry, various features are not drawn to scale. In fact, the dimensions of the various features may be arbitrarily increased or reduced for clarity of discussion.
The following disclosure provides different embodiments, or examples, for implementing different features of the provided subject matter. Specific examples of components and arrangements are described below to simplify the present disclosure. These are, of course, merely examples and are not intended to be limiting. For example, the formation of a first feature over or on a second feature in the description that follows may include embodiments in which the first and second features are formed in direct contact, and may also include embodiments in which additional features may be formed between the first and second features, such that the first and second features may not be in direct contact. In addition, the present disclosure may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed.
Further, spatially relative terms, such as “beneath,” “below,” “lower,” “above,” “upper” and the like, may be used herein for ease of description to describe one element or feature's relationship to another element(s) or feature(s) as illustrated in the figures. The spatially relative terms are intended to encompass different orientations of the device in use or operation in addition to the orientation depicted in the figures. The apparatus may be otherwise oriented (rotated 90 degrees or at other orientations) and the spatially relative descriptors used herein may likewise be interpreted accordingly.
Analog computing is useful in a wide range of applications, especially in computations, simulations and implementations relating to complex systems, such as neurological systems. For example, artificial neural networks (or simply “neural networks” as used in this disclosure) have been used to achieve machine learning in artificial intelligence (“AI”) systems. One example type of neural network includes logically sequentially arranged layers of nodes, or artificial neurons. The layers include an input layer, an output layer, a one or more intermediate layers (so-called “hidden layers”). The nodes in input layer receives input signals from external sources, akin to signals from synapse in biological systems, and output signals to the hidden layers. Each node in a hidden layer receives signals from the nodes in the immediate upstream layer and outputs signals to the nodes in the immediate downstream layer. Each node in the output layer receives the signals from the last hidden layer and produces an output signal. In some example neural networks, the output signal from each node in a layer is a function of the weighted sum of signals from all nodes in the upstream layer. That is, each signal, xi, from the upstream layer is multiplied (weighted) by a weight, wij, and a function, ƒ of the sum of the weighted signals, Σiwijxi, is the signal, YjB, generated at the j-th node of the downstream layer:
Y
j
B=ƒ(Σiwijxi),
where ƒ can be of a variety of forms suitable for neural network implementations. Examples of such functions include linear (e.g., ƒ(x)=x, or YjB=Σiwijxi) the sigmoid function (ƒ(x)=1 (1+e−x)), and rectifier function (ƒ(x)=max(0, x)).
In some embodiments, analog computing system includes a resistor network connecting successive layers of nodes, such as between the input layer and the first hidden layer. In these embodiments, each node in a layer is connected to several, or all, of the nodes in the upstream layer through a respective resistive assembly, the resistance of which is adjustable. The signal, in the form of the current, generated at each node is thus the sum of the products between the currents at the nodes of the upstream layer and the conductances of the respective resistive assemblies. The conductances are therefore the weights for the signals from the respective nodes in the upstream layer. In some embodiments, each resistive assembly includes multiple switched resistive branches connected in parallel to each other.
As a specific example, as shown in
The resistive assembly is symbolically illustrated by a resistive element R 132 and a selecting element 134 but are implemented by multiple resistive and selective elements, as explained in more detail below. Each resistive assembly 130 has a conductance, Gij=1/Rij, where Rij is the resistance of the resistive assembly. For a node current, Ij, from the node Ni, the resistive assembly contributes a current Gij·Ii to the current, at the node Mj. Ij is thus a sum of Gij·Ii over all i=1, 2, . . . , n:
I
j=Σi=1nGijIi,j=1,2, . . . ,m.
The signal (current) at each node M1 is thus a weighted sum of the signals (currents) at the upstream nodes:
I
j=Σi=1nwijIi, where wij=Gij,j=1,2, . . . ,m.
Referring to
Each branch includes a resistive element and a selecting, or switching element. In this example, resistive branch 210 includes a resistor 212, having a resistance R0 (and conductance G0=1/R0), in series with a switching transistor 214 (such as a field-effect transistor (FET)), which can be turned on or off by a selecting signal S0 at the control terminal (such as the gate) of the switching transistor 214 to select and de-select the resistor 212. Resistive branch 220 includes a resistor 222, having a resistance R1 (and conductance G1=1/R1), in series with a switching transistor 224 (such as a field-effect transistor (FET)), which can be turned on or off by a selecting signal S1 at the control terminal (such as the gate) of the switching transistor 224 to select and de-select the resistor 222. Resistive branch 230 includes a resistor 222, having a resistance R2 (and conductance G2=1/R2), in series with a switching transistor 234 (such as a field-effect transistor (FET)), which can be turned on or off by a selecting signal S2 at the control terminal (such as the gate) of the switching transistor 234 to select and de-select the resistor 232. Resistive branch 240 includes a resistor 242, having a resistance R3 (and conductance G3=1/R3), in series with a switching transistor 244 (such as a field-effect transistor (FET)), which can be turned on or off by a selecting signal S3 at the control terminal (such as the gate) of the switching transistor 244 to select and de-select the resistor 242. Different
As shown in
The resistive elements depicted in the specific example in
Given a pattern of the resistive elements selected (i.e., the corresponding switching transistors turned on) in a resistive assembly 130, the conductance, G, of the assembly is the sum of the conductances of the selected resistors. For a resistive assembly 130 having N switched resistive elements 210, . . . , 250 of equal conductance g, G has N+1 values: 0, 1g, 2g, . . . , Ng. More values of G can be attained with resistive elements of different conductances.
For resistive elements, such as RRAMs, that are capable of having multiple levels of resistance values, more values of total conductance G can be achieved. For a parallel combination of N resistive elements, each of which have L programmable conductance values, g1, g2, . . . , gL, the total conductance G can be programmed to have the levels:
Thus, the dynamic range of the connection between two nodes is increased from L levels spanning g1 to gL for a single resistive element to Total(N,L) levels spanning N·g1 to N·gL, where Total(N,L) is defined as
In some embodiments, all resistive elements have substantially the same conductance. In such embodiments, the number of the levels of G, Total(N,L), as a function of N and L are shown in the following table:
For example, it is possible to achieve 330 levels of G with four resistive elements, each having eight levels of resistance, in a resistive assembly. More resistance levels can be achieved with resistive elements of non-identical resistances.
Even though, as shown in the table above, a large number of programmable conductance levels can be achieved with some embodiments, a subset of the levels may be selected in certain actual applications. For example, in certain neural network implementations by analog computing it is desirable to have the conductance G be linear in the programming pulse, i.e., having substantially equal size increments. However, as conductance of a resistive element is the reciprocal of its resistance, the conductance, 1/R, is highly non-linear, with the resistance states of interest clustered around the lower end of the resistance values of the element. At the same time, the smaller the difference between the value of resistive states the more precision and time are required to write the state. Improved linearity, in accordance with some embodiments, can be achieved by limiting the selection of levels to a subset of all possible levels. For example, in selecting the resistance of a multi-level resistance device, the selection can be limited to only some of the available higher-resistance values, such that the successively selected conductance values are spaced from each other substantially equally. At the same time, to address the more precision and time required to write the state for low-resistance end of the resistive states, more resistive elements can be connected in parallel to fine tune the conductance levels.
In some embodiments, such as those shown in
In some embodiments, in operation 500, as outlined in
In some embodiments, during operation, such as inferencing for neural network, all the selectors are enabled in an initialization step. During training, the resistive elements are programmed (have their respective resistances set) one at time by disasserting N−1 signals and only asserting the control signal corresponding to the resistive element to be programmed. Resistive elements are selected according to the conductance required, or incrementally selected or deselected. The selected programmed resistive elements are then placed in parallel connection with each other to provide the total conductance (i.e., weight) for each pair of node in the neighboring layers.
In some embodiments, each group of N parallel-connected resistive elements, such as those shown in
In further embodiments, an example of which is shown in
Certain embodiments disclosed herein offers advantages over traditional analog computing devices. With the use of parallel resistive elements, fewer resistance levels per resistive element are needed, the precision in the signals for setting those levels in each resistive element (e.g., RRAM) becomes less critical; the write time for such resistive element can thus be reduced, thereby increasing the speed of analog computing. Moreover, redundancy can be achieved by providing extra resistive elements in each resistive assembly; the extra resistive elements can be selected as needed, such the case in which any other resistive element is defective.
Thus, certain embodiments disclosed herein achieve a high number of weight levels for analog computing using resistive elements, such as MRAM, that have limited number of resistance levels, and afford faster write time and redundancy.
According to certain embodiments, each resistive unit in a resistive network of resistive units is made up of multiple resistive elements, which can be arranged in a parallel configuration. Each of the resistive elements can be programmable (e.g., switched on or off). For example, each of the resistive elements can be a serial combination of a resistor and a switch device, such as a switching transistor.
According to certain other embodiments, a method of analog computing includes configuring multiple resistive elements in each of multiple resistive units and configuring the resistive units into a network. The configuration of the resistive elements can be, for example, arranging them into a parallel combination. The method further includes programming each resistive unit, for example, by switching individual resistive elements into, or out of, the parallel combination.
Thus, in accordance with a disclosed embodiment, an analog neural network, or more generally, an analog computer, includes a first plurality of nodes, a second plurality of nodes, and a first plurality of resistive assemblies. Each of the first plurality of resistive assemblies has a plurality of resistive elements connected to each other in parallel, and at least a subset of the first plurality of resistive elements are switched resistive elements. At least a subset of the first plurality of nodes are connected to a common one of the second plurality of nodes via respective ones of the first plurality of resistive assemblies.
In accordance with other disclosed embodiments, a method of computing includes receiving a first plurality of signals at a respective first plurality of nodes, and selecting at least one subset of a plurality of resistive elements in each of a first plurality of resistive assemblies. For each of the first plurality of resistive assemblies, the selected resistive elements are configured into a parallel combination. At least one subset of the signals from a respective subset of the first plurality of nodes is transmitted to a first one of a second plurality of nodes via the respective parallel combinations.
In accordance with a further embodiment, an artificial neural network includes an upstream layer comprising a first plurality of nodes adapted to receive a respective plurality of signals, and a downstream layer comprising a second plurality of nodes. A plurality of resistance assemblies is provided, each of which comprises a plurality of resistive elements in a parallel configuration, where at least one of the resistive elements is a switched resistive element. The switched resistive element in each resistive assembly include a resistive device and a switching device configured to connect the resistive device in parallel with the other resistive elements in the resistive assembly.
The foregoing outlines features of several embodiments so that those skilled in the art may better understand the aspects of the present disclosure. Those skilled in the art should appreciate that they may readily use the present disclosure as a basis for designing or modifying other processes and structures for carrying out the same purposes and/or achieving the same advantages of the embodiments introduced herein. Those skilled in the art should also realize that such equivalent constructions do not depart from the spirit and scope of the present disclosure, and that they may make various changes, substitutions, and alterations herein without departing from the spirit and scope of the present disclosure.
Number | Date | Country | |
---|---|---|---|
62750349 | Oct 2018 | US |