Patent Application
20220092396
The present invention relates to a product-sum operation device, a logical operation device, a neuromorphic device, and a product-sum operation method.
In a conventional neural network, input signals are multiplied by weighting factors, and a value of a total sum thereof is input to an activation function, whereby an output is acquired. Thus, a product-sum operation has been attempted to be realized using an analog circuit by combining two or more memristors of which resistance continuously changes and reading a total sum of current values output therefrom.
In a learning process of a neural network, a memristor assigned to each synapse has resistance value changing such that it has a predetermined weighting factor and maintains this value even when power is cut off. In a inference process, pulse width modulation (PWM) control in which a length of a voltage pulse changes in accordance with a level of input data using a value of a memristor in which information is maintained is used.
For example, in Patent Literature 1, a product-sum operation device in which information written into a resistance-changing-type variable resistance element from a data line and a bit line is stored in a capacitor as an amount of electric charge has been disclosed.
Japanese Patent No. 5160304
A memristor has a circuit configuration in which a parasitic capacitor and a parasitic resistor are connected in parallel as an equivalent circuit. For this reason, in a case in which an input signal that is a voltage pulse is input to the memristor, a transient response due to charging of the parasitic capacitor and a transient response due to discharging from the parasitic capacitor are generated. Thus, in the product-sum operation device described above, electric charge according to such transient responses is stored in the capacitor in addition to electric charge in a steady state in which such transient responses are not generated, and thus there are cases in which the capacitor becomes saturated, and a product-sum operation is not able to be executed.
An object of the present invention is to provide a product-sum operation device, a logical operation device, a neuromorphic device, and a product-sum operation method capable of avoiding a situation in which a product-sum operation is not able to be executed due to saturation of a capacitor.
According to one aspect of the present invention, there is provided a product-sum operation device including: a plurality of product operation units configured to generate output signals by multiplying input signals corresponding to input values by weighting factors and output the output signals; a current detection unit configured to execute a current detecting process in which a current output from the plurality of product operation units with a predetermined time delay from input of the input signal and a current output from the plurality of product operation units at a predetermined time interval thereafter are detected in a time span from a first transient response being converged to a steady state to before occurrence of a second transient response, the first transient response being due to charging to a parasitic capacitance of the product operation units by input of the input signal and the second transient response being due to discharging from the parasitic capacitance of the product operation units by input of the input signal; and a sum operation unit configured to calculate a value relating to a total sum of the output signals based on currents detected at the predetermined time intervals by the current detecting unit.
In addition, in one aspect of the present invention, each of the plurality of product operation units includes a magnetoresistive effect element exhibiting a magnetoresistive effect.
In addition, in one aspect of the present invention, the sum operation unit calculates a product of a total current that is a sum of currents detected at the predetermined time intervals by the current detecting unit and a coefficient time, as the value relating to the total sum of the output signals.
In addition, in one aspect of the present invention, the coefficient time is a shortest length that can be taken by the input signals, the input signals have a length of an integer multiple of the coefficient time and are simultaneously input to the plurality of product operation units, and the current detecting unit executes the current detecting process in a period equal to the coefficient time.
In addition, in one aspect of the present invention, the current detecting unit ends the current detecting process at a time point at which a time equal to a longest length that can be taken by the input signals elapses after the current detecting process is executed for the first time.
In addition, in one aspect of the present invention, the current detecting unit ends the current detecting process in a case in which a current detected in the current detecting process is equal to a current acquired in a case in which no input signal is input to the plurality of product operation units.
In addition, according to one aspect of the present invention, there is provided a logical operation device including any one of the product-sum operation devices described above.
In addition, according to one aspect of the present invention, there is provided a neuromorphic device including any one of the product-sum operation devices described above.
In addition, according to one aspect of the present invention, there is provided a product-sum operation method by the product-sum operation device according to any one of claims 1 to 6, the product-sum method including: a product operation step that is a step of generating output signals by multiplying input signals corresponding to input values by weighting factors and outputting the output signals, using a plurality of product operation units; a current detecting step that is a step of executing a current detecting process in which a current output from the plurality of product operation units with a predetermined time delay from input of the input signal and a current output from the plurality of product operation units at a predetermined time interval thereafter are detected in a time span from a first transient response being converged to a steady state to before occurrence of a second transient response, the first transient response being due to charging to a parasitic capacitance of the product operation units by input of the input signal and the second transient response being due to discharging from the parasitic capacitance of the product operation units by input of the input signal; and a sum operation step that is a step calculating a value relating to a total sum of the output signals based on currents detected at the predetermined time intervals in the current detecting step.
According to the product-sum operation device, the logical operation device, the neuromorphic device, and the product-sum operation method described above, a product-sum operation device, a logical operation device, a neuromorphic device, and a product-sum operation method capable of avoiding a situation in which a product-sum operation is not able to be executed due to saturation of a capacitor can be provided.
An example of configuration of a product-sum operation device according to an embodiment will be described with reference to
Here, for example, the variable resistor 111R included in the product operation unit 111, as illustrated in
The magnetization direction of the magnetization fixing layer 1111 is fixed in the +z direction or the −z direction. Here, the magnetization being fixed means that the magnetization direction does not change at the time of an initial period for introducing the magnetic domain wall 1114 and before and after writing using a write current. In addition, for example, the magnetization fixing layer 1111 may be an in-plane magnetization film having in-plane magnetic anisotropy or a vertical magnetization film having vertical magnetic anisotropy.
The non-magnetic layer 1112 has a face directed in the +z direction being in contact with a face of the magnetization fixing layer 1111 that is directed in the −z direction and a face directed in the −z direction being in contact with the first region 1113 and the second region 1115. As illustrated in
The first region 1113, the magnetic domain wall 1114, and the second region 1115 form a magnetization free layer. The magnetization free layer is produced using a ferromagnetic material. A magnetization direction of the first region 1113 and a magnetization direction of the second region 1115 are opposite to each other in a direction parallel to the z axis. The magnetic domain wall 1114 is interposed between the first region 1113 and the second region 1115 in a direction parallel to the y axis.
The first magnetization supplying layer 1116 preferably does not overlap the magnetization fixing layer 1111 in a direction parallel to the z axis and has a face directed in the +z direction to be in contact with a face of the first region 1113 directed in the −z direction. The first magnetization supplying layer 1116 has a function of fixing a magnetization direction of a range of the first region 1113 that overlaps the first magnetization supplying layer 1116 in a direction parallel to the z axis to a desired direction. Furthermore, the write terminal 111Z is connected to a face of the first magnetization supplying layer 1116 directed in the −z direction. For example, the first magnetization supplying layer 1116 is produced using the same material as a ferromagnetic material that can be used for the magnetization fixing layer 1111, an antiferromagnetic body of IrMn or the like, a ferromagnetic body having a nonmagnetic intermediate layer of Ru, Ir, or the like interposed therebetween, or a material having a synthetic antiferromagnetic structure including a nonmagnetic body and a ferromagnetic body.
The second magnetization supplying layer 1117 preferably does not overlap the magnetization fixing layer 1111 in a direction parallel to the z axis and has a face directed in the +z direction being in contact with a face of the second region 1115 directed in the −z direction. The second magnetization supplying layer 1117 has a function of fixing a magnetization direction of a range of the second region 1115 that overlaps the second magnetization supplying layer 1117 in a direction parallel to the z axis to a desired direction. Furthermore, the common terminal 111Y is connected to a face of the second magnetization supplying layer 1117 directed in the −z direction. For example, the first magnetization supplying layer 1116 is produced using the same material as a ferromagnetic material that can be used for the magnetization fixing layer 1111, an antiferromagnetic body of IrMn or the like, a ferromagnetic body having a nonmagnetic intermediate layer of Ru, Ir, or the like interposed therebetween, or a material having a synthetic antiferromagnetic structure including a nonmagnetic body and a ferromagnetic body.
In the variable resistor 111R, each of the magnetization directions of the magnetization fixing layer 1111, the first region 1113, the second region 1115, the first magnetization supplying layer 1116, and the second magnetization supplying layer 1117 may be not only a direction parallel to the z axis but also a direction parallel to the x axis or a direction parallel to the y axis. Also in such a case, it is preferable that the magnetization direction of the magnetization fixing layer 1111 and the magnetization directions of the first region 1113, the second region 1115, the first magnetization supplying layer 1116, and the second magnetization supplying layer 1117 are parallel to each other. For example, in a case in which the magnetization direction of the magnetization fixing layer 1111 is the +y direction, the magnetization direction of the first region 1113 is the +y direction, the magnetization direction of the second region 1115 is the −y direction, the magnetization direction of the first magnetization supplying layer 1116 is the +y direction, and the magnetization direction of the second magnetization supplying layer 1117 is the −y direction.
The product operation unit 111 changes the position of the magnetic domain wall 1114 in a direction parallel to the y axis by adjusting a magnitude and a time of a write current flowing between the common terminal 111Y and the write terminal 111Z. In accordance with this, the product operation unit 111 can change the resistance value of the variable resistor 111R approximately linearly by continuously changing an area ratio between an area in which the magnetization directions are parallel and an area in which the magnetization directions are antiparallel. Here, the area in which the magnetization directions are parallel is an area of a part of the first region 1113 that overlaps the magnetization fixing layer 1111 in a direction parallel to the z axis. In addition, the area in which the magnetization directions are antiparallel is an area of a part of the second region 1115 that overlaps the magnetization fixing layer 1111 in a direction parallel to the z axis. A write current is input to the write terminal 111Z. A magnitude and a time of the write current are adjusted by at least one of the number and the time of current pulses.
The product operation units 111, 121, 211, 221, 311, 321, . . . , k11, k21 may be tunnel magnetoresistive effect elements. A tunnel magnetoresistive effect element includes a magnetization fixing layer, a magnetization free layer, and a tunnel barrier layer as a nonmagnetic layer. The magnetization fixing layer and the magnetization free layer are produced using ferromagnetic materials and have magnetization. The tunnel barrier layer is interposed between the magnetization fixing layer and the magnetization free layer. The tunnel magnetoresistive effect element can change a resistance value by changing a relation between the magnetization of the magnetization fixing layer and the magnetization of the magnetization free layer.
Referring back to
The input unit 101E inputs an input signal corresponding to an input value to the read terminals 111X and 121X. Similarly, the input unit 201E inputs an input signal corresponding to an input value to the read terminals 211X and 221X. The input unit 301E inputs an input signal corresponding to an input value to the read terminals 311X and 321X. The input unit k01E inputs an input signal corresponding to an input value to the read terminals k11X and k21X. All these input signals are voltage signals for which pulse width modulation (PWM) corresponding to input values is performed.
The product operation unit 111 generates an output signal by multiplying an input signal corresponding to an input value by a weighting factor and outputs the output signal. In other words, the product operation unit 111 reads a resistance value of the variable resistor 111R as a weighting factor, executes a product operation on an input signal input to the read terminal 111X to generate an output signal, and outputs the output signal from the common terminal 111Y. Similarly, each of the product operation units 121, 211, 221, 311, 321, . . . , k11, k21 multiplies an input signal corresponding to an input value by a weighting factor to generate an output signal and outputs the output signal.
The current detecting unit 10D executes a current detecting process in which a current output from the plurality of product operation units with a predetermined time delay from input of the input signal and a current output from the product operation units 111, 211, 311, . . . , k11 at a predetermined time interval thereafter are detected in a time span from a first transient response being converged to a steady state to before occurrence of a second transient response, the first transient response being due to charging to a parasitic capacitance of the product operation units by input of the input signal and the second transient response being due to discharging from the parasitic capacitance of the product operation units by input of the input signal. Similarly, the current detecting unit 20D executes a current detecting process in which a current output from the plurality of product operation units with a predetermined time delay from input of the input signal and a current output from the product operation units 121, 221, 321, k21 at a predetermined time interval thereafter are detected in a time span from a first transient response being converged to a steady state to before occurrence of a second transient response, the first transient response being due to charging to a parasitic capacitance of the product operation units by input of the input signal and the second transient response being due to discharging from the parasitic capacitance of the product operation units by input of the input signal. Details of the current detecting units 10D and 20D will be described later.
The sum operation unit 10S calculates a value relating to a total sum of output signals based on currents detected at predetermined time intervals by the current detecting unit 10D. Similarly, the sum operation unit 20S calculates a value relating to a total sum of output signals based on currents detected at predetermined time intervals by the current detecting unit 20D. Details of the sum operation units 10S and 20S will be described later.
Next, an example of a process executed by a product-sum operation device according to an embodiment will be described with reference to
For example, the input signal V1 is a voltage pulse having a length of 20 [ns] as a coefficient time and a pulse height of 0.1 mV. This 20 [ns] is an example of a shortest length that can be taken by the input signal. The input signal V2 is a voltage pulse having a length of 40 [ns] that is twice the coefficient time described above and a pulse height of 0.1 mV. The input signal V3 is a voltage pulse having a length of 80 [ns] that is four times the length of the coefficient time described above and a pulse height of 0.1 mV. A time at which the input signal V1 is input to the product operation unit 111, a time at which the input signal V2 is input to the product operation unit 211, and a time at which the input signal V3 is input to the product operation unit 311, as illustrated in
The output signal A1 illustrated in
The output signal A2 illustrated in
The output signal A3 illustrated in
In a case that differences in the capacitance values of the parasitic capacitors 111C, 211C, and 311C are substantially ignorable, the first transient response T11, the first transient response T21, and the first transient response T31 are transient responses of which lengths and heights are almost the same, and the second transient response T12, the second transient response T22, and the second transient response T32 are transient responses of which lengths and heights are almost the same.
The first transient response T4 is a transient response that is generated by adding the first transient response T11, the first transient response T21, and the first transient response T31 illustrated in
The second transient response T41 is a transient response that is generated by adding the steady part S2 and the steady part S3 to the second transient response T12 illustrated in
The current detecting unit 10D executes a current detecting process of detecting currents output by the product operation units 111, 211, and 311 for every coefficient time between a time point at which the generation of the first transient response or the second transient response described above ends and a time point at which the generation of the next first transient response or the second transient response starts. The coefficient time described here, for example, is a period corresponding to the steady part S41, the steady part S42, and the steady part S43 illustrated in
For example, as denoted by a point D1 in
In addition, in a case in which a current detected in the current detecting process is the same as a current of a case in which no input signal is input to the product operation units 111, 211, 311, . . . , k11, the current detecting unit 10D ends the current detecting process. For example, as denoted by the point D5 in
Alternatively, the current detecting unit 10D ends the current detecting process at a time point at which a time that is equal to a longest length that can be taken by an input signal elapses after the current detection process is executed for the first time. For example, in a case in which the length of an input signal can take one of one time, two times, three times, . . . , 254 times, 255 times, and 256 times that of a shortest length of the input signal, the current detecting unit 10D stops the current detecting process at the 257-th period and thereafter.
In the example described with reference to
For example, a case in which, at a time t0, the input signal V1 is input to the product operation unit 111, the input signal V2 is input to the product operation unit 211, and the input signal V3 is input to the product operation unit 311 will be considered. In this case, as described with reference to
A state in which the magnitude of the current has entered the range of 3C-Δ [nA] to 3C+Δ[nA] is a steady state in which a steady current flows. For example, the magnitude of Δ [nA] is preferably 10 % p-p of the current 3 [nA] and is more preferably 5% p-p of the current 3 [nA].
For example, as illustrated in
The convergence time of the first transient response described above is equal to a time required for discharging from the parasitic capacitor and also can be calculated using the following Equation (1).
[Math. 1]
t=−C×R×ln(V1/V0) (1)
In addition, the convergence time of the second transient response described above is equal to a time required for discharging from the parasitic capacitor and also can be calculated using the following Equation (2).
[Math. 2]
t=−C×R×ln(1−V1/V0) (2)
Meanwhile, the second transient response T41 starts to be generated at a time 20 [ns]. Thus, the steady part 41 described above occurs in a period from the time t1 to a time 20 [ns]. The current detecting unit 10D executes a current detecting process at an arbitrary timing in the period of the steady part 41, in other words, from the time t1 to a time 20 [ns]. In addition, by determining the time such that a period from the time t1 to the time 20 [ns] is longer than the convergence time calculated using the time constant described above, the current detecting unit 10D can reliably execute the current detecting process in the steady part 41.
The current acquired using the current detecting process executed by the current detecting unit 10D may be converted into digital data through analog-digital conversion and be stored in a storage medium. In addition, the current detecting unit 10D may detect a current of a predetermined time in a period from a time at which the first transient response converges to form a steady state to a time before generation of the second transient response or may detect a current during a predetermined period. Furthermore, in a case in which a current during a predetermined period is detected, the current detecting unit 10D may set a statistical value of the detected current, for example, an average value or a median value thereof as a result of the current detecting process.
The sum operation unit 10S calculates a product of a sum current that is a sum of currents detected at predetermined time intervals by the current detecting unit 10D and the coefficient time as a value relating to a total sum of output signals.
For example, the sum operation unit 10S calculates a sum current 7 [nA] of currents 3C [nA], 2C [nA], C [nA], C [nA], 0 [nA], and 0 [nA] detected at times denoted by the point D1, the point D2, the point D3, the point D4, the point D5, and the point D6 illustrated in
Details described with reference to
Next, an example of a neural network operation executed by the product-sum operation device according to this embodiment will be described with reference to
Nodes 101, 201, 301, . . . , k01 form an input layer. Perceptrons 10 and 20 form a hidden layer or an output layer. The node 101 corresponds to the input unit 101E illustrated in
An arrow 111A corresponds to the product operation unit 111 and represents that an input value output by the node 101 is multiplied by a weighting factor, and a value corresponding to an output signal is input to the perceptron 10. Similarly, an arrow 121A corresponds to the product operation unit 121 and represents that an input value output by the node 101 is multiplied by a weighting factor, and a value corresponding to an output signal is input to the perceptron 20. These similarly apply also to arrows 211A, 221A, 311A, 321A, . . . , k11A, k21A.
The perceptron 10 corresponds to the current detecting unit 10D and the sum operation unit 10S illustrated in
As above, the product-sum operation device 1 according to the embodiment has been described. The product-sum operation device 1 does not need to include a capacitor that stores electric charge for calculating a value relating to a total sum of output signals based on currents detected at predetermined time intervals. Thus, the product-sum operation device 1 can avoid that the capacitor is saturated, and a product sum operation cannot be executed. Since the capacitor can be omitted in the product-sum operation device 1, space saving and cost saving can be realized by reducing the circuit scale.
In addition at least one of the product operation units 111, 121, 211, 221, 311, 321, . . . , k11, k21 included in the product-sum operation device 1 includes a magnetoresistive effect element exhibiting a magnetoresistive effect. The magnetoresistive effect element has higher parasitic capacitance than that of the other variable resistance elements and thus can more easily cause the saturation of the capacitor according to a transient response to occur than the other variable resistance elements. Thus, in a case in which a magnetoresistive effect element is included as the product operation unit described above, the effects described above are particularly useful.
In addition, in the product-sum operation device 1, input signals have lengths that are integer multiples of the coefficient time, which is a shortest length that can be taken, and are simultaneously input respectively to the product operation units 111, 121, 211, 221, 311, 321, . . . , k11, k21. Then, the product-sum operation device 1 executes the current detecting process in the same period as that of the coefficient time. Thus, the product-sum operation device 1 can reliably detect a current in a case in which the first transient response and the second transient response reliably converge to form a steady state and can execute an accurate product sum operation.
In addition, the product-sum operation device 1 ends the current detecting process at a time point at which a time equal to the longest length that can be taken by input signals has elapsed after the current detecting process is executed for the first time. For this reason, the product-sum operation device 1 can set a time in which the current detecting process is executed to be constant and can simply the process.
In addition, the product-sum operation device 1 ends the current detecting process in a case in which a current detected in the current detecting process is equal to a current acquired in a case in which no input signal is input to a plurality of the product operation units 111, 121, 211, 221, 311, 321, . . . , k11, k21. For this reason, the product-sum operation device 1 can end the current detecting process early.
In the embodiment described above, although a case in which the time at which the input signal V1 is input to the product operation unit 111, the time at which the input signal V2 is input to the product operation unit 211, and the time at which the input signal V3 is input to the product operation unit 311 are the time t0 has been described as an example, the configuration is not limited thereto. In other words, the times at which these three input signals are input may be different from each other. In such a case, the coefficient time may be a period from a time point at which generation of the first transient response or the second transient response ends to a time point at which generation of the next first transient response starts.
The product-sum operation device 1 described above may be included in a logical operation device or a neuromorphic device. The logical operation device described here is a logical circuit, for example, such as an AND circuit or an OR circuit that is formed by combining a plurality of product-sum operation devices 1. In addition, a logical operation described here has a concept that includes deep learning. Furthermore, the neuromorphic device described here is a device that applies the structure of the brain and a structure in which nerve cells called neurons ignite and is used for machine learning and the like.
A program used for realizing the function of each device such as the product-sum operation device 1 according to the embodiment described above may be recorded on a computer-readable recording medium, and a computer system may be caused to read and execute the program recorded on this recording medium for performing the process.
The computer system described here may include an operating system (OS) and hardware such as peripherals. The computer-readable recording medium, for example, includes a writable nonvolatile memory such as a floppy dick, a magneto-optical disk, a read only memory (ROM), or a flash memory, a portable medium such as a digital versatile disc (DVD), and a storage device such as a hard disk built into the computer system and includes a medium storing a program for a predetermined time such as an internal volatile memory of a computer system serving as a server or a client in a case in which the program is transmitted through a network or a communication line.
Furthermore, the program described above may be transmitted from a computer system storing the program in a storage device or the like to another computer system through a transmission medium or using transmission waves in a transmission medium. Here, the transmission medium transmitting the program represents a medium having a function of transmitting information such as a network including the Internet and the like or a communication line including a telephone line and the like.
In addition, the program described above may be used for realizing some of the functions described above and may be a program realizing the functions described above by being combined with a program recorded in the computer system in advance, a so-called a differential program. The program described above, for example, is read and executed by a processor such as a central processing unit (CPU) included in a computer.
While a preferred embodiment of the present invention has been described in detail with reference to the drawings, a specific configuration is not limited to such an embodiment, and various modifications and substitutions can be made within a range not departing from the concept of the present invention. The components described in the embodiments described above may be combined.
1 product-sum operation device
111, 121, 211, 221, 311, 321, k11, k21 product operation unit
10D, 20D current detecting unit
10S, 20S sum operation unit
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/JP2019/000317 | 1/9/2019 | WO | 00 |
