The disclosure relates to a calculation apparatus. More particularly, the disclosure relates to a sum-of-products calculation apparatus.
With the development of semiconductor technologies, various types of semiconductor apparatuses have been constantly introduced, some of which are capable of performing calculations, e.g., a sum-of-products calculation. The sum-of-products calculation is considerably useful in the field of artificial intelligence (AI).
In an analog neural network, it is often necessary to perform nonlinear conversion on the output of nodes in the previous layer, so that the analog neural network may deal with nonlinear issues. As to the implementation of circuits, the analog neural network firstly performs an analog-to-digital (A-to-D) conversion and then performs a calculation on an activation function. However, the complex activation function often requires accuracy of an A-to-D convertor, which leads to an increase in manufacturing costs or poor performance of the analog neural network.
The disclosure provides a sum-of-products calculation apparatus capable of reducing requirements for accuracy of an A-to-D converter and effectively enhancing performance of an analog neural network.
In an embodiment of the disclosure, a sum-of-products calculation apparatus including an A-to-D conversion circuit that has an encoder circuit and a plurality of inverters is provided. Threshold voltages of the inverters are set according to classification threshold values of an activation function. The inverters generate a plurality of bit signals in response to an analog sum-of-products signal. The encoder circuit encodes the bit signals to generate a digital signal.
In view of the above, the sum-of-products calculation apparatus provided in one or more embodiments of the disclosure includes the A-to-D conversion circuit having the encoder circuit and the inverters, the threshold voltages of the inverters are set according to the classification threshold values of the activation function, the inverters generate the bit signals in response to the analog sum-of-products signal, and the encoder circuit encodes the bit signals to generate the digital signal. According to the classification threshold values of the activation function, the threshold voltages of the inverters may be set, so as to complete the calculation of the activation function during the A-to-D conversion, whereby the requirements for accuracy of the A-to-D conversion circuit may be reduced, and the performance of the analog neural network may be effectively enhanced.
To make the aforementioned more comprehensible, several embodiments accompanied with drawings are described in detail as follows.
The accompanying drawings are included to provide a further understanding of the disclosure and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the disclosure and, together with the description, serve to explain the principles of the disclosure.
Reference will now be made in detail to the exemplary embodiments of the disclosure, examples of which are illustrated in the accompanying drawings. Whenever possible, the same reference numerals are used to represent the same drawings or similar parts in the accompanying and description, and the description of the same technical content is omitted. The description of the omitted part may be derived from the previous embodiment and will not be repeated in the following embodiments.
Please refer to
To be specific, the A-to-D conversion circuit 104 may include a plurality of inverters InV1-InV15 and an encoder circuit 106, and input terminals and output terminals of the inverters InV1-InV15 are respectively coupled to the sum-of-products calculation circuit 102 and the encoder circuit 106. Each of the inverters InV1-InV15 has a different threshold voltage which corresponds to the quantitative conversion voltages of each level of an A-to-D converter, so as to generate corresponding A-to-D output bit signals in response to the analog sum-of-products signal SMA1. For instance, in this embodiment, the threshold voltage of the inverter InV1 may serve to generate the lowest order bit signal, and the threshold voltage of the inverter InV15 may serve to generate the highest order bit signal. The bit signals generated by the inverters InV1-InV15 may, for instance, constitute a thermometer code (which should however not be construed as a limitation in the disclosure) to represent the value of the analog sum-of-products signal SMA1. In this embodiment, the threshold voltages of the inverters InV1-InV15 may be set according to classification threshold values of an activation function, so as to complete the calculation of the activation function during the A-to-D conversion, whereby the requirements for accuracy of the A-to-D converter may be reduced, and performance of an analog neural network may be effectively enhanced.
For instance,
Particularly, each of the inverters InV1-InV15 may be implemented in the manner shown in
In addition, the encoder circuit 106 may encode the bit signals generated by the inverters InV1-InV15 to generate the digital signal SB1. For instance, the encoder circuit 106 may encode the thermometer code constituted by the bit signals generated by the inverters InV1-InV15 into a binary signal (a 4-bit binary signal in this embodiment, which should however not be construed as a limitation in the disclosure), and the binary signal may be output as the digital signal SB1. In some embodiments, the encoder circuit 106 may be implemented, for instance, in form of a logic circuit, which should however not be construed as a limitation in the disclosure. The encoder circuit 106 may also encode the bit signals generated by the inverters InV1-InV15 into the digital signal SB1 by referring to a look-up table (e.g., a look-up table where the thermometer code is converted to a binary code).
As mentioned above, in addition to the advantages of reducing the requirements for accuracy of the A-to-D converter and enhancing the performance of the analog neural network, the sum-of-products calculation apparatus has the inverters InV1-InV15 with different threshold voltages and the encoder circuit 106 to quickly convert the analog sum-of-products signal SMA1 to the digital signal SB1; besides, it is not necessary to provide additional current or voltage, and there is no quiescent bias current but transient current. Moreover, the transient time is extremely short, thus ensuring low power consumption and high conversion efficiency. Additionally, the circuit configuration of the inverters InV1-InV15 and the encoder circuit 106 has the advantage of occupying a relatively small circuit area.
Note that the A-to-D conversion circuit 104 described in the embodiment above includes 15 inverters InV1-nV15. However, the number of inverters is not limited to what is provided in the previous embodiment, and in other embodiments, the A-to-D conversion circuit 104 may include more or fewer inverters.
In this embodiment, the current sources IA1-IA4 may be implemented in form of transistors, for instance, which should however not be construed as a limitation in the disclosure. The current sources IA1-IA4 may be controlled by a control voltage VCON and provide different currents, respectively; for instance, a ratio of current values of the currents provided by the current sources IA1-IA4 may be a geometric progression, e.g., the current values of the currents provided by the current sources IA1-IA4 may be 0.1 uA, 0.2 uA, 0.4 uA, and 0.8 uA in sequence, which should however not be construed as a limitation in the disclosure. The switches SWA1-SWA4 may be controlled by the analog input signals SA1-SA4 to change the state of the switches SWA1-SWA4, and the switches in an on state may provide the currents from the corresponding current sources to the current mirror circuit 502. For instance, in this embodiment, it is assumed that the switches SWA1-SWA3 are in an on state, and the switch SWA4 is in an off state; as such, the switches SWA1-SWA3 may respectively provide the currents with the current values of 0.1 uA, 0.2 uA, and 0.4 uA. That is, the current value of the current I received by the current mirror circuit 502 is 0.7 uA.
The current mirror circuit 502 may output a plurality of currents from the output terminals O1-O4 according to the currents provided by the switches SWA1-SWA3 in the on state, and the ratio of the current values of these currents may be a geometric progression. For instance, in this embodiment, the output terminals O1-O4 may output the currents with the current values of I/15, 2I/15, 4I/15, and 8I/15, respectively, which should however not be construed as a limitation in the disclosure. The switches SWB1-SWB4 may be controlled by the weight signals SC1-SC4 to change their states, and the switches in the on state may provide the currents from the corresponding output terminals to the negative input terminal of the comparator A1. For instance, in this embodiment, it is assumed that the switches SWB1 and SWB3 are in the on state, while the switches SWB2 and SWB4 are in the off state; as such, the switches SWB1 and SWB3 may respectively provide the currents with the current values of I/15 and 4I/15. That is, the current value of the current ISM received by the negative input terminal of the comparator A1 is 5I/15. After the current ISM passes through the comparator A1 and the feedback resistor RFB, the voltage output by the comparator A1 may represent the sum of products of the analog input signals SA1-SA4 and the weight signals SC1-SC4 (the analog sum-of-products signal SMA1). Note that the sum-of-products calculation circuit 102 described in this embodiment includes 4 current sources IA1-IA4, 4 switches SWA1-SWA4, and 4 switches SWB1-SWB4, while the number of the switches and the number of the current sources are not limited to what is described in this embodiment, and the relationship of the current values of the currents provided by the current sources IA1-IA4 and the relationship of the current values of the currents provided by the output terminals O1-O4 of the current mirror circuit 502 are also not limited to what is described in this embodiment.
In addition, in some embodiments, the sum-of-products calculation apparatus may further include a control voltage generating circuit as shown in
A threshold voltage of the input inverter TinV1 may be set to be greater than the threshold voltage of any of the inverters InV1-InV15, and a threshold voltage of the input inverter TinV2 may be set to be less than the threshold voltage of any of the inverters InV1-InV15. For instance, it is assumed that the threshold voltages of the inverters InV1-InV15 decrease sequentially; that is, the inverter InV1 has the maximum threshold voltage, and the inverter InV15 has the minimum threshold voltage, then the threshold voltage of the input inverter TinV1 may be set to be greater than the threshold voltage of the inverter InV1, and the threshold voltage of the input inverter TinV2 may be set to be less than the threshold voltage of the inverter InV15. For instance, the difference between the threshold voltage of the input inverter TinV1 and the threshold voltage of the inverter InV1 may be equal to the difference between the threshold voltage of the inverter InV1 and the threshold voltage of the inverter InV2, and the difference between the threshold voltage of the inverter InV15 and the threshold voltage of the input inverter TinV2 may be equal to the difference between the threshold voltage of the inverter InV14 and the threshold voltage of the inverter InV15, which should however not be construed as a limitation in the disclosure. By connecting the input terminals and the output terminals of the input inverters TinV1 and TinV2, input voltages VH and VL provided by the input inverters TinV1 and TinV2 may be equal to the threshold voltages of the input inverters TinV1 and TinV2. To be specific, for instance, the input inverters TinV1 and TinV2 may be implemented by the circuit configuration shown in
The input inverters TinV1 and TinV2 may respectively provide the input voltages VH and VL to the negative input terminal of the comparator A2 and the positive input terminal of the comparator A3, so as to generate a current Iu on a conductive path of the transistor M3. The voltage generating circuit 604 may, according to the current Iu, generate a corresponding control voltage VCON to the control terminals of the current sources IA1-IA4 depicted in
The input voltages VH and VL provided by the input inverters TinV1 and TinV2 may be dynamically changed together with circuit variables, such as temperature, process variations, circuit aging degree, and so on. Therefore, if the input inverters TinV1 and TinV2 are applied to respectively provide the input voltages VH and VL to the negative input terminal of the comparator A2 and the positive input terminal of the comparator A3 and provide the input voltage VL to the positive input terminal of the comparator A1, when other devices in the sum-of-products calculation apparatus encounter changes to the voltage operating range due to said circuit variables, the changes to the voltage operating range caused by the circuit variables may be automatically compensated. In addition, the circuit configuration of the input inverters TinV1 and TinV2 does not require any additional reference voltage and additional reference current, and the input inverters TinV1 and TinV2 have the advantages of fast response speed, favorable monotonicity, and favorable nonlinearity.
To sum up, the sum-of-products calculation apparatus provided in one or more embodiments of the disclosure includes the A-to-D conversion circuit having the encoder circuit and the inverters, the threshold voltages of the inverters may be set according to the classification threshold values of the activation function, the inverters may generate the bit signals in response to the analog sum-of-products signal, and the encoder circuit may encode the bit signals provided by the inverters to generate the digital signal. As such, based on the classification threshold values of the activation function, the threshold voltages of the inverters may be set, so as to complete the calculation of the activation function during the A-to-D conversion, whereby the requirements for the accuracy of the A-to-D conversion circuit may be reduced, and the performance of the analog neural network may be effectively enhanced.
It will be apparent to those skilled in the art that various modifications and variations can be made to the structure of the disclosed embodiments without departing from the scope or spirit of the disclosure. In view of the foregoing, it is intended that the disclosure cover modifications and variations of this disclosure provided they fall within the scope of the following claims and their equivalents.
Number | Date | Country | Kind |
---|---|---|---|
202110970487.6 | Aug 2021 | CN | national |
202210032449.0 | Jan 2022 | CN | national |
This application claims the priority benefit of U.S. provisional application Ser. No. 63/162,502, filed on Mar. 17, 2021, the priority benefit of China application serial no. 202110970487.6, filed on Aug. 23, 2021, the priority benefit of U.S. provisional application Ser. No. 63/278,468, filed on Nov. 11, 2021, and the priority benefit of China application serial no. 202210032449.0, filed on Jan. 12, 2022. The entirety of each of the above-mentioned patent applications is hereby incorporated by reference herein and made a part of this specification.
Number | Date | Country | |
---|---|---|---|
63162502 | Mar 2021 | US | |
63278468 | Nov 2021 | US |