Patent Application
20240005977
The present application is based upon and claims the benefit of priorities of Chinese Patent Application No. 202210769401.8, filed on Jun. 30, 2022; Chinese Patent Application No. 202211599037.1, filed on Dec. 12, 2022; Chinese Patent Application No. 202211728423.6, filed on Dec. 30, 2022; and Chinese Patent Application No. 202310259263.3, filed on Mar. 9, 2023, the entire contents of all of which are incorporated herein by reference.
The present disclosure relates to a field of storage technology and integrated circuit technology, and particularly to a compute-in-memory devices, neural network accelerators, and electronic devices.
With the wide application of technologies such as artificial intelligence, the demand for data processing is increasing rapidly, but the computing performance and power consumption of modern computing systems based on the von Neumann architecture are limited by the movement of data between the storage part and the computing part. Compute-in-memory is an attempt to solve this problem, which attempts to reduce energy consumption and improve performance by reducing data movement activities. Conventional storage cells are optimized for storage without considering the cost of computing. Therefore, a compute-in-memory system that can be applied on a large scale is to combine the computing part with the storage part, and has a convenient interface, with consideration of cost, power consumption, performance, scalability, and stability.
Because of the advantages of Static Random-Access Memory (SRAM) such as fast read/write speed, low read/write energy consumption, good durability, and available mature technology, many compute-in-memory designs based on SRAM have emerged in recent years. Some existing designs use foundry-provided SRAM cells comprising 6 transistors (6T). Due to the more compact layout and smaller transistors used by foundries, this SRAM cell has a smaller cell area than non-6T SRAM cells. However, when 6T SRAM cells are used for computing, the swing range of the bit line voltage is limited, which leads to problems such as read disturbance and too small signal margin. Therefore, some designs choose to sacrifice a certain area efficiency and use SRAM composing more transistors as the computing unit.
The area of compute cells has a large impact on the overall cost of a chip and is thus a very imterminalant issue in edge artificial intelligence devices. In actual industrial applications, area efficiency is one of the core indicators of compute-in-memory design. Even with the adoption of foundry-provided 6T SRAM as the compute cell, compute-in-memory designs still face the problem of inadequate area efficiency. For large neural networks, SRAM-based compute-in-memory designs face difficulties storing all weights on the chip, thus needing to repeatedly read weight values from dynamic random access memory (DRAM), causing huge energy consumption and delay. Therefore, it is necessary to further improve the area efficiency of SRAM-based compute-in-memory.
As shown in
To address the aforementioned issues, compute-in-memory is a viable solution. As shown in
Currently, the compute-in-memory research conducted by academia and industry mainly relies on devices such as Static Random-Access Memory (SRAM) and emerging non-volatile memories (e-NVM) represented by Resistor Random-Access Memory. SRAM has advantages of fast read/write speed, low read/write power consumption, good durability, and mature manufacturing processes. However, the significant drawback of using SRAM units for compute-in-memory is high static power consumption and low area efficiency. In addition, there is a trade-off between noise margin and storage density in SRAM units, and read interference problems limit the voltage swing of bitlines. To ensure system reliability, many design schemes have abandoned the compact 6T SRAM structure and introduced more transistors as shown in
According to the specific implementation method, compute-in-memory can be divided into two categories: digital compute-in-memory and analog compute-in-memory. The advantage of digital compute-in-memory lies in its ability to maintain accuracy on par with traditional computing platforms, while its improvement in computational efficiency is relatively limited. The advantage of analog compute-in-memory lies in its higher energy efficiency, but it has drawbacks such as certain accuracy loss and additional data conversion overhead. Analog compute-in-memory can be further divided into charge-domain computing and current-domain computing. The former is based on the principle of charge conservation and performs weighted accumulation operations based on the capacitance values. It has the advantage of high matching accuracy but introduces delay costs and area overhead due to capacitors. The latter is based on Kirchhoff s current law and performs weighted accumulation operations. It has the advantage of fast operation speed but relatively lower accuracy.
In the design of compute-in-memory circuits, the area efficiency of the computational unit is a key metric. On one hand, the area of the computational unit directly determines the manufacturing cost and commercial potential of the chip. On the other hand, due to the large number of parameters trained by mainstream neural network algorithms, compute-in-memory chips require frequent data interaction with Dynamic Random-Access Memory (DRAM) during the execution of AI algorithms, introducing additional delays and energy consumption. Therefore, improving the area efficiency of compute-in-memory can reduce the number of DRAM accesses, contributing to the overall performance improvement of the system.
Content-addressable memory (CAM) is a type of memory with search functionality. Its characteristic is that when performing a search operation, input data is used as the search content, the contents stored in the memory are matched with the input data, and the address of the matching data in the memory is outputted as the search result.
The characteristic of content-based search in CAM enables it to support the needs of some key applications. In computer networks, a routing table is a collection of data stored in a router or networked computer. The routing table stores paths pointing to specific network addresses, and the search for the routing table is a content-based search method as described earlier. Traditional search methods such as linear search and hash table search are software search methods based on random access memory (RAM), which are slow and cannot meet the search requirements of high-speed real-time communication systems. However, the hardware search method based on CAM completes the search within one clock cycle by querying all data in the memory in the same clock cycle, making the search speed unaffected by the amount of routing table data. The average search speed is much faster than that of the RAM-based search method, which significantly improves network transmission speed and network performance.
Currently, the common CAM in academia is mainly based on storage units composed of Static Random-Access Memory (SRAM) cells, Resistor Random-Access Memory (ReRAM) cells, Ferroelectric gate field-effect transistors (FeFET), and so on. These storage units are usually composed of multiple devices, with a large area and high power consumption. Since the area determines the manufacturing cost of the CAM chip, and power consumption affects the application value of the CAM chip, it is necessary to reduce the complexity of CAM storage units, thereby improving the area efficiency and reducing the power consumption of CAM.
To improve overall system throughput and energy efficiency, a compute-in-memory architecture has been proposed to eliminate the “memory wall” bottleneck in von Neumann architectures. Compute-in-memory integrates the memory and processing units, enabling computations to be performed within the memory units themselves, reducing the additional consumption associated with data movement. This makes compute-in-memory architectures highly promising for deploying machine learning models on edge devices.
Compute-in-memory shares similar computing principles among different types of memory devices, such as Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Flash memory, Resistive Random-Access Memory (ReRAM), and other emerging memory technologies. However, due to its mature manufacturing process, durability, flexibility in read/write operations, and fast access speed, SRAM is currently the mainstream choice for compute-in-memory architectures. However, using 6T SRAM for compute-in-memory faces challenges like read disturbance due to the limited voltage swing range of the bit-lines during 6T SRAM cell computation. Therefore, to improve the performance of SRAM-based compute-in-memory, some circuits utilize SRAM configurations with more transistors, such as 8T SRAM, 10T SRAM, and 12T SRAM.
To enable the widespread adoption of AI applications on edge devices, the area efficiency of the storage units in compute-in-memory devices is a crucial consideration during design. However, due to the low storage density characteristics of SRAM, even with the smallest 6T SRAM cells employed on the chip, large-scale neural networks still face challenges in complete deployment on a chip. This leads to the need for frequent reading of weight values from off-chip memory during computations, resulting in significant data transfer energy consumption and latency. Therefore, improving the area efficiency of compute-in-memory devices becomes a core optimization objective.
According to a first aspect of the present disclosure, a compute-in-memory apparatus is provided, and the apparatus comprises:
In an embodiment of the present disclosure, the storage cell comprises a first storage switch and a second storage switch, wherein the storage detection terminals of both the first storage switch and the second storage switch are connected to the output terminal of the storage cell.
In an embodiment of the present disclosure, the computing module additionally comprises a selection switch. The selection switch comprises a selection control terminal, a first detection terminal, and a second detection terminal, wherein,
In an embodiment of the present disclosure, the storage cell comprises a first selection switch, a second selection switch, a third storage switch, a fourth storage switch, a fifth storage switch, and a sixth storage switch,
In an embodiment of the present disclosure, the compute operation includes a multiply-and-accumulate operation. The control module is additionally used to:
In an embodiment of the present disclosure, the compute operation includes a logic AND operation. The control module is additionally used to:
In an embodiment of the present disclosure, within each column of the computing array, the control bit lines and computing bit-lines of at least one computing module are connected; within each row of the computing array, the control word-lines, data storage lines and reset state voltage lines of at least one computing module are connected.
In an embodiment of the present disclosure, the compute operation includes a multiply-and-accumulate operation. The control module is additionally used to:
In an embodiment of the present disclosure, the control module is additionally used to:
In an embodiment of the present disclosure, the control module is additionally used to:
According to a second aspect of the present disclosure, a neural network accelerator is provided. The neural network accelerator comprises at least one neural network module, the neural network module comprises at least one original convolutional layer, and the original convolutional layer comprises a backbone layer that has fixed weights and a branch layer that has adjustable weights. The backbone layer comprises one or more convolutional layers, and the branch layer at least comprises a first branch convolutional layer, a second branch convolutional layer, and a third branch convolutional layer, which are sequentially connected. The input channel number of the first branch convolutional layer is equal to that of the backbone layer, the output channel number of the third branch convolutional layer is equal to that of the backbone layer, the input channel number of the second branch convolution layer is smaller than that of the backbone layer, and the output channel number of the second branch convolution layer is smaller than that of the backbone layer,
In an embodiment of the present disclosure, the backbone layer and the first branch layer are used to receive an input to the neural network, and an output of the neural network module is obtained by aggregating the output of the backbone layer and the output of the third branch convolution layer.
In an embodiment of the present disclosure, in a training process of the neural network accelerator, the weights of each backbone layer are fixed, and/or the weight gradient of the backbone layer is zero in a back-propagation stage of the training process, and the weights of each branch layer are adjusted by gradient descent.
According to a third aspect of the present disclosure, an electronic device is provided, which includes the compute-in-memory apparatus, or includes the neural network accelerator.
The compute-in-memory apparatus of embodiments of the present disclosure comprises a computing array and a control module. The computing array comprises a plurality of computing modules, wherein each computing module comprises at least one storage cell, a reset switch Qf and a capacitor. The storage cell comprises at least one storage switch, wherein: the storage switch comprises a storage control terminal, a storage detection terminal, and a storage terminal, the storage terminal is connected to a data storage line to receive a storage state voltage and the information associated with the storage state voltage; the storage control terminal is connected to a control word-line to receive a control voltage to adjust the impedance characteristic between the storage detection terminal and the storage terminal. The reset switch Qf comprises a reset control terminal, a reset detection terminal, and a reset terminal, the reset control terminal is connected to a control word-line to receive a reset voltage to adjust the impedance characteristic between the reset detection terminal and the reset terminal, and the reset terminal is connected to a reset state voltage line to receive a reset state voltage. The reset detection terminal and a first terminal of the capacitor are connected to the output terminals of at least one storage cell, and a second terminal of the capacitor is connected to the compute word-line. The control module controls the computing array to perform at least one of a store operation, a read operation, and a compute operation. Embodiments of the present disclosure include at least one storage switch, while the storage cells of the related art include at least six transistors, therefore, embodiments of the present disclosure have higher area efficiency. In addition, embodiments of the present disclosure store data through storage state voltages, and a single storage switch can realize multi-bit storage, which further improves area efficiency and significantly reduces the power consumption of data access and compute-in-memory.
According to a fourth aspect of the present disclosure, a compute-in-memory apparatus is provided, the apparatus comprises: a plurality of read-only memory devices, a plurality of current sources, and a control module, wherein:
In an embodiment of the present disclosure, the apparatus comprises a plurality of compute modules, wherein each compute module comprises a read-only memory device; the output terminal of the read-only memory device is directly connected to the corresponding compute bit line to output the result data.
In an embodiment of the present disclosure, the apparatus comprises a plurality of compute modules, wherein each compute module comprises two or more read-only memory devices and at least one selection device; the control terminals of the selection devices are connected to the corresponding control bit lines, the input terminals of the selection devices are connected to the output terminals of the read-only memory devices, and the output terminals of the selection devices are connected to the corresponding compute bit lines to output the result data.
In an embodiment of the present disclosure, the read-only memory devices in each compute module are arranged in a multi-row and multi-column layout, the control terminals of each row of read-only memory devices are connected to the same control word line, the output terminals of each column of read-only memory devices are connected to the same selection devices, and the output terminals of each selection device are connected to the same compute bit line.
In an embodiment of the present disclosure, a plurality of compute modules are combined into a multi-row and multi-column layout through electrical connections, wherein the control word lines of the compute modules in the same row are electrically connected, the control bit lines of the compute modules in the same column are electrically connected, the compute bit lines of the compute modules in the same column are electrically connected, and the compute bit lines are connected to the power supply voltage through selection devices.
In an embodiment of the present disclosure, the operation further includes a multiply-and-accumulate operation, and the result data includes the multiply-and-accumulate result; the control module is further used to:
In an embodiment of the present disclosure, the control module is further used to adjust the signal timing of the control word lines and control bit lines to achieve pipelining operations between different compute bit lines.
In an embodiment of the present disclosure, the control module is further used to control the compute modules to enter a working mode or an idle mode, wherein
According to a fifth aspect of the present disclosure, a neural network accelerator is provided, wherein the neural network accelerator comprises the compute-in-memory apparatus.
According to a sixth aspect of the present disclosure, an electronic device is provided, wherein the electronic device comprises a neural network accelerator.
According to a seventh aspect of the present disclosure, a content-addressable storage device is provided, including:
The control module, connected to various storage units, is used to control the voltage of the first and second input terminals of the read-only memory devices in each storage unit, in order to perform the required operation.
The operation result of the target operation is determined based on the voltage of the second terminal of the capacitor as described.
In one possible implementation, the required operation includes a XNOR logic operation, and the first and second input terminals of each memory device are voltage-controlled to perform the required operation, including:
The required operation's result is then determined based on the voltage of the second terminal of the capacitor, obtaining the XNOR logic operation result between the input data and the stored data.
In one possible implementation, the control module is connected to the first input terminal of the read-only memory device via the first bitline, and to the second input terminal of the read-only memory device via the second bitline. The control module is also connected to the second terminal of the capacitor through the matching line. Multiple storage units are combined into a layout of multiple rows and columns through electrical connections, where the electrical connection method is such that the matching lines of some or all storage units in the same row are connected, and the first and second bitlines of some or all storage units in the same column are respectively connected.
In one possible implementation, the required operation mentioned includes content-addressable operation, which involves controlling the voltage of the first and second input terminals of the read-only memory device for each storage unit to perform the required operation, including:
In one possible implementation, the required operation includes a content-addressable operation, which involves controlling the voltage of the first and second input terminals of the read-only memory device for each storage unit to perform the required operation, including:
In one possible implementation, the required operation also includes a read operation to retrieve the data stored in the storage unit. The control module is also used to: maintain complementary voltage levels at the first and second input terminals, distinguish whether the output terminal is connected to the first or second input terminal based on the magnitude of the output voltage, and thereby obtain the stored data.
In one possible implementation, the control module is also used to control at least one storage unit to operate in a working mode or an idle mode, wherein the storage unit in the working mode performs the required operation, or the first input terminal, the second input terminal, and the output terminal of the read-only memory device of the storage unit in the idle mode are set to low level.
According to an eighth aspect of this disclosure, a memory is provided, which includes a content-addressable storage device.
According to a ninth aspect of this disclosure, an electronic device is provided, which includes the memory as described above.
Various aspects of the implementations disclosed herein implement content-addressable storage devices based on read-only memory devices, which can improve the area efficiency of the content-addressable storage device, reduce or even eliminate unnecessary energy consumption caused by memory access, and thus achieve the goal of reducing power consumption.
According to a tenth aspect of the present disclosure, a high-density CiM device based on read-only memory (ROM) devices is provided, characterized by the following components:
In an embodiment of the present disclosure, the device comprises selection devices, specifically data selection devices, the data selection devices consist of upper data selection devices and lower data selection devices, the upper data selection devices include a first upper data selection device and a second upper data selection device, the lower data selection devices include a first lower data selection device and a second lower data selection device, the device also includes data selection control lines, namely the first data selection control line and the second data selection control line, additionally, it comprises data storage lines, which are the first data storage lines and the second data storage lines;
In an embodiment of the present disclosure, the device is characterized by the inclusion of column selection devices as part of the selection devices, wherein:
In an embodiment of the present disclosure, the device is characterized by having multiple current sources or multiple voltage sources as the electrical source, if the electrical source is multiple voltage sources, the computing module further includes a reset switch and a capacitor, the control terminal of the reset switch is connected to a reset control word-line to receive a reset signal, the reset terminal of the reset switch is used to receive a reset state voltage, the reset detection terminal of the reset switch and the first terminal of the capacitor are connected to the output terminal of the upper data selection device, the second terminal of the capacitor is connected to the computing bit-line.
In an embodiment of the present disclosure, the device is characterized by having each ROM devices in the computing module in a layout of multiple rows and columns, wherein:
In an embodiment of the present disclosure, the device is characterized by the feature that the first data storage lines connected to each individual ROM device in the (K+1)th column is the second data storage lines connected to each individual ROM device in the Kth column, where K is an integer.
In an embodiment of the present disclosure, the device is characterized by the following:
In an embodiment of the present disclosure, the device is characterized by having each ROM devices in the computing module in a layout of multiple rows and columns, wherein:
In an embodiment of the present disclosure, the device is characterized by the feature that the first data storage lines connected to each individual ROM device in the (K+1)th column is the second data storage lines connected to each individual ROM device in the Kth column, where K is an integer;
In an embodiment of the present disclosure, the device is characterized by the following:
In an embodiment of the present disclosure, the device is characterized by the following:
In an embodiment of the present disclosure, the device is characterized by the following:
In an embodiment of the present disclosure, the device is characterized by the following:
According to an eleventh aspect of the present disclosure, a neural network accelerator, characterized by comprising a high-density CiM based on ROM, is provided.
According to a twelfth aspect of the present disclosure, an electronic device, characterized by comprising a high-density CiM device based on ROM, is provided.
The disclosed embodiment of the high-density CiM device based on ROM implements multi-level cell storage for a single ROM, enabling each computing module to have multi-bit data storage and computation capabilities. This improves the storage density of the device and enhances the area efficiency of CiM, thereby reducing or even eliminating the device's need for off-chip memory access.
It should be understood that the general description above and the detailed description below are exemplary and explanatory only, and are not intended to limit the disclosure. The other features and aspects of the disclosure will become apparent from the following detailed description of exemplary implementations with reference to the accompanying drawings.
The accompanying drawings here are incorporated into the description and constitute a part of the present description. These drawings show embodiments consistent with the present disclosure, and are used together with the description to explain the technical solution of the present disclosure.
Various exemplary embodiments, features, and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference signs in the drawings represent the same or similar elements with the same or similar functions. While various aspects of the embodiments are illustrated in drawings, the drawings are not necessarily drawn to scale unless specifically indicated.
In the description of the present disclosure, it shall be understood that the orientations or positional relationships indicated by terms such as “length”, “width”, “upper”, “lower”, “front”, “rear”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inner”, “outer”, etc. are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present disclosure and simplifying the description, but not to indicate or imply that an apparatus or element referred to must have a particular orientation or must be constructed or operated in a particular orientation, and thus shall not be understood as a limitation of the present disclosure.
In addition, the terms “first” and “second” are used for descriptive purposes only, and shall not be understood as indicating or implying relative significance or implicitly specifying the quantity of indicated technical features. Thus, a feature defined as “first” or “second” may explicitly or implicitly include one or more of these features. In the description of the present disclosure, “plurality” means two or more, unless otherwise clearly and specifically defined.
In the present disclosure, unless otherwise clearly and specifically defined and limited, terms such as “install”, “connect”, “connection” and “fix” shall be understood in a broad sense, for example, the indicated relationship can be a fixed connection, a detachable connection, or integrated; it can be mechanically connected or electrically connected; it can be directly connected or indirectly connected via a medium, and it can be an internal connection of two components or an interaction relationship between two components. A person skilled in the art can understand the specific meanings of the above terms in the present disclosure according to specific situations.
The word “exemplary” is used exclusively herein to mean “serving as an example, embodiment, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be understood as superior or better than other embodiments.
The term “and/or” herein is just an association relationship describing associated objects, which means that there can be three relationships, for example, A and/or B can mean the following three situations: A exists alone, A and B exist simultaneously, or B exists alone. In addition, the term “at least one of” herein means any one of a variety or any combination of two or more of a variety, for example, including at least one of A, B, and C can mean including any one or more elements selected from the set formed by A, B and C.
In addition, in order to better describe the present disclosure, numerous details are given in the following specific embodiments. It will be understood by a person skilled in the art that the present disclosure may be practiced without some of the specific details. In some instances, methods, means, components, and circuits that are well known to a person skilled in the art have not been described in detail in order to highlight the subject of the present disclosure.
Storage cells in the related art of compute-in-memory include at least 6 transistors, thus having relatively low area efficiency and high power consumption.
The compute-in-memory apparatus of embodiments of the present disclosure comprises a computing array and a control module. The computing array comprises a plurality of computing modules, wherein each computing module comprises at least one storage cell, a reset switch Qf and a capacitor. The storage cell comprises at least one storage switch, wherein: the storage switch comprises a storage control terminal, a storage detection terminal, and a storage terminal, the storage terminal is connected to a data storage line to receive a storage state voltage and the information associated with the storage state voltage; the storage control terminal is connected to a control word-line to receive a control voltage to adjust the impedance characteristic between the storage detection terminal and the storage terminal. The reset switch Qf comprises a reset control terminal, a reset detection terminal, and a reset terminal, the reset control terminal is connected to a control word-line to receive a reset voltage to adjust the impedance characteristic between the reset detection terminal and the reset terminal, and the reset terminal is connected to a reset state voltage line to receive a reset state voltage. The reset detection terminal and a first terminal of the capacitor are connected to the output terminals of at least one storage cell, and a second terminal of the capacitor is connected to the compute word-line. The control module controls the computing array to perform at least one of a store operation, a read operation, and a compute operation. Embodiments of the present disclosure include at least one storage switch, while the storage cells of the related art include at least six transistors, therefore, embodiments of the present disclosure have higher area efficiency. In addition, embodiments of the present disclosure store data through storage state voltages, and a single storage switch can realize multi-bit storage, which further improves area efficiency and significantly reduces the power consumption of data access and compute-in-memory.
Please refer to
As illustrated in
The above-mentioned k can be any positive integer.
Embodiments of the present disclosure do not limit the specific implementation manner of the storage cell, and a person skilled in the art may select an appropriate implementation manner according to the actual situation and needs. The possible implementation manners of the storage cell are exemplarily introduced below.
Please refer to
In an embodiment of the present disclosure, the storage cell may include at least one storage switch, as illustrated in
Please refer to
In an embodiment of the present disclosure, as illustrated in
In an embodiment of the present disclosure, the computing module 110 may additionally comprise a selection switch, which comprises a selection control terminal, a first detection terminal and a second detection terminal, wherein,
Please refer to
In an embodiment of the present disclosure, as illustrated in
Possible implementations of the storage cell are described above exemplarily, but they shall not be understood as a limitation to embodiments of the present disclosure. The number of storage switches in the storage cell of embodiments of the present disclosure can be arbitrary, and the number of storage cells in a computing module can also be arbitrary.
Please refer to
In an embodiment of the present disclosure, as illustrated in
In an example, the compute-in-memory apparatus as illustrated in
In an example, as illustrated in
Please refer to
The capacitor CM of embodiments of the present disclosure may be implemented with the metal-insulator-metal capacitor (MIM) as illustrated in
Please refer to
The storage switches, selection switches, and reset switches of embodiments of the present disclosure may be metal-oxide-semiconductor field effect transistors (MOSFET) (the circuit symbol is shown in
In embodiments of the present disclosure, devices such as metal-oxide-semiconductor field effect transistors may be used as read-only storage devices, and the electrical connections of the devices is used to store information without write operations, and the in-memory computation is performed in the charge domain. Metal-oxide-semiconductor field effect transistor process is currently the most common integrated circuit process, and the process is widely used and highly mature.
In embodiments of the present disclosure, the adoption of the metal-oxide-semiconductor field effect transistors has the advantage of convenient operation. Taking the N-type metal-oxide-semiconductor field effect transistor as an example, a single metal-oxide-semiconductor field effect transistor can change the impedance characteristic between the source and drain of the metal-oxide-semiconductor field effect transistor by controlling the difference between the gate voltage and the source voltage. When the voltage difference is in a certain range, the metal-oxide-semiconductor field effect transistor exhibits a low resistance between the source and drain, and the circuit between the source and drain is turned on. The drains are connected to different storage state voltages, taking only two storage state voltages for example, connecting to a high voltage is equivalent to storing “1”, and connecting to a low voltage is equivalent to storing “0”. In practical applications, connecting to a low voltage may also represent “1” and connecting to a high voltage may also represent “0”. In addition, the drains may be connected to multiple storage state voltages, achieving multi-bit storage with a single transistor. When performing read operations on a single metal-oxide-semiconductor field effect transistor, the gate-source voltage difference is kept within a certain range and the source is pulled down to a low voltage, and the voltage of the data storage line to which the drain is connected can be distinguished through the magnitude of the current between drain and source, thus the stored information is obtained.
Please refer to introductions of the related art for detailed descriptions of the metal-oxide-semiconductor field effect transistor MOSFET, which are not repeated here.
Taking MOSFET as an example for the storage switch, the connection relationships of each device in the computing module as illustrated in
Refer to
In an example, as illustrated in
In an embodiment of the present disclosure, the compute operations include a logic AND operation, and the control module 20 is additionally used to:
In embodiments of the present disclosure, a voltage difference is maintained between the two terminals of the capacitor of the target computing module by activating the reset control terminal of the reset switch of the target computing module and the computing bit-line, the computing bit-line is floating by turning off the reset control terminal, and a set of operands for the logic operation is inputted via the reset control terminal, the control word-lines connected to the storage control terminals of the storage switches and the data storage lines connected to the storage terminals of the storage switches; thereby a logic AND operation on the set of input operands is performed quickly and efficiently, and the result of the logic AND operation is obtained via the computing bit-lines.
The “activate” used herein can also be referred to as “enable”, for example, the activated terminal is brought to a high voltage, whereby the activated device is enabled.
The computing module illustrated in
Please refer to
In an example, as illustrated in the left diagram of
Then, as illustrated in the right diagram of
The computing module based on the read-only storage devices for compute-in-memory in the charge domain according to embodiments of the present disclosure has the advantages of high area efficiency and low power consumption, and can effectively mitigate the leakage problem in the idle state, and is therefore a class of compute-in-memory circuit that can greatly improve area efficiency and reduce energy consumption.
In an embodiment of the present disclosure, the compute operations include a multiply-and-accumulate operation, and the control module 20 is additionally used to:
In embodiments of the present disclosure, by activating the storage control terminal of the storage switch of the storage cell 1110 of the target computing module 110, a logic AND operation is performed on the information carried by the word-line connected to the storage control terminal of the activated storage switch and the information associated with the storage state voltage of the storage terminal of the activated storage switch, and the result of the multiply-and-accumulate operation is obtained via the computing bit-line. It can be seen that for the computing module that does not include a selection switch, the multiply-and-accumulate operation can be realized quickly and efficiently.
In an embodiment of the present disclosure, the compute operations include a multiply-and-accumulate operation, and the control module 20 is additionally used to: activate the storage control terminal of the storage switch of the storage cell 1110 of
In embodiments of the present disclosure, by activating the storage control terminal of the storage switch of the storage cell 1110 of the target computing module 110 and the selection control terminal of the selection switch connected to the storage switch, a logic AND operation is performed on the information carried by the control word-line connected to the storage control terminal of the activated storage switch and the information associated with the storage state voltage of the storage terminal of the activated storage switch, and the result of the multiply-and-accumulate operation is obtained via the computing bit-line. It can be seen that for the computing module that includes a selection switch, the multiply-and-accumulate operation can also be realized quickly and efficiently.
The multiply-and-accumulate operation based on the computing array is introduced below.
Please refer to
In an embodiment of the present disclosure, as illustrated in
It can be understood that the energy consumption of the computing array of the compute-in-memory apparatus of embodiments of the present disclosure mainly lies in charging the capacitors. Since the equivalent capacitance of embodiments of the present disclosure does not increase linearly with the increase of the ratio of “1”'s in the result of the logic AND operation, when the number of cells whose result of the logic AND operation are “1” exceeds a certain value, embodiments of the present disclosure can greatly reduce the energy consumption of computation.
Please refer to
As illustrated in
In an embodiment of the present disclosure, when performing a multiply-and-accumulate operation with the computing array circuit, one or more computing bit-lines, and/or computing modules connected to the same computing bit-line may be controlled simultaneously to perform logic AND operations, and the result of the multiply-and-accumulate operation is obtained by evaluating the effect of the process on the electrical characteristics of the computing bit-lines.
Please refer to
First, as illustrated in the left diagram (a) of
Then, as illustrated in the right diagram (b) of
In an embodiment of the present disclosure, the control module 20 is additionally used to:
In an example, the compute-in-memory apparatus of embodiments of the present disclosure supports an operating mode and an idle mode, wherein in the operating mode the write operation and compute operations (such as the logic AND operation and the multiply-and-accumulate operation, as described above) are performed, while in the idle mode, by controlling the voltages of the word-lines and the bit lines, the information stored by the circuit cells are nonvolatile.
Please refer to
In an example as illustrated in
In an example, in the operating mode, the voltages of each word-line and bit line can be configured for compute operations such as the multiply-and-accumulate operation and the logic AND operation, as described above.
In an embodiment of the present disclosure, the control module 20 is used additionally to:
Please refer to
In an example, using the compute-in-memory apparatus as illustrated in
Exemplarily, the pipelined operation may refer to passing the compute result of the former computing module to the current computing module via the computing bit-line CBL, and input it to the compute operation of the current computing module, and then pass the compute result of the current computing module to the next computing module via the computing bit-line CBL.
Of course, the embodiments of the present disclosure do not limit the specific timing of signals of the control word-lines and the control bit lines during the pipelined operation, and a person skilled in the art can set them according to actual conditions and needs.
Please refer to
As illustrated in
The backbone layer and the convolutional layers in the branch layer are implemented using the compute-in-memory apparatus.
The symbols N, M, D, and U represent integers.
In an example as illustrated in
In an example as illustrated in
In embodiments of the present disclosure, using the neural network accelerator, deep neural networks can be transformed into models suitable for fixed-weight neural network accelerators, with the backbone layer that has a huge number of parameters deployed in the high-density fixed-weight accelerator, and the branch layer with only a small amount of parameters deployed in the low-density but flexible-weight accelerator; thereby, when the task has changed, only the branch layer needs to be retrained, and the backbone layer stays unchanged.
Exemplarily, the branch convolutional layer may be 16 times smaller than the backbone layer. It can be seen that the embodiments of the present disclosure can significantly reduce the size of neural network accelerators.
The neural network accelerator according to embodiments of the present disclosure may be based on fixed-point numbers. In convolutional neural networks, the multiply-and-accumulate operation is the key operation and main source of energy consumption of the inference process. According to reports from existing research, neural networks with multi-bit weights and inputs can decompose multiplication operations into logic AND operations and addition operations. Fixed-point number neural networks have been shown to be successful on many datasets and have achieved very low energy consumption and marginal accuracy loss compared to float-point number neural networks.
Because of the advantages of high area efficiency and low energy consumption of the above-described compute-in-memory apparatus and the applicability thereof to various applications such as neural network acceleration, through the use of the compute-in-memory apparatus or the computing array or the storage cell as described above to implement the neural network accelerator, the embodiments of the present disclosure can to some extent solve the technical problems related to high-area efficiency compute-in-memory design and low-power compute-in-memory design.
In an embodiment of the present disclosure, the backbone layer and the first branch convolutional layer are used to receive an input to the neural network, and the output of the neural network module is obtained by aggregating the outputs of the backbone layer and the third branch convolutional layer.
In an example, the fixed-weight read-only memory computing array of embodiments of the present disclosure may be fine-tuned after fabrication; thereby, the neural network accelerator can adapt to the change of recognition targets after hardware deployment.
Please refer to
Exemplarily, as illustrated in
In an example, in embodiments of the present disclosure, all parameters of a large-scale network may be stored on the chip, the cache is used to temporarily store the intermediate values of the computation, and the controller is used to schedule the data flow and to execute non-neural network computations. With this method, only a small fraction of weights are deployed in the low-density variable-weight computing array, and most (more than percent, for example) of the parameters may be stored in the high-density fixed-weight computing array. Meanwhile, this method is able to fine-tune the network weights even after chip deployment, thus realizing the transfer of tasks.
The compute-in-memory apparatus of embodiments of the present disclosure has the advantages of high area efficiency and low energy consumption, and can effectively mitigate the leakage current problem faced with SRAM storage cells in the idle state, and is a class of compute-in-memory circuit that can greatly improve area efficiency and reduce energy consumption.
According to an aspect of the present disclosure, an electronic device is provided. The electronic device includes the compute-in-memory apparatus or the neural network accelerator.
The electronic device may be provided in the form of a terminal, a server, or a device of other forms.
Please refer to
For example, the electronic device 1900 may be provided as a server. Referring to
The electronic device 1900 may additionally comprise a power supply component 1926 which is configured to perform the power management of the electronic device 1900, a wired or wireless network interface 1950 which is configured to connect the electronic device 1900 to a network, and an input-output (I/O) interface 1958. The electronic device 1900 may operate based on the operating system stored in the memory 1932, such as Microsoft server operating system (Windows Server™), the graphical user interface-based operating system (Mac OS X™) introduced by Apple company, a multi-user and multi-process computer operating system (Unix™), a free and open-source Unix-like operating system (Linux™), an open-source Unix-like operating system (FreeBSD™) or alike.
In the exemplary embodiment of the present disclosure, a non-transitory computer-readable storage medium is provided, such as the memory 1932 which stores computer program instructions, and the computer program instructions may be executed by the processing component 1922 of the electronic device 1900 to accomplish the above method.
Please refer to
As shown in
The control terminal of each read-only memory device 2110 is connected to a corresponding control wordline WL<n> for receiving data to be computed. The output terminal of each read-only memory device 2110 is used to output result data through a computation bitline CBL<n>.
The control module 2100 selects the corresponding read-only memory device 2110 for operation through the control word line WL<n>, so as to output the result data. The operations include multiplication and data reading operations. For a multiplication operation, the result data is the product of the stored data corresponding to the preset voltage received by the current source 2120 and the data to be computed input through the control wordline WL<n>. For a data reading operation, the result data is the stored data corresponding to the preset voltage received by the current source 2120.
The present disclosure does not limit the specific implementation of the control module. For example, the control module may include a processing component, such as a stand-alone processor or a combination of discrete components. The processor can include a controller that executes instructions in any appropriate manner, such as by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components. Within the processor, hardware circuits such as logic gates, switches, ASICs, programmable logic controllers, and embedded microcontrollers can execute the executable instructions.
Read-only memory (ROM) is a type of storage device with high storage density, non-volatility, and low read power consumption. The present disclosure implements compute-in-memory based on read-only memory, making it possible to fully store large-scale neural network parameters on-chip, thereby eliminating the additional power consumption caused by repeated transfer of weights between on-chip and off-chip, greatly reducing the energy consumption of on-chip inference operations, and allowing deep learning algorithms to be deployed massively and efficiently on intelligent terminal devices.
The present disclosure utilizes read-only memory to implement a storage-based computing device, fully utilizing the high-density characteristics of read-only memory to improve the area efficiency of storage-based computing, and hence reduce or even eliminate unnecessary energy consumption caused by memory access.
The present implementation does not limit the number of read-only memory devices and current sources in the compute-in-memory device, nor does it limit the specific current source connected to the storage terminal of the read-only memory. The size of the preset voltage received by the current source is also not limited. The read-only memory devices in the compute-in-memory device can be separately set up, that is, one or more read-only memory devices in the compute-in-memory device output calculation results through separate calculation bitlines and are controlled by separate control wordlines. Alternatively, the read-only memory devices in the compute-in-memory device can be jointly set up, that is, two or more read-only memory devices in the compute-in-memory device can output calculation results through a single calculation bitline and can be controlled by the same control wordline. The present implementation does not limit the setting method of the read-only memory devices in the compute-in-memory device, and those skilled in the art can set them according to actual application situations and needs.
For example, the compute-in-memory device may include multiple calculation modules, each of which includes at least one read-only memory device. The calculation module can be viewed as the basic unit for data storage and calculation in the compute-in-memory device. The minimum size of the calculation module in the present implementation is that of one read-only memory device. Compared with the solution of using multiple transistors to implement the calculation module in the related art, the present implementation can significantly improve area efficiency and storage density.
Please refer to
As described above, the mainstream CMOS storage computing circuit design currently uses SRAM as the storage unit, which consists of at least six transistors. However, by using the characteristics of the read-only memory device, the disclosed implementation only uses one transistor to complete the information storage of one computation module, thereby improving the area efficiency. If each computation module in the disclosed implementation stores 1 bit of information, the area of the computation module is only 0.07 times that of the 6T SRAM under the same process. Moreover, the disclosed implementation also has the potential to use one transistor to achieve multi-bit data storage, which further improves the storage density.
The read-only memory device in the disclosed implementation can be a metal-oxide-semiconductor field-effect transistor (MOSFET) or a switching device with similar high-pass cutoff characteristics. The gate of the read-only memory device is connected to the control terminal, and the drain and source are respectively connected to the storage terminal and the output terminal, or the drain and source are respectively connected to the output terminal and the storage terminal, as shown in
It should be noted that in the disclosed implementation, the MOSFET is only an example, and all devices with switching characteristics can theoretically be used to build the compute-in-memory device proposed in the disclosed implementation.
In one possible implementation, the compute-in-memory device may include multiple computation modules, each of which includes a read-only memory device. The output end of the read-only memory device is directly connected to the corresponding computation bitline to output the result data through the computation bitline.
Please refer to
In a possible implementation, as shown in
Of course, the selector devices can also be metal-semiconductor-oxide field-effect transistors or switch devices with similar high-pass characteristics, where the gate electrodes of the selector devices are connected to the control terminals, and the drain and source electrodes are respectively connected to the storage and output terminals, or the drain and source electrodes are respectively connected to the output and storage terminals.
For example, as shown in
The control wordlines WL<1:2>, control bitlines BL<1:2>, and calculation bitline CBL can also serve as interfaces for the calculation module and be considered part of the calculation module, but this is not limiting according to the present disclosure.
In a possible implementation, the read-only memory devices in the calculation module can be arranged in a layout of multiple rows and columns. The control terminal of each row of read-only memory devices is connected to the same control wordline, and the output terminal of each column of read-only memory devices is connected to the same selector device. The output terminal of each selector device is connected to the same calculation bitline. For example, the gate electrodes of read-only memory devices M22 and M25 in the same row are connected to the control wordline WL<1>, and the gate electrodes of read-only memory devices M23 and M26 in the same row are connected to the control wordline WL<2>. The gate electrode of selector device M21 is connected to the control bitline BL<1>, and the gate electrode of selector device M24 is connected to the control bitline BL<2>. The drain electrode of read-only memory devices M22 and M23 in the same column is connected to the source electrode of selector device M21, and the drain electrode of read-only memory devices M25 and M26 in the same column is connected to the source electrode of selector device M24.
The present disclosure can control each storage module to perform multiplication operations (and logic operations), and an exemplary introduction of the structure of the calculation module shown in
Please refer to
In one example, as shown in (a) of
In another example, as shown in (b) of
Please refer to
In one possible implementation, as shown in
The wordline driver and bitline driver may be located in the control module as an example.
Please refer to
Input a set of data through control wordlines and activate the control terminals of the corresponding selector components through control bitlines to select the calculation modules participating in the multiply-accumulate operation.
Use the control wordlines to select the read-only memory components involved in the multiplication operation in each calculation module and complete the multiplication operation between the input data and the stored data corresponding to the preset voltage received by the current source.
Accumulate the multiplication results obtained from each calculation module and output them in the form of current through the calculation bit line to obtain the multiply-accumulate result.
For example, when performing a multiply-accumulate operation, all relevant control wordlines and control bitlines of the calculation module can be set to the lowest voltage level. One or more rows of selector components for the calculation bitline are opened. A set of data is input through the control wordlines, and if the output terminal of the read-only memory connected to the activated control wordline is connected to the input terminal of the selector component, the control terminal of the corresponding selector component is activated through the control bitline to select the read-only memory components participating in the calculation in each unit circuit. The multiplication operation is performed within the unit circuit, the results obtained are accumulated in the form of current through the calculation bitline, and the multiply-accumulate operation result is distinguished by detecting changes in the electrical characteristics of the calculation bitline.
In one example, as shown in (a) of
In another example, as shown in (b) of
In one possible implementation, the control module is also used to:
In working mode, each read-only memory component in the calculation module performs the aforementioned operations.
In idle mode, there is no current flow within the calculation module.
Please refer to
As for the operation mode, please refer to the previous introduction for the setting of multiplication and multiply-accumulate operations, which will not be repeated here. For the read operation, taking the control module in
The present implementation can adjust the voltage of the control wordlines and control bitlines according to the actual computational needs, selectively activate the calculation module to participate in the computation, and effectively reduce the overall power consumption of the system.
In one possible implementation, the control module is also used to adjust the signal timing of the control wordlines and control bitlines to achieve pipelining between different calculation bitlines. Please refer to
In an example shown in
It can be seen that the present implementation can achieve pipelining between different column calculation modules for multiply-accumulate operations by reasonably adjusting the signal timing on the control wordlines and control bitlines, thereby improving the throughput and utilization of the output detection interface. According to one aspect of the disclosure, a neural network accelerator is provided, which includes the aforementioned CiM device.
Please refer to
According to one aspect of the present disclosure, an electronic device is provided, which includes the above-mentioned neural network accelerator. As described above, the present implementation uses read-only memory devices to store weights, and each computing module requires at least one transistor and has the voltage for storing multiple bits. Compared with the traditional 6T SRAM structure, the storage density is significantly improved. More neural network parameters can be stored on limited chip areas, effectively reducing unnecessary memory access overhead. Each computing module of the CiM device in the present disclosure can switch between idle mode and working mode to adapt to different computing load scenarios, improve computing energy efficiency, and because the read-only memory device is non-volatile, there is no static power consumption. The multiply-accumulate operations on different calculation bitlines in the CiM device of the present disclosure can be pipelined during system implementation to improve the overall throughput rate and utilization rate of the output detection module circuit in a time-division multiplexing manner.
Please refer to
As shown in
Multiple storage units 310, each comprising a read-only memory device M31 (Read-only memory, ROM) and a capacitor CM. The read-only memory device M31 includes the first input terminal, the second input terminal, and the output terminal. The output terminal of the read-only memory device M31 is connected to the first terminal of the capacitor CM. The read-only memory device M31 stores data based on the connection relationship between its first input terminal, second input terminal, and output terminal. Specifically, when the first input terminal and the output terminal of the read-only memory device M31 are connected (i.e., the read-only memory device M31 includes a connection relationship between the first input terminal and the output terminal), the memory device stores first storage data (such as a high level ‘1’). Alternatively, when the second input terminal of the memory device is connected to the output terminal (i.e., the memory device includes a connection relationship between the second input terminal and the output terminal), the read-only memory device M31 stores second storage data (such as a low level ‘0’). The levels of the first and second storage data are different.
The control module 320, connected to each storage unit 310, is used for:
Controlling the voltage of the first and second input terminals of the read-only memory device M31 in each storage unit 310 to perform a required operation;
Determining the operational result of the required operation based on the voltage level at the second terminal of the capacitor CM.
The read-only memory device M31 used in this implementation is a type of non-volatile memory that only reads data and does not write data. The read-only memory device M31 used in this implementation has the characteristics of high storage density and low reading power consumption. Based on the read-only memory device M31, this implementation implements a content-addressable storage device, which can improve the area efficiency of the content-addressable storage device, and reduce or even eliminate unnecessary energy consumption caused by memory access to achieve the purpose of reducing energy consumption.
Currently, the storage units used in related CAM technologies mostly use SRAM circuit structures based on at least six transistors (6T), and also require additional logic operation circuits. Therefore, they contain at least ten transistors per storage unit 310. In contrast, this implementation achieves data storage based on electrical connection relationships (such as for three-terminal devices, when the read-only memory device M31 includes a connection relationship between the first input terminal and the output terminal, the storage device stores the first storage data; or, when the storage device includes a connection relationship between the second input terminal and the output terminal, the read-only memory device M31 stores the second storage data) without the need for transistors, thus further improving area efficiency. This implementation does not limit the specific implementation of the read-only memory device M31 as long as it can achieve data storage based on electrical connection relationships.
It should be noted that in this implementation of the invention, the read-only memory device implemented through electrical connections is just an example, and theoretically any device with similar electrical characteristics can be used to construct the CAM circuit proposed in this implementation of the invention.
The matching and addressing calculations implemented in this disclosure have the advantages of high area efficiency and low power consumption, and can be used in various applications such as route searching in high-speed real-time communication systems.
This disclosure does not limit the specific implementation of control module 320. As an example, control module 320 may include a processing component. In one example, the processing component includes but is not limited to a separate processor, discrete components, or a combination of processors and discrete components. The processor may include a controller in the electronic device with a function of executing instructions and can be implemented in any suitable manner, such as by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components. Inside the processor, the executable instructions can be executed through hardware circuits such as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers.
This disclosure does not limit the implementation of the capacitor CM in each storage unit 310. As an example, as shown in
It should be noted that the read-only memory device M31 in this disclosure includes electrical connections (i.e., the connection relationship), and the output terminal of the read-only memory device M31 is connected to the first or the second input terminal of the read-only memory device M31 according to the different data stored in the read-only memory device M31. This disclosure provides an example in
As an example, the operation of this disclosure may include a read operation to read the data stored in storage unit 310. When performing a read operation on storage unit 310, the voltages of the first and the second input terminals can be kept complementary (high level 1, low level 0), and the output terminal connected to the first or the second input terminal can be distinguished based on the voltage level of the output terminal, thereby obtaining the stored data.
When performing a matching addressing calculation using CAM, the XNOR logic calculation and weighted accumulation required for calculating the matching value are key operations and the main sources of power consumption. For the matching value calculation of input data and stored data, this disclosure can be decomposed into the XNOR logic calculation of each bit position of the input data and stored data, and then the results of the XNOR logic calculation are weighted and accumulated to obtain the matching value. The output of the XNOR logic calculation is ‘1’ when both inputs are ‘0’ or ‘1’, otherwise it is ‘0’. Therefore, the closer the input data and stored data are, the higher the resulting matching value.
In one possible implementation, the required operation may include an XNOR logic operation and voltage control can be applied to the first and second input terminals of the read-only memory device M31 for each storage unit 310 to perform the required operation. This may include:
The required operation's result is then determined based on the voltage of the second terminal of the capacitor CM, obtaining the XNOR logic operation result between the input data and the stored data.
Please refer to
The storage unit 310 of the CAM proposed in the present implementation has the advantages of high area efficiency, low power consumption, and can effectively improve the leakage problem in idle states. It is a type of circuit that can greatly improve area efficiency and reduce power consumption for computing within memory.
Of course, this implementation can provide interfaces for configuring each storage unit 310. For example, the interfaces of each storage unit 310 may include the first bitline BL, the second bitline BLB, and the matching line ML. In one possible implementation, the control module 320 is connected to the first input terminal of the read-only memory M31 via BL and to the second input terminal of the read-only memory M31 via BLB. The control module 320 is also connected to the second terminal of the capacitor CM via ML.
In practice, the output terminal of the storage unit 310 is connected to BL to represent a stored ‘1’, or to BLB to represent a stored ‘0’. Alternatively, in actual usage, the output terminal of the storage unit 310 can be connected to BLB to represent a stored ‘1’, and to BL to represent a stored ‘0’.
When reading from the storage unit 310, the voltage of BL and the voltage of BLB are kept complementary. Based on the voltage level at the output terminal, it can be determined whether the output terminal is connected to BL or BLB, thereby obtaining the stored information.
With this setup, the control module 320 can directly control the interfaces of the storage unit 310. Taking the XNOR logic operation shown in
Please refer to
In a possible implementation, as shown in
As shown in
Please refer to
It can be understood that the power consumption of the array circuit for matching addressing calculation in the present disclosed implementation is due to charging the capacitance CM. As shown in
It should be noted that the parameters of the device used to obtain the results in
In addition, in the idle state, the matching addressing calculation circuit in the present disclosed implementation has almost no leakage current, which can greatly improve the leakage problem of SRAM storage unit 310 in the idle state.
In one possible implementation, the required operation may also include a content addressing operation, where the first input and second input of the read-only memory device M31 for each storage unit 310 are voltage-controlled to perform the required operation. This may include:
Please refer to
In one example, as shown in (a) of
In another example, as shown in (b) of
Of course, in addition to arranging the storage units 310 in rows or columns, the implementation can also use other ways to arrange them. When performing content addressing, certain data can be addressed and matched based on the matching line ML. For example, assuming that multiple storage units 310 are distributed in different rows and columns but connected to the same matching line ML, the data that needs to be addressed and matched can be converted into binary numbers, resulting in multiple bits of data to be operated on. Each bit of data is assigned to each storage unit 310 on the matching line ML for an XNOR operation. The entire circuit connected to the matching line ML performs a normalization and accumulation operation. The control module 320 obtains a voltage value ranging from VDD to VSS through the matching line ML. The closer the voltage value is to 1, the more successful the match is.
In a possible implementation, the target operation includes a content-addressed operation, which involves voltage control of the first and second input terminals of read-only memory devices M1 for each storage unit 310 to perform the required operation. This may include:
This public implementation example does not limit the arrangement of multiple storage units 310 and can use a multi-row and multi-column array arrangement or other methods. For different arrangements, this example can achieve addressing matching for the required data, increasing flexibility.
In one possible implementation, the control module 320 can also be used to control at least one storage unit 310 to work in either a working mode or an idle mode. In the working mode, the storage unit 310 performs the required operation, while in the idle mode, the first input terminal, second input terminal, and output terminal of the read-only memory device M31 of the storage unit 310 are set to low voltage level VSS. The configuration of the first bitline BL and the second bitline BLB of each storage unit 310 in the working mode is described above and will not be repeated here.
Please refer to
Please refer to
According to one aspect of this disclosure, a memory is provided, which includes the above-mentioned content-addressable storage device.
According to one aspect of the present disclosure, an electronic device is provided which comprises a memory as described. The content-addressable storage device with high area efficiency and low power consumption disclosed in this embodiment can effectively improve the leakage problem of the SRAM storage unit 310 in an idle state and is a kind of content-addressable memory circuit that can greatly improve area efficiency and reduce power consumption.
A computing module and a computing array of the CiM device based on ROM with a current/charge domain computing functionality characteristic according to embodiments of the present disclosure will be described below with reference to the accompanying drawings.
As illustrated in
Multiple computing modules 410, each computing module 410 comprising at least one ROM device (Q1), multiple selection devices (Q11, Q12, Q21, Q22), electrical sources, data storage lines, control word-line (WL<1>), computing bit-lines (CBL), and data selection control lines (Ctrl, CtrlB). The control terminal of the ROM device (Q1) is connected to the corresponding control word-line (WL<1>) to receive control signals. The two data terminals of the ROM device (Q1) are connected to different data storage lines. The data storage lines are connected to the electrical source and the computing bit-line (CBL) through the selection devices. The data state of each data storage line is represented in the form of current or voltage, depending on the type of electrical source. The control terminal of the selection devices (Q11, Q12, Q21, Q22) is connected to the corresponding data selection control lines to receive data selection control signals. The data selection control signals are used to enable the corresponding selection devices.
Control module 420, the control module is connected with the control word-line and data selection control line of each computing module 410, which is used to select the corresponding ROM device and select the device for target operation through the control word-line and data selection control line, and output the result data through the computing bit-line.
The disclosed embodiment of the high-density CiM device based on ROM implements multi-level cell storage for a single ROM, enabling each computing module to have multi-bit data storage and computation capabilities. This allows each computing module to have multi-bit data storage and computation capabilities, thereby reducing or eliminating the need for off-chip memory access.
The high-density CiM device based on ROM described in this embodiment can be applied to neural network operations. Due to the multi-bit data storage and computation capabilities of each computing module, it has the potential to store all parameters of large-scale neural networks on-chip, reducing or even eliminating additional power consumption and delays caused by data movement between on-chip processing unit and off-chip memory. This reduces energy consumption and latency during inference operations, enabling efficient deployment of AI algorithms on edge devices. Additionally, due to its high-density storage characteristics, the high-density CiM device based on ROM can also achieve high energy efficiency and low latency when applied in other scenarios.
This embodiment does not impose limitations on the specific implementation of the ROM device. Those skilled in the art can choose suitable implementation methods based on practical situations and requirements. For example, the ROM device can be implemented using switches, diodes, bipolar transistors, metal-oxide-semiconductor field-effect transistors (MOSFETs), or other means.
This embodiment does not impose limitations on the specific implementation of the control module 410. Those skilled in the art can choose suitable implementation methods based on practical situations and requirements. In one example, the control module 410 may include a processing component, which can be an individual processor, discrete components, or a combination of a processor and discrete components. The processor can be implemented as a controller in an electronic device with instruction execution capabilities. The processor can be implemented in any suitable manner, such as using one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components. Inside the processor, the executable instructions can be executed by hardware circuits such as logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers.
This embodiment does not impose limitations on the number, type, or arrangement of the ROM devices and selection devices in the CiM device. It does not impose limitations on the number of data storage lines, control word-lines, computing bit-lines, or data selection control lines. It does not specify the specific data states corresponding to the data storage lines of the ROM device, nor does it impose limitations on the representation form of the data states (voltage or current). It does not impose limitations on the type of electrical source. For example, the electrical source can include multiple voltage sources or multiple current sources. The specific parameters of the current sources and voltage sources are not limited in this embodiment and can be set by those skilled in the art based on practical situations and requirements.
This embodiment does not impose limitations on the arrangement of the ROM devices in the computing module 410. For example, the memory array formed by the ROM devices in the computing module 410 can have multiple rows and one column, where each ROM device is controlled by an independent control word-line and outputs the corresponding computation result through the same computing bit-line. Alternatively, the memory array formed by the ROM devices in the computing module 410 can have multiple columns and one row, where multiple ROM devices are controlled by the same control word-line and output the corresponding computation results through computing bit-lines corresponding to different columns. The memory array formed by the ROM devices in the computing module 410 can have multiple rows and multiple columns, where each row of ROM devices is controlled by independent control word-lines and outputs the corresponding computation result through the same computing bit-line, or through computing bit-lines corresponding to different columns. This embodiment does not impose limitations on the specific configuration of the ROM devices in the CiM device. Those skilled in the art can choose suitable methods of implementation based on practical situations and requirements.
Based on the above description, the CiM device of this embodiment can include multiple computing modules 410, each of which can include at least one ROM device. Each ROM device in this disclosure is capable of storing multiple bits of data, which significantly increases the storage density on the processor chip compared to other related CiM techniques.
The specific implementation of the ROM devices and selection devices is not limited in this disclosure. Those skilled in the art can choose appropriate devices based on practical situations and requirements. For example, metal-oxide-semiconductor field-effect transistors (MOSFETs) can be used to implement the ROM devices and selection devices as switches.
As illustrated in
This disclosure utilizes MOSFETs to implement ROM devices, data selection devices, column selection devices, and reset switches. The impedance characteristics between the drain and source terminals of the MOSFETs are leveraged to create switches. The gate of the MOSFET serves as the control terminal, enabling the device to store information and perform switching functions.
The chosen MOSFETs in this disclosure are specifically the N-type MOSFETS. A single MOSFET can change its impedance characteristics between the drain and source terminals by controlling the potential difference between the gate voltage and the source voltage. When the drain and source are in a low-impedance state, the circuit is conductive, and when they are in a high-impedance state, the circuit is disconnected.
It should be noted that in this disclosure, the MOSFET is just one example, and theoretically, any device with switch-like characteristics can be used to construct the CiM device proposed in this disclosure.
In one possible implementation, as shown in
Both the input terminals of the first lower data selection device Q21 and the second lower data selection device Q22 are connected to the electrical source;
The output terminal of the first lower data selection device Q21 is connected to the input terminal of the second upper data selection device Q12 through the first data storage line; The output terminal of the second lower data selection device Q22 is connected to the input terminal of the first upper data selection device Q11 through the second data storage line;
The control terminal of the first lower data selection device Q21 and the control terminal of the first upper data selection device Q11 are both connected to the first data selection control line CtrlB;
The control terminal of the second lower data selection device Q22 and the control terminal of the second upper data selection device Q12 are both connected to the second data selection control line Ctrl;
The output terminals of the first upper data selection device Q11 and the second upper data selection device Q12 are connected to the computing bit-lines;
The first data terminal and the second data terminal of each ROM device (Q1) are respectively connected to the corresponding first data storage line and second data storage line.
In one possible implementation, as illustrated in
For example, the number of data selection devices may be directly related to the number of bits stored in the ROM devices. When more bits are stored in the ROM devices, the number of data selection devices will increase. In the example shown in
It should be noted that the descriptions in
In one possible implementation, as illustrated in
In one example, as illustrated in
It should be noted that the specific connection of the serially connected components is not limited in this disclosed embodiment. As illustrated in
In one possible implementation, if the electrical source is multiple voltage sources, the computing module further includes a reset switch and a capacitor, the control terminal of the reset switch is connected to a reset control word-line to receive a reset signal, the reset terminal of the reset switch is used to receive a reset state voltage, the reset detection terminal of the reset switch and the first terminal of the capacitor are connected to the output terminal of the upper data selection device, the second terminal of the capacitor is connected to the computing bit-line.
In one possible implementation, as illustrated in
As an example, as illustrated in
In one possible implementation, as illustrated in
For example, as illustrated in
For example, as illustrated in
For example, each computing module may be connected to one or more current sources or voltage sources with different operating values. In current-domain computing mode, the current sources may be connected to one or more ROM devices. In charge-domain computing mode, the voltage sources may be connected to one or more ROM devices. This embodiment combines the high storage density characteristics of the ROM devices through the reuse of the current sources and voltage sources, thereby improving the overall area efficiency.
For example, in the computing modules shown in
In one possible implementation, this disclosure enables the sharing of selection devices and data storage lines, further increasing the storage density of the CiM device.
In one possible implementation, the individual ROM devices within the computing module are combined in a layout consisting of multiple rows and columns. The control terminals of each row of ROM devices are connected to the same control word-line. The two data terminals of each ROM device in each column are respectively connected to the corresponding first data storage line and second data storage line. The output terminals of each upper data selection device are connected to the corresponding computing bit-line through capacitors or directly, and the desired result data is output through these computing bit-lines.
In one possible implementation, the first data storage line connected to each ROM device in the (K+1)th column is the same as the second data storage line connected to each ROM device in the Kth column, where K is an integer.
The following provides an illustrative explanation of such implementation.
In one possible implementation, as illustrated in
For example, as illustrated in
Similarly, as illustrated in
The above describes partial possible implementations of the computing module in the charge domain and current domain, as well as the first type of sharing of selection devices in each possible implementation. However, this disclosure is not limited to these implementations. The computing module may have other implementations in the charge domain and current domain, as well as other sharing methods. The following provides an illustrative explanation.
In one possible implementation, as shown in
The input terminals of the first lower data selection device Q21 and the input terminals of the second lower data selection device Q22 are connected to each other and connected to the electrical source (such as current sources and voltage sources); The output terminal of the first lower data selection device Q21 is connected to the input terminal of the second upper data selection device Q12, and it is also connected to the input terminal of the corresponding column selection device;
The output terminal of the second lower data selection device Q22 is connected to the input terminal of the first upper data selection device Q11, and it is also connected to the input terminal of another column selection device, the output terminals of each column selection device are connected to the corresponding data storage lines;
The output terminals of the first upper data selection device Q11 and the second upper data selection device Q12 are connected to the computing bit-line;
The control terminal of the first lower data selection device Q21 and the control terminal of the first upper data selection device Q11 are both connected to the first data selection control line;
The control terminal of the second lower data selection device Q22 and the control terminal of the second upper data selection device Q12 are both connected to the second data selection control line;
The first data terminal and the second data terminal of each ROM device are respectively connected to the corresponding first data storage line and second data storage line.
In one possible implementation, the individual ROM devices within the computing module are combined in a layout consisting of multiple rows and columns. The control terminals of each row of ROM devices are connected to the same control word-line. The two data terminals of each ROM device in each column are respectively connected to the corresponding first data storage line and second data storage line. The output terminals of each upper data selection device are connected to the corresponding computing bit-line through capacitors or directly, and the desired result data is output through these computing bit-lines.
In one possible implementation, the first data storage line connected to each ROM device in the (K+1)th column is the same as the second data storage line connected to each ROM device in the Kth column, where K is an integer.
Wherein, multiple column selection devices corresponding to each column of ROM devices share the same group of data selection devices, as illustrated in
In one possible implementation, as illustrated in
Furthermore, as shown in
For example, as shown in
In the current domain and charge domain, the control method for performing read or multiplication operations on the ROM devices in the second type of sharing method is the same as the first type of sharing method, and will not be further elaborated here.
In one possible implementation, as illustrated in
In one possible implementation, as illustrated in
In one possible implementation, the target operations include multiplication, data reading, and MAC operations, in the case of multiplication, the result data is the product of the stored data of the corresponding ROM device at the respective moment and the input data from the control word-line, in the case of data reading operation, the result data is the stored data of the corresponding read-only memory device at the respective moment, the result data also includes the MAC result.
The control module is further used to control one or multiple columns of computing modules to perform MAC operations, and/or control a portion or all of the computing modules connected to the same compute bit-line to perform MAC operations;
Specifically, the control module input a set of data through the control word-line and select the computing modules participating in the MAC operation, select the ROM devices participating in the calculation in each computing module using the control word-line, select the stored data participating in the calculation using the data selection control line, perform the multiplication operation between the input data and the stored data in each computing module, accumulate the multiplication results from each computing module through the computing bit-line and output the MAC result.
The following provides an illustrative example of possible implementations to achieve the target operations.
The embodiments of the present disclosure control each computing module to realize multiplication operation. The following is an illustrative description on the operation using the computing modules in the current domain and charge domain shown in
In one example, as shown in
In another example, as shown in
In one example, as shown in
In another example, as shown in
In one possible implementation, the disclosed example may also include MAC operations. The control module may control the corresponding control word-lines, control bit-lines, and data selection control lines to select the corresponding stored data from the ROM devices participating in the computation within the CiM device's computing modules. By realizing one or more computing bit-lines, MAC operations of the computing modules connected with the same computing bit-line are carried out simultaneously.
In an example, as illustrated in
In another example, as shown in
In an example, as illustrated in
In another example, as shown in
In one possible implementation, the control module is further used to control each computing module to enter the working mode or idle mode. In the working mode, each computing module performs the target operations.
Exemplarily, the computing modules of the CiM device of the embodiments of the present disclosure can be configured to the working mode or the idle mode. Wherein, in the working mode, the read, multiply and MAC operations can be performed; by controlling the voltage levels of the control word-lines and control bit-lines, the computing module can be set to the idle mode, thereby reducing energy consumption. In addition, data are stored through hard-wiring the data terminal of the ROM device with the corresponding data storage lines to realize the non-volatile characteristics of the CiM device according to embodiments of the present disclosure.
In an example, as illustrated in
In an example, the computing modules in the aforementioned examples that perform MAC operations can be used with other circuits, for example, one or more switching devices can be inserted between the computing modules. In embodiments of the present disclosure, MOSFET is adopted as the switching device; by adjusting the timing of signals such as the control word-lines and the control bit-lines, pipelined operations between the computation and output detection interfaces of computing modules connected to different computing bit-lines can be implemented.
In another example, the pipelined operation may refer to the operation such that the result of the previous computing module is passed to the current computing module via the computing bit-line CBL and participates the compute operation of the current computing module, then the result of the current computing module is passed to the next computing module via the computing bit-line CBL.
In an example, as illustrated in
In an example, as illustrated in
In the disclosed embodiments, by adjusting the timing of the signals of the control word-lines and the control bit-lines, pipelined operations can be achieved between the MAC operations and the output detection interfaces of computing modules connected to different computing bit-lines, therefore improving the throughput and output detection interface utilization of the system.
The specific timing of the control word-lines and control bit-lines during pipeline operations is not limited in the disclosed example. Those skilled in the art can choose appropriate methods to implement it based on practical considerations and requirements.
According to one aspect of the disclosure, a neural network accelerator is provided, which includes the aforementioned CiM device.
As illustrated in
In an exemplary embodiment, this disclosed example can be used to accelerate the inference of fixed-point neural network.
According to one aspect of the disclosure, an electronic device is provided, which includes the aforementioned neural network accelerator.
Based on the above description, in this disclosed example, the use of ROM for weight storage in the in-memory computing device is utilized. By making full use of both terminals of the ROM devices and employing selection device sharing methods, the CiM device exhibits high storage density characteristics. Traditional 6T SRAM structures use 6 transistors to store 1-bit of information, whereas in this disclosed example, a single transistor can be used to store multi-bit information in a ROM device. In an exemplary embodiment, a single ROM device can be used to store 4-bit information, greatly increasing the storage density and enabling the chip to have the potential to store all parameters of large-scale neural networks on-chip, thereby reducing or even eliminating the additional power consumption and delay caused by data movement between on-chip processing unit and off-chip memory, and reducing the energy consumption and delay during inference. This enables efficient deployment of artificial intelligence algorithms on edge or terminal devices.
The present disclosure may be a system, a method and/or a computer program product. The computer program product may comprise the computer-readable storage medium, which carried the compute-readable program instructions that are used for the processor to realize various aspects of the present disclosure.
The computer-readable storage medium may be a tangible device that can hold and store instructions used by the instruction execution device. The computer-readable storage medium may be, for example, but is not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage medium (a non-exhaustive list) include terminalable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), compact disc read-only memory (CD-ROM), digital versatile disc (DVD), memory stick, floppy disk, mechanically encoded device, a hole card or a raised structure in a groove with instructions stored thereon, and any suitable combination of the above. The computer-readable storage medium as used herein shall not be construed as transient signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., pulses of light through fiber optic cables), or transmitted electrical signals.
The computer-readable program instructions described herein may be downloaded from the computer-readable storage medium to each computing/processing device or downloaded to an external computer or external storage device through a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or a network interface in each computing/processing device receives computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium of each computing/processing device.
Computer program instructions for performing the operations of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in one or any combination of more programming languages, wherein the programming language includes object-oriented programming languages—such as Smalltalk, C++, etc., and conventional procedural programming languages—such as the “C” language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user computer through any kind of network, such as a local area network (LAN) or a wide area network (WAN), or it may be connected to an external computer (such as through the Internet via an Internet service provider). In some embodiments, the computer-readable program instructions may be executed by electronic circuits that can be customized using the state information of computer-readable program instructions such as programmable logic circuits, field programmable gate arrays (FPGAs), and programmable logic arrays (PLAs), thereby realizing the various aspects of the present disclosure.
Various aspects of the present disclosure are described here with reference to the methods and the flowcharts and/or block diagrams of the apparatuses (systems) and computer program products according to embodiments of the present disclosure. It shall be understood that each block of the flowcharts and/or block diagrams, and combinations of the blocks of the flowcharts and/or block diagrams, may be implemented with computer-readable program instructions.
These computer-readable program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine so that when the instructions are executed by the processor of the computer or other programmable data processing apparatus, the apparatus for realizing the functions/actions specified in one or more blocks of the flowcharts and/or block diagrams is produced. These computer-readable program instructions may also be stored in a computer-readable storage medium, and these instructions cause computers, programmable data processing devices, and/or other devices to work in a specific manner so that the computer-readable medium storing instructions includes an article of manufacture comprising instructions for implementing the various aspects of the functions/actions specified in one or more blocks of the flowcharts and/or block diagrams.
The computer-readable program instructions may also be loaded into a computer, other programmable data processing device, or other equipment; thereby a series of operational steps are performed on the computer, other programmable data processing device, or other equipment to produce a computer-implemented process, so that instructions executed on computers, other programmable data processing devices, or other devices implement the functions/actions specified in one or more blocks in the flowcharts and/or block diagrams.
The flowcharts and block diagrams in the drawings illustrate the systems, methods, as well as system architectures, functionalities, and operations that can be implemented by computer program products according to the various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, a terminalion of a program segment, or an instruction that includes one or more executable instructions. In some alternative implementations, the functionalities noted in the blocks may occur out of the order noted in the drawings. For example, two blocks in successive may in fact be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It shall also be noted that each block of the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowchart illustration, may be implemented with a dedicated hardware-based system that performs the specified function or action, or may be implemented with a combination of dedicated hardware and computer instructions.
The computer program product may be implemented by means of hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.
Various embodiments of the present disclosure are described above. The foregoing descriptions are exemplary, not exhaustive, and are not limited to the disclosed embodiments. Many modifications and alterations will be apparent to a person skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles of each embodiment, the practical applications, or improvements over technologies in the market, or to enable a person skilled in the art to understand each embodiment disclosed herein.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202210769401.8 | Jun 2022 | CN | national |
| 202211599037.1 | Dec 2022 | CN | national |
| 202211728423.6 | Dec 2022 | CN | national |
| 202310259263.3 | Mar 2023 | CN | national |
