Patent Application
20220108203
Machine learning (ML) is a type of artificial intelligence (Al) that helps a software system learn to recognize patterns from data without being directly programmed to do so. Machine learning can refer to a wide range of techniques. Examples of ML techniques include neural networks and deep learning. Broadly, ML techniques operate in two phases, which are: (a) a training phase, in which models and/or weights are adjusted based on received training data; and (b) an operating phase, in which those models and/or weights are applied to received actual data.
ML techniques are useful in smart systems to detect or recognize various aspects of the environment. Examples of such detection or recognition scenarios include speech recognition, object recognition in video and/or images, gesture and motion recognition, sound signature detection, anomaly detection, and the like. Machine learning techniques may consume large amounts of processing, data and power resources. These requirements impose challenges that interfere with executing complex ML techniques on a portable, battery-operated and/or low-powered device. To address those challenges, one workaround offloads processing of some or all ML operations to a remote network (such as a cloud) and returns the results of those offloaded ML operations to the device. However, that workaround may send irrelevant information to the cloud for processing. For example, an “always on” device may constantly listen for specific commands or keywords, capture a substantial amount of irrelevant audio, and upload the irrelevant audio to the cloud for processing, which results in wasted bandwidth, processing power and potential privacy issues.
In a memory device, a static random access memory (SRAM) circuit includes an array of SRAM cells arranged in rows and columns and configured to store data. The SRAM array is configured to: store a first set of information for a machine learning (ML) process in a lookup table in the SRAM array; and consecutively access, from the lookup table, information from a selected set of the SRAM cells along a row of the SRAM cells. A memory controller circuit is configured to select the set of the SRAM cells based on a second set of information for the ML process.
In another aspect, a system includes one or more microprocessors coupled to a memory circuit. The memory circuit includes static random access memory (SRAM) circuit including an array of SRAM cells arranged in rows and columns and configured to store data, the SRAM array configured to: store a first set of information for a machine learning (ML) process in a lookup table in the SRAM array; and consecutively access, from the lookup table, information from a selected set of the SRAM cells along a row of the SRAM cells. A memory controller circuit is configured to select the set of the SRAM cells based on a second set of information for the ML process.
In another aspect, a method includes receiving a first set of information for a machine learning (ML) process; storing the first set of information in a lookup table in an array of SRAM cells, the SRAM cells arranged in rows and columns; selecting a set of the SRAM cells based on a second set of information for the ML process; and consecutively accessing, from the lookup table, information from the selected set of the SRAM cells along a row of the SRAM cells.
To increase efficiency in devices, special-purpose hardware executes a specific operation along with or instead of more general-purpose hardware. One example is a graphics processing unit (GPU) along with a central processing unit (CPU). In aspects of this description, an ML hardware accelerator based on distributed arithmetic (DA) and near memory computing may improve efficiency of operating ML networks.
Accordingly, a circuit 200 (
Accordingly, one example performs a right shift 206 of the contents of the accumulator 204, and then adds (via an adder 208) the result to the contents of accumulator 204. This process is repeated for all K bits of the received data to determine a solution for the equation. Generally, solving using DA increases a number of addition operations as compared to conventional techniques for solving such equations, but without complex multiplications. This helps the DA computation to be more efficient than conventional techniques, because the LUT lookup, bit shift and addition steps consume smaller amounts of processing, data and power resources as compared to complex multiplication operations of conventional techniques. Using DA computation, an alternative way to evaluate the weighted-sum is populating the LUTs using X[i]'s and using A[i]'s to address the LUTs.
In certain cases, a size of the LUT may be reduced, such as by using multiple smaller LUTs or offset binary coding.
As a more detailed example, LUT 212 may be coupled to a multiplexer (mux) 276, as shown in another example DA circuit 275 of
For example, the input for X[m], destined for Y[n], may be multiplied by a certain weight A[m, n], and each input into Y[n] may be then summed to determine a value for Y[n]. One or more LUTs may then be determined for Y as a function of weight A for the X layer, and these calculations for the ML network 300 may be performed using DA as described above.
Generally, the speed and simplicity of static random access memory (SRAM) have allowed SRAM to be fabricated in a single integrated circuit that includes a microprocessor, such as for a near processor cache. In aspects of this description, a burst SRAM may be modified to help accelerate processing for ML networks based operations. Burst SRAM is further described in detail in conjunction with U.S. Pat. No. 9,734,896, which is hereby incorporated by reference. Generally, in conventional SRAM (such as with a mux-factor of eight), for each read cycle, eight bit lines may be precharged, but only a single column is read based on a column decoder address. After the value is read, the lines are precharged/recharged again prior to reading out another value. In contrast, burst SRAM allows multiple memory values to be provided, or burst, from SRAM at a rate of one output per cycle, without required additional time and/or power for bit line precharging/recharging.
In operation, the SRAM memory circuit 400 memory access cycle begins when a clock signal CLK goes high. The input address ADD is latched, and row decoder 405 begins decoding a portion of the address field and provides a high voltage on a selected one of the word lines WLa-WLc, selected by a portion of the address. The column decoder 407 begins by decoding a second portion of the address field ADD and provides a select signal to the Y-select multiplexer 409. The Y-select multiplexer determines which ones of the bit line pairs BLTO, BLCO-BLTn, BLCn is selected.
When the word line voltage on the selected word line WLa-WLc rises, the complementary bit lines for the SRAM cells along the selected row are coupled to the storage nodes within each of the SRAM cells. The voltages on the complementary bit line pairs in each column begin to spread apart as the differential voltage in the SRAM cells is shared with the bit lines. Each bit line along the active row will transition to the differential voltage value of the storage nodes of the SRAM cells in the corresponding columns.
The column select multiplexer 409 then couples the selected bit line pair to the complementary output signals YT and YC. Column decoder 407 determines which column is selected, based on a column portion of the input address ADD. Sense amplifier 413 then receives the differential voltage signal, senses the differential voltage, latches and amplifies it, and provides the data from the selected SRAM cell as the output data signal DATA.
As described above, the memory access cycle includes several steps performed within the SRAM memory circuit 400 during each clock cycle. In certain cases, multiple sequential (e.g., consecutive, or back-to-back) SRAM reads of SRAM cells arranged along a particular row may be performed. Before the first SRAM cell access to a newly addressed row (or, for a first SRAM access after a power up, reset, or wake up operation), a precharge operation precharges all the complementary bit line pairs in the SRAM array. The selected row line is determined for the first SRAM cell for consecutive access, and the row decoder asserts the selected word line, such as WLa. When the word line voltage is raised above a transistor threshold voltage for the SRAM cells along a particular row in the SRAM array, the complementary bit line pairs coupled to the SRAM cells each receive a differential voltage corresponding to the stored data within each of the SRAM cells along the selected row line. The column decoder simultaneously decodes the column portion of the address field and controls the Y-select multiplexer to select the column for the read operation. A sensing operation is then performed by the sense amplifier 413, and the first read data becomes available as the sense amplifier 413 provides the latched sense data.
After the first SRAM memory cell access, the second SRAM cell is accessed by selecting the next addressed column (which is on the same row in the SRAM array, and which is optionally adjacent to the column of the previously accessed SRAM cell), in the consecutive access. The corresponding bit line pair for this column is then coupled to the sense amplifier 413 (or another sense amplifier), the differential voltage is sensed, and the data from this SRAM cell is provided. The SRAM array may be configured with any number of columns and one or more corresponding multiplexers and sense amplifiers. In certain cases, the SRAM array may include eight columns of cells, and the Y-select multiplexer may likewise be configured to perform eight reads from the cells in a row. LUT values may be stored in the SRAM array, so eight columns of the SRAM array correspond to values of in eight-bit wide LUT. Consecutive accesses to the LUT may then be performed by accessing multiple, back-to-back accesses from the cells in a row. The consecutive access is performed for a given row, but the specific order in which the columns of the given row may be any order. For example, in an eight entry lookup table, the ML process can make eight back-to-back accesses, but these could be entry numbers 1, 5, 3, 2, 5, 6, 4, 7 from the LUT.
Storing the LUT in the burst SRAM thus allows multiple LUT values (stored in the LUT) to be read from the SRAM, one at a time. As described above, DA-based computations include K lookups in the LUT for values corresponding to each bit of the received data. However, these lookups may not be consecutive, because the looked-up value depends on the received bit value. Instead, the received bit value drives the Y-select multiplexer, so the appropriate LUT value is returned for the received bit value.
For example, in a single output computation (e.g., convolution) with 64 inputs Y=A0.X0+A1.X1+A2.X2++A63.X63, sixteen 8-entry LUTs may be constructed of groups of four weights (A0-A3, A4-A7, A8-A11, . . . , A60-A63). An output register of the accumulator circuit 506 may be initialized to all zeros. For each of the 16 LUTs, received bits are loaded in groups of 4 (X[4i], X[4i+1], X[4i+2], X[4i+3], for i=0 to 15), into address registers. The output register may be initialized based on the first address in the LUT (0,0,0). Then, once per set of entries in the LUT (e.g., eight times in this example), the operations include right shifting address registers by 1 to get a next address bit (X0, X1, X2, X3), performing an XOR operation to obtain the LUT address bits (a0, a1, a2) and a sign control signal, reading the contents of the LUT [i][a0:a1:a2], and conditionally summing (e.g., adding or subtracting) to the right shifted accumulator. In certain cases, the accumulator circuit 506 includes logic configured to perform an appropriate bit shift and addition logic, as described in conjunction with
In convolutional layers of a neural network, the same convolutional kernel is applied to the entire input feature map. In aspects of this description, throughput of an ML hardware accelerator is increased by processing multiple sets of received data using the same LUT concurrently.
As described above, ML networks may include various layers. These layers may include convolutional, depth-wise, point-wise, fully connected, etc. layers. Each type of layer may be handled slightly differently. For example, the circuits as described in conjunction with
In certain cases, an ML network may include binary or ternary weights having values of +1/−1 or +1/0/−1, respectively. In such cases, it may be relatively inefficient to build a LUT based on the weights, because the core convolution in such cases primarily includes addition or subtraction functions. Instead, information in a LUT may be populated based on the received data A[K] and the weights that access the LUT. This is because the weights are defined as fixed values, instead of varying based on nodes. The received data is unknown, so the data values may be precomputed at run time and placed into the LUT based on the weights.
As illustrated in
As shown in
A non-volatile storage device 1020 can include one or more disk drives, optical drives, solid-state drives (SSDs), tap drives, flash memory, electrically programmable read only memory (EEPROM), and/or any other type memory configured to maintain data for a duration of time after a power loss or shut down operation. The non-volatile storage device 1020 may also store programs that are loaded into the RAM when such programs executed.
Software programs may be developed, encoded and compiled in a variety of computing languages for a variety of software platforms and/or operating systems and subsequently loaded and executed by processor 1005. In one embodiment, the compiling process of the software program may transform program code written in a programming language to another computer language, so the processor 1005 is able to execute the programming code. For example, the compiling process of the software program may generate an executable program that provides encoded instructions (e.g., machine code instructions) for processor 1005 to accomplish specific, non-generic, particular computing functions.
After the compiling process, the encoded instructions may then be loaded as computer executable instructions or process steps to the processor 1005 from the storage device 1020, from memory 1010, and/or embedded within processor 1005 (e.g., via a cache or internal ROM). Processor 1005 may be configured to execute the stored instructions or process steps, in order to perform instructions or process steps to transform the computing device into a non-generic, particular, specially programmed machine or apparatus. Stored data (e.g., data stored by the storage device 1020) may be accessed by processor 1005 during the execution of computer executable instructions or process steps to instruct one or more components within the computing device 1000. Storage device 1020 may be partitioned or split into multiple sections that may be accessed by different software programs. For example, storage device 1020 may include a section designated for specific purposes, such as storing program instructions or data for updating software of the computing device 1000. In one embodiment, the software to be updated includes the ROM, or firmware, of the computing device. In certain cases, the computing device 1000 may include multiple operating systems. For example, the computing device 1000 may include a general-purpose operating system for normal operations. The computing device 1000 may also include another operating system, such as a bootloader, for performing specific tasks, such as upgrading and recovering the general-purpose operating system, and allowing access to the computing device 1000 at a level generally unavailable through the general-purpose operating system. Both the general-purpose operating system and another operating system may have access to the section of storage device 1020 designated for specific purposes.
The communications interface 1025 may include a radio communications interface configured to interface with one or more radio communications devices. In certain cases, elements coupled to the processor 1005 may be integrated on hardware shared with the processor 1005. For example, the communications interface 1025, storage device 1020, and memory 1010 may be integrated, along with other elements such as a digital radio, in a single integrated circuit or package, such as in a system on a chip (SOC). Computing device 1000 may also include an input (and/or output) device 1030, such as sensors, cameras, human input devices (e.g., mouse, keyboard, touchscreen), monitors, display screen, tactile or motion generators, speakers, lights, etc. Processed data, such as from the input device 1030, may be provided from the computing device 1000 via the communications interface 1025 to one or more other devices.
Modifications are possible in the described examples, and other examples are possible, within the scope of the claims.
