Patent Application
20220012570
This disclosure relates generally to machine learning, and, more particularly, to a Bayesian compute unit with reconfigurable sampler and methods and apparatus to operate the same.
In recent years, artificial intelligence (e.g., machine learning, deep learning, etc.) have increased in popularity. Artificial intelligence may be implemented using neural networks. Neural networks are computing systems inspired by the neural networks of human brains. A neural network can receive an input and generate an output. The neural network includes a plurality of neurons corresponding to weights can be trained (e.g., can learn, be weighted, etc.) based on feedback so that the output corresponds a desired result. Once the weights are trained, the neural network can make decisions to generate an output based on any input. Neural networks are used for the emerging fields of artificial intelligence and/or machine learning. A Bayesian neural network is a particular type of neural network that includes neurons that generate a variable weight as opposed to a fixed weight. The variable weight falls within a probability distribution defined by a mean value and a variance determined during training of the Bayesian neural network.
The figures are not to scale. In general, the same reference numbers will be used throughout the drawing(s) and accompanying written description to refer to the same or like parts. Connection references (e.g., attached, coupled, connected, and joined) are to be construed broadly and may include intermediate members between a collection of elements and relative movement between elements unless otherwise indicated. As such, connection references do not necessarily infer that two elements are directly connected and in fixed relation to each other. Although the figures show layers and regions with clean lines and boundaries, some or all of these lines and/or boundaries may be idealized. In reality, the boundaries and/or lines may be unobservable, blended, and/or irregular.
Descriptors “first,” “second,” “third,” etc. are used herein when identifying multiple elements or components which may be referred to separately. Unless otherwise specified or understood based on their context of use, such descriptors are not intended to impute any meaning of priority, physical order or arrangement in a list, or ordering in time but are merely used as labels for referring to multiple elements or components separately for ease of understanding the disclosed examples. In some examples, the descriptor “first” may be used to refer to an element in the detailed description, while the same element may be referred to in a claim with a different descriptor such as “second” or “third.” In such instances, it should be understood that such descriptors are used merely for ease of referencing multiple elements or components.
Machine learning models, such as neural networks, are used to perform a task (e.g., classify data). Machine learning can include a training stage to train the model using ground truth data (e.g., data correctly labelled with a particular classification). Training a traditional neural network adjusts the weights of neurons of the neural network. After trained, data is input into the trained neural network and the weights of the neurons are applied to input data to be able to process the input data to perform a function (e.g., classify data).
Overfitting and/or sensitivity to malicious attacks negatively affect the performance and/or reliability of traditional neural networks. Overfitting occurs when a model is trained to have too small of an error. If the training results in too small of an error, the model has a difficult time generalizing for new situations. Malicious attacks can exploit a combination of overfitting and/or knowledge of the underlying neural network model. Sensitivity to malicious attacks is the result of a trained model being overconfident in its outputs. If a model is overconfident, small perturbations to the inputs can result in undesired and/or unpredictable changes in the output. Both of the above problems are caused by the failure of traditional neural networks to include uncertainty information in a finite set of training data.
Bayesian neural networks (BNNs) and/or Bayesian Deep Neural Networks (BDNN) introduce uncertainty information to overcome the problems of overfitting and sensitivity to malicious attacks. Instead of using fixed weights, BDNNs introduce weights associated with conditioned probability distribution (e.g., the output weight may be a value within a probability distribution defined by a mean (herein also referred to as mu or u) and standard deviation and/or variance (generally referred to as σ2)). Because BDNNs introduce some amount of randomness, BDNNs can be trained with smaller training data without sacrificing accuracy. However, traditional BDNNs with neurons that generate weights corresponding to a probability distribution require a lot of power and/or hardware to implement. Therefore, such traditional BDNNs are slow due to bottlenecks caused by the sampling of a probability distribution and/or the multiple iterations of forward passes of a BDNN with different weight value(s) sampled from the distribution. For example, traditional BDNN generate a single weight per probability distribution which requires a lot of overhead to generate and/or store multiple weights because the traditional systems access the mean and variance from system memory for every weight generated.
The computations in a BDNN layer for multiple forwarded passes can be represented using the below Equation 1.
O
nj=ΣiIniWnij=ΣiIniN(μij, ρij2) (Equation 1)
In Equation 1, O is the output, I is the input, W is the weight n is the index of the forward pass, i and j are indices of the weight element in a filter, μ is the mean, and σ2 is the variance. The below Equation 2 is a reparameterization of Equation 1.
O
nj=ΣiIniWnij=N(ΣiIniμij, ΣiIniσij2)=N(βnj, δnj2) (Equation 2)
Based on Equation 2, examples disclosed herein can sample the jth output element of nth forward pass from the distribution with mean and variance δ2. Through this mathematical re-organization, the number of samples that need to be generated is reduced from number of weights in dimension i and j, to number of outputs in dimension j. For a convolutional layer with configuration k×k×I×O, total number of weights are k2I O and number of outputs are H×W×O where k×k is the kernel size, I is input channels, O is output channels, H and W are height and width of output image. For a fully connected layer, number of weights are I×0 and number pf outputs are O. Thus, examples disclosed herein leverage the mathematical re-organization of Equation 2 to reduce sampling overhead.
Examples disclosed herein leverage the advantage of the above Equation 2 to provide an efficient BDNN using a sampling unit that supports both Gaussian distribution models and Gaussian mixture model (GMM) distribution models to leverage input re-use for BDNN workloads. As used herein, a unit may include hardware (e.g., a circuit, a processor, etc.), software, firmware, and/or any combination thereof. Examples disclosed herein take input data and apply a mean and variance to generate a mean-based result (e.g., a product of the mean and the input) and a variance-based result (e.g., the product of the variance and the input). The results are provided to a programmable sampling unit. The programmable sampling unit can be configured to output multiple samples from a parameterized Gaussian distribution as well as GMM models based on the mean-based result and the variance-based result. In this manner, the mean and variance value(s) corresponding to the signal probability distribution and the input data are accessed once and multiple different outputs can be generated and used based on the single distribution, thereby allowing low sampling overhead. Accordingly, examples disclosed herein result in a more efficient artificial intelligence-based compute unit that reduces the amount of data movement to generate and apply weights corresponding to a probability distribution.
The computations of examples disclosed herein may be unique because the activations corresponding to different forward passes are typically unique. To obtain throughput improvements, examples disclosed herein re-use computations through input reuse (IR). Accordingly, examples disclosed herein leverage the below Equation 3. Equation 3 is a mathematical repurposing of Bayesian computations to leverage IR. This mathematical formulation considers the same input across different forward passes:
O
nj=ΣiIiWnij=ΣiIiN(μij, σij2)=N(ΣiIiμij, ΣiIi2σij2)=N(βj, δj2) (Equation 3)
Through this mathematical reparameterization, examples disclosed herein obtain a single output distribution parameter for each activation (e.g., pixel) corresponding to multiple forward passes. By sampling multiple output samples from this distribution, examples disclosed herein perform Bayesian computations with lower number of total computations.
In general, implementing a machine learning (ML)/artificial intelligence (AI) system involves two phases, a learning/training phase and an inference phase. In the learning/training phase, a training algorithm is used to train a model to operate in accordance with patterns and/or associations based on, for example, training data. In general, the model includes internal parameters that guide how input data is transformed into output data, such as through a series of nodes and connections within the model to transform input data into output data. Additionally, hyperparameters may be used as part of the training process to control how the learning is performed (e.g., a learning rate, a number of layers to be used in the machine learning model, etc.). Hyperparameters are defined to be training parameters that are determined prior to initiating the training process.
Different types of training may be performed based on the type of ML/AI model and/or the expected output. As used herein, labelling refers to an expected output of the machine learning model (e.g., a classification, an expected output value, etc.). Alternatively, unsupervised training (e.g., used in deep learning, a subset of machine learning, etc.) involves inferring patterns from inputs to select parameters for the ML/AI model (e.g., without the benefit of expected (e.g., labeled) outputs).
In examples disclosed herein, training is performed until a threshold number of actions have been predicted. In examples disclosed herein, training is performed either locally (e.g., in the device) or remotely (e.g., in the cloud and/or at a server). Training may be performed using hyperparameters that control how the learning is performed (e.g., a learning rate, a number of layers to be used in the machine learning model, etc.). In some examples re-training may be performed. Such re-training may be performed in response to a new program being implemented or a new user using the device. Training is performed using training data. When supervised training may be used, the training data is labeled. In some examples, the training data is pre-processed.
Once training is complete, the model is deployed for use as an executable construct that processes an input and provides an output based on the network of nodes and connections defined in the model. The model is stored locally in memory (e.g., cache and moved into memory after trained) or may be stored in the cloud. The model may then be executed by the computer cores.
Once trained, the deployed model may be operated in an inference phase to process data. In the inference phase, data to be analyzed (e.g., live data) is input to the model, and the model executes to create an output. This inference phase can be thought of as the AI “thinking” to generate the output based on what it learned from the training (e.g., by executing the model to apply the learned patterns and/or associations to the live data). In some examples, input data undergoes pre-processing before being used as an input to the machine learning model. Moreover, in some examples, the output data may undergo post-processing after it is generated by the AI model to transform the output into a useful result (e.g., a display of data, an instruction to be executed by a machine, etc.).
In some examples, output of the deployed model may be captured and provided as feedback. By analyzing the feedback, an accuracy of the deployed model can be determined. If the feedback indicates that the accuracy of the deployed model is less than a threshold or other criterion, training of an updated model can be triggered using the feedback and an updated training data set, hyperparameters, etc., to generate an updated, deployed model.
The example NN trainer 102 of
The example BDNN 104 of
The example neurons 110a-f of
The example distribution buffer 202 of
The example distribution memory 206 of
The example PEs 208 of
The example output memory 212 of
The example PSU 214 of
As described above, the example IRAM 210 stores the input and/or activation data. The example IRAM 210 outputs the stored input and/or activation data to the example squaring logic circuitry 220 and a first input of the example MUX 222. The example squaring logic circuitry 220 performs a mathematical squaring function to multiply the input data to itself (e.g., to generate I2). The example squaring logic circuitry 220 outputs the squared input to a second input of the example MUX 222.
The example MUX 222 of
The example distribution memory 206 of
The example MUX 224 of
The example PE 208 of
ORAM 212 to be stored.
The example MUX control circuitry 225 of
The example MUX 230 of
The example Gaussian sample circuitry 232 of
The example average circuitry 234 of
The example activations 300 are stored in the example input memory 210 of
The example PEs 208 of the first tile (Tile 0)
The example PEs 208 of
The example GRNG circuitry 400 of
The example GRNG circuitry 400 of
The example logic gate 404 of
The example parameterization circuitry 408 generates multiple weights in parallel (e.g., simultaneously) that correspond to a probability distribution based on a single mean and variance value. The example parameterization circuitry 408 obtains the single mean and variance value from the system memory 106 and/or the distribution memory 206 via the memory interface 420. To generate the weights, the parameterization circuitry 408 multiplies the variance by each of the numbers in the sequence (G1-GN) output by the GRNG circuitry 400 and adds the mean to the resulting product. For example, the multiplier array 410 multiplies the single variance value (e.g., from the system memory 106 for
O
n=(Gn)(δ2)+β, for n=1, 2, . . . N (Equation 4)
In Equation 4, O is the output, G is a number of the sequence output by the GRNG circuitry 400, β is the mean-based value stored in the ORAM 212 (e.g., μI) and δ2 is the variance-based value stored in the ORAM 212 (e.g., I2σ2).The example parameterization circuitry 208 supports FP16 representation formal, INT8 representation format, and/or any other representation format. For some representation formals (e.g., FP16), floating point conversion circuitry may be included to convert the fixed point output of the GRNG circuitry 400 to floating point prior to inputting into the example parameterization circuitry 408. In some examples, the output of the Hadamard transform circuit 406 includes a total number of numbers in the sequence sufficient to apply to N forward passes and/or N activations. In this manner, the parameterization circuitry 408 can generate a plurality of outputs for the N activations. Thus, one or more of the PEs 208 can process N activations corresponding to N forward passes or N images in a batch using the N outputs that correspond to a single probability distribution. The output of the parametrization circuitry (e.g., the outputsO1-ON), may be output to the example average circuitry 234 of
The PSU(s) 200 of
The example GMM processing circuitry 414 of
N
GMM=ΣiϕiN(βi, δi2) (Equation 5)
In the above-Equation 2, βi and δi2 are the mean and variance-based values of the ith Gaussian distribution in the GMM and ϕi is the weight of that distribution. GMM is characterized by multiple trained distributions to help improve the accuracy of Neural networks. The example GMM processing circuitry 414 generates the GMM based on the below Equation 6, which corresponds to a rewritten version of Equation 5.
N
GMM
=N(Σiϕiβi, Σiϕi2δij2) (Equation 6)
For example, the MINI processing circuitry 414 includes an example multiplier array 416 that multiplies the output of the example parameterization circuitry 408 by the ϕ1-M additional variance-based value(s). The example adder tree and mean circuit 418 obtains the products (e.g., the outputs multiplied by ϕi) and accumulates to generate samples representative of the mixture model distribution. The example adder tree and means circuit 418 outputs the GMM based outputs to the example distribution buffer 202.
The example memory interface 420 of
While an example manner of implementing the BDNN 104 of
Flowchart representative of example hardware logic, machine readable instructions, hardware implemented state machines, and/or any combination thereof for implementing the example BDNN 104 of
The machine readable instructions described herein may be stored in one or more of a compressed format, an encrypted format, a fragmented format, a compiled format, an executable format, a packaged format, etc. Machine readable instructions as described herein may be stored as data (e.g., portions of instructions, code, representations of code, etc.) that may be utilized to create, manufacture, and/or produce machine executable instructions. For example, the machine readable instructions may be fragmented and stored on one or more storage devices and/or computing devices (e.g., servers). The machine readable instructions may require one or more of installation, modification, adaptation, updating, combining, supplementing, configuring, decryption, decompression, unpacking, distribution, reassignment, compilation, etc. in order to make them directly readable, interpretable, and/or executable by a computing device and/or other machine. For example, the machine readable instructions may be stored in multiple parts, which are individually compressed, encrypted, and stored on separate computing devices, wherein the parts when decrypted, decompressed, and combined form a set of executable instructions that implement a program such as that described herein.
In another example, the machine readable instructions may be stored in a state in which they may be read by a computer, but require addition of a library (e.g., a dynamic link library (DLL)), a software development kit (SDK), an application programming interface (API), etc. in order to execute the instructions on a particular computing device or other device. In another example, the machine readable instructions may need to be configured (e.g., settings stored, data input, network addresses recorded, etc.) before the machine readable instructions and/or the corresponding program(s) can be executed in whole or in part. Thus, the disclosed machine readable instructions and/or corresponding program(s) are intended to encompass such machine readable instructions and/or program(s) regardless of the particular format or state of the machine readable instructions and/or program(s) when stored or otherwise at rest or in transit.
The machine readable instructions described herein can be represented by any past, present, or future instruction language, scripting language, programming language, etc. For example, the machine readable instructions may be represented using any of the following languages: C, C++, Java, C#, Perl, Python, JavaScript, HyperText Markup Language (HTML), Structured Query Language (SQL), Swift, etc.
As mentioned above, the example processes of
“Including” and “comprising” (and all forms and tenses thereof) are used herein to be open ended terms. Thus, whenever a claim employs any form of “include” or “comprise” (e.g., comprises, includes, comprising, including, having, etc.) as a preamble or within a claim recitation of any kind, it is to be understood that additional elements, terms, etc. may be present without falling outside the scope of the corresponding claim or recitation. As used herein, when the phrase “at least” is used as the transition term in, for example, a preamble of a claim, it is open-ended in the same manner as the term “comprising” and “including” are open ended. The term “and/or” when used, for example, in a form such as A, B, and/or C refers to any combination or subset of A, B, C such as (1) A alone, (2) B alone, (3) C alone, (4) A with B, (5) A with C, (6) B with C, and (7) A with B and with C. As used herein in the context of describing structures, components, items, objects and/or things, the phrase “at least one of A and B” is intended to refer to implementations including any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein in the context of describing structures, components, items, objects and/or things, the phrase “at least one of A or B” is intended to refer to implementations including any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. As used herein in the context of describing the performance or execution of processes, instructions, actions, activities and/or steps, the phrase “at least one of A and B” is intended to refer to implementations including any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B. Similarly, as used herein in the context of describing the performance or execution of processes, instructions, actions, activities and/or steps, the phrase “at least one of A or B” is intended to refer to implementations including any of (1) at least one A, (2) at least one B, and (3) at least one A and at least one B.
As used herein, singular references (e.g., “a”, “an”, “first”, “second”, etc.) do not exclude a plurality. The term “a” or “an” entity, as used herein, refers to one or more of that entity. The terms “a” (or “an”), “one or more”, and “at least one” can be used interchangeably herein. Furthermore, although individually listed, a plurality of means, elements or method actions may be implemented by, e.g., a single unit or processor. Additionally, although individual features may be included in different examples or claims, these may possibly be combined, and the inclusion in different examples or claims does not imply that a combination of features is not feasible and/or advantageous.
At block 502, the example MUX control circuitry 225 determines if input data has been obtained (e.g., stored in the input memory 210). If the example MUX control circuitry 225 has determined that input data has not been obtained (block 502: NO), control returns to block 502 until input data has been obtained. If the example MUX control circuitry 225 has determined that input data has been obtained (block 502: YES), the example MUX control circuitry 225 determines if the layer 108b should operate as a DNN or a B-DNN (block 504) (e.g., based on user preferences, manufacturer preferences, and/or instructions from another component).
If the example MUX control circuitry 225 determines that layer 108b should act as a DNN (block 504: DNN), the example MUX control circuitry 225 controls the MUXs 222, 224 to operate in a DNN mode (block 516). For example, the MUX control circuitry 225 outputs one or more control signals to the select inputs of the MUXs 222, 224 to cause the MUXs to output the mean and the input data to the example PE 208. If the example MUX control circuitry 225 determines that layer 108b should act as a B-DNN (block 504: B-DNN), the example MUX control circuitry 225 controls the MUXs 222, 224 to operate in a B-DNN mode (block 506). For example, the MUX control circuitry 225 outputs one or more control signals to the select inputs of the MUXs 222, 224 to cause the MUXs to output the mean, variance, input, and squared input to the example PE 208.
At block 508, the example PEs 208 apply the mean and variance values to the input/activation data. For example, one or more of the PEs 208 can multiply the mean to an input/activation value and multiply the variance to the square of the input/activation value to generate a mean-based value and a variance-based value. At block 510, the example output memory 212 stores the results (e.g., the mean-based value and the variance-based value). At block 512, the example MUX control circuitry 225 determines if it is time to generate the output value based on the mean-based value and variance-based value stored in the example output memory 212. As described above, if the PSU 214 computes weights for all the nodes in a layer, then the MUX control circuitry 225 controls the MUXs 230 so that the PSU 214 handles all the output data for each node in series (e.g., processing data for the first node at time one, the second node at time two, the third node at time three, etc.).
If the example MUX control circuitry 225 determines that it is not time to generate the output value (block 512: NO), control returns to block 512 until it is time. If the example MUX control circuitry 225 determines that it is time to generate the output value (block 512: YES), the MUX control circuitry 225 outputs one or more control signals to the select input of the example MUXs 230 so that the PSU 214 can generate the output based on the information of the node 110c and the example PSU generates the output (block 514), as further described below in conjunction with
At block 518, the example input memory 210 determines if additional input data has been obtained (e.g., stored in the input memory 210). If the example input memory 210 has determined that input data has not been obtained (block 502: NO), the instructions end. If the example input memory 210 has determined that additional input data has been obtained (block 502: YES), control returns to block 504.
At block 602, the example GRNG circuitry 400 generates a random number (or pseudo random number) sequence (e.g., including a plurality of numbers G1-GN corresponding to the number of activations to be processed at the entire layer 108b or the number of activations to be processed by a particular one of the compute nodes 110c-e. The GRNG circuitry 400 may generate the random/pseudo random number sequence based on a Hadamard transform of an XOR operation of outputs of multiple registers 402, as described above in conjunction with
At block 606, the example multiplier array 410 generates a plurality of products (δ2G1, δ2G2, . . . , δ2GN) by performing a scalar multiplication based on (a) an array of the random/pseudo random numbers from the random/pseudo random number sequence (e.g., G1-GN) and (b) the first element of the results (e.g., the variance-based value (e.g., δ2)). At block 608, the example adder array 412 generates a plurality of outputs (e.g., O1-ON) corresponding to the single mean-based value (β) and variance-based value (δ2) by adding the second element of the results (e.g., the mean-based value (β)) to the plurality of products. Because the random/pseudo random number sequence includes N total numbers, the adder array 412 generate N weights corresponding to a probability distribution based on the single mean and variance value.
At block 610, the example GMM processing circuitry 414 determines the PSU 214 is operating in GMM model (e.g., to generate a probability distribution corresponding to multiple modes and variances). For example, the system memory 106 may store an indication that the probability distribution to be used is mixture model and/or may include multiple means/mean-based values and variances/variance-based values when the probability distribution to be used is a mixture model distribution. If the example GMM processing circuitry 414 determines that mixture model samples are not needed (block 610: NO), control continues to block 614.
If the example GMM processing circuitry 414 determines that mixture model samples are needed (block 610: YES), the example GMM processing circuitry 414 generates mixture model based outputs based on the plurality of outputs and mixture model distribution data (block 612). For example, the array multiplier 416 multiplies the outputs O1-ON by the value ϕi-N (e.g., which is stored in the example output memory 212) to generate products corresponding to the mixture model distribution. Additionally, the example adder tree and mean circuit 418 adds one or more additional means/mean-based values to the mixture model based products to generate the mixture model samples. At block 614, the example average circuitry 234 averages the outputs by performing a mathematical average and/or mean-based on the N outputs. At block 616, the example output memory 212 stores the average as the output of the node. After 616, control returns to block 514 of
The processor platform 700 of the illustrated example includes a processor 712. The processor 712 of the illustrated example is hardware. For example, the processor 712 can be implemented by one or more integrated circuits, logic circuits, microprocessors, GPUs, DSPs, or controllers from any desired family or manufacturer. The hardware processor may be a semiconductor based (e.g., silicon based) device. In this example, the processor 712 implements at least one of the example layers 108a-c, the example compute nodes 110c-e, the example PSU 214, the example PEs 208, the example squaring circuitry 220, the example MUXs 222, 224, 230, the example MUX control circuitry 225, the example averaging circuitry 234, the example GRNG circuitry 400, the example logic gate 404, the example Hadamard transform circuitry 406, the example parameterization circuitry 408, the example multiplier array 410, the example adder array 412, the example GMM processing circuitry 414, the example multiplier array 416, the example adder tree and mean circuitry 418, the example memory interface 420 of
The processor 712 of the illustrated example includes a local memory 713 (e.g., a cache). In the example of
The processor platform 700 of the illustrated example also includes an interface circuit 720. The interface circuit 720 may be implemented by any type of interface standard, such as an Ethernet interface, a universal serial bus (USB), a Bluetooth® interface, a near field communication (NFC) interface, and/or a PCI express interface.
In the illustrated example, one or more input devices 722 are connected to the interface circuit 720. The input device(s) 722 permit(s) a user to enter data and/or commands into the processor 712. The input device(s) can be implemented by, for example, an audio sensor, a microphone, a camera (still or video), a keyboard, a button, a mouse, a touchscreen, a track-pad, a trackball, and/or a voice recognition system.
One or more output devices 724 are also connected to the interface circuit 720 of the illustrated example. The output devices 724 can be implemented, for example, by display devices (e.g., a light emitting diode (LED), an organic light emitting diode (OLED), a liquid crystal display (LCD), a cathode ray tube display (CRT), an in-place switching (IPS) display, a touchscreen, etc.), a tactile output device, and/or speaker. The interface circuit 720 of the illustrated example, thus, typically includes a graphics driver card, a graphics driver chip and/or a graphics driver processor.
The interface circuit 720 of the illustrated example also includes a communication device such as a transmitter, a receiver, a transceiver, a modem, a residential gateway, a wireless access point, and/or a network interface to facilitate exchange of data with external machines (e.g., computing devices of any kind) via a network 726. The communication can be via, for example, an Ethernet connection, a digital subscriber line (DSL) connection, a telephone line connection, a coaxial cable system, a satellite system, a line-of-site wireless system, a cellular system, etc.
The processor platform 700 of the illustrated example also includes one or more mass storage devices 728 for storing software and/or data. Examples of such mass storage devices 728 include floppy disk drives, hard drive disks, compact disk drives, Blu-ray disk drives, redundant array of independent disks (RAID) systems, and digital versatile disk (DVD) drives.
The machine executable instructions 732 of
The cores 802 may communicate by an example bus 804. In some examples, the bus 804 may implement a communication bus to effectuate communication associated with one(s) of the cores 802. For example, the bus 804 may implement at least one of an Inter-Integrated Circuit (I2C) bus, a Serial Peripheral Interface (SPI) bus, a PCI bus, or a PCIe bus. Additionally or alternatively, the bus 804 may implement any other type of computing or electrical bus. The cores 802 may obtain data, instructions, and/or signals from one or more external devices by example interface circuitry 806. The cores 802 may output data, instructions, and/or signals to the one or more external devices by the interface circuitry 806. Although the cores 802 of this example include example local memory 820 (e.g., Level 1 (L1) cache that may be split into an L1data cache and an L1instruction cache), the microprocessor 800 also includes example shared memory 810 that may be shared by the cores (e.g., Level 2 (L2_cache)) for high-speed access to data and/or instructions. Data and/or instructions may be transferred (e.g., shared) by writing to and/or reading from the shared memory 810. The local memory 820 of each of the cores 802 and the shared memory 810 may be part of a hierarchy of storage devices including multiple levels of cache memory and the main memory (e.g., the main memory 714, 716 of
Each core 802 may be referred to as a CPU, DSP, GPU, etc., or any other type of hardware circuitry. Each core 802 includes control unit circuitry 814, arithmetic and logic (AL) circuitry (sometimes referred to as an ALU) 816, a plurality of registers 818, the L1cache 820, and an example bus 822. Other structures may be present. For example, each core 802 may include vector unit circuitry, single instruction multiple data (SIMD) unit circuitry, load/store unit (LSU) circuitry, branch/jump unit circuitry, floating-point unit (FPU) circuitry, etc. The control unit circuitry 814 includes semiconductor-based circuits structured to control (e.g., coordinate) data movement within the corresponding core 802. The AL circuitry 816 includes semiconductor-based circuits structured to perform one or more mathematic and/or logic operations on the data within the corresponding core 802. The AL circuitry 816 of some examples performs integer based operations. In other examples, the AL circuitry 816 also performs floating point operations. In yet other examples, the AL circuitry 816 may include first AL circuitry that performs integer based operations and second AL circuitry that performs floating point operations. In some examples, the AL circuitry 816 may be referred to as an Arithmetic Logic Unit (ALU). The registers 818 are semiconductor-based structures to store data and/or instructions such as results of one or more of the operations performed by the AL circuitry 816 of the corresponding core 802. For example, the registers 818 may include vector register(s), SIMD register(s), general purpose register(s), flag register(s), segment register(s), machine specific register(s), instruction pointer register(s), control register(s), debug register(s), memory management register(s), machine check register(s), etc. The registers 818 may be arranged in a bank as shown in
Each core 802 and/or, more generally, the microprocessor 800 may include additional and/or alternate structures to those shown and described above. For example, one or more clock circuits, one or more power supplies, one or more power gates, one or more cache home agents (CHAs), one or more converged/common mesh stops (CMSs), one or more shifters (e.g., barrel shifter(s)) and/or other circuitry may be present. The microprocessor 800 is a semiconductor device fabricated to include many transistors interconnected to implement the structures described above in one or more integrated circuits (ICs) contained in one or more packages. The processor circuitry may include and/or cooperate with one or more accelerators. In some examples, accelerators are implemented by logic circuitry to perform certain tasks more quickly and/or efficiently than can be done by a general purpose processor. Examples of accelerators include ASICs and FPGAs such as those discussed herein. A GPU or other programmable device can also be an accelerator. Accelerators may be on-board the processor circuitry, in the same chip package as the processor circuitry and/or in one or more separate packages from the processor circuitry.
More specifically, in contrast to the microprocessor 800 of
In the example of
The interconnections 910 of the illustrated example are conductive pathways, traces, vias, or the like that may include electrically controllable switches (e.g., transistors) whose state can be changed by programming (e.g., using an HDL instruction language) to activate or deactivate one or more connections between one or more of the logic gate circuitry 908 to program desired logic circuits.
The storage circuitry 912 of the illustrated example is structured to store result(s) of the one or more of the operations performed by corresponding logic gates. The storage circuitry 912 may be implemented by registers or the like. In the illustrated example, the storage circuitry 912 is distributed amongst the logic gate circuitry 908 to facilitate access and increase execution speed.
The example FPGA circuitry 900 of
Although
In some examples, the processor circuitry 712 of
A block diagram illustrating an example software distribution platform 1005 to distribute software such as the example computer readable instructions 732 of
Example methods, apparatus, systems, and articles of manufacture to provide a Bayesian compute unit with reconfigurable sampler and methods and apparatus to operate the same are disclosed herein. Further examples and combinations thereof include the following: Example 1 includes a node of a neural network to apply a plurality of weights in an artificial intelligence-based model, the node comprising a processor element to generate (a) a first element by applying a mean value to an activation and (b) a second element by applying a variance value to a square of the activation, the mean value and the variance value corresponding to a single probability distribution, a programmable sampling unit to generate a pseudo random number, and generate an output based on the pseudo random number, the first element, and the second element, wherein the output corresponds to the single probability distribution, and output memory to store the output.
Example 2 includes the node of example 1, wherein the processor element is to generate the first element by multiplying the mean value with the activation, and generate the second element by multiplying the variance value with the square of the activation.
Example 3 includes the node of example 1, wherein the programmable sampling unit is to generate the output by generating a product by multiplying the second element by the pseudo random number, and adding the first element by the product.
Example 4 includes the node of example 1, wherein the programmable sampling unit is to generate a plurality pseudo random numbers, the plurality of pseudo random numbers including the pseudo random number, generate a plurality of outputs, average the plurality of outputs, and generate the output based on the average.
Example 5 includes the node of example 1, wherein the programmable sampling unit is to generate a plurality of outputs for a plurality of nodes, the plurality of nodes including the node.
Example 6 includes the node of example 1, wherein the programmable sampling unit is to generate the pseudo random number by generating a pseudo random sequence using shift registers, adjust the pseudo random sequence by performing an exclusive OR (XOR) function to the pseudo random sequence, and convert the adjusted pseudo random sequence to a Gaussian pseudo random number sequence, the Gaussian pseudo random number sequence including the pseudo random number.
Example 7 includes the node of example 1, wherein the output memory is to output the output to input memory of a subsequent node.
Example 8 includes a non-transitory computer readable storage medium comprising instructions which, when executed, cause one or more processors to at least generate (a) a first element by applying a mean value to an activation and (b) a second element by applying a variance value to a square of the activation, the mean value and the variance value corresponding to a single probability distribution, generate a pseudo random number, generate an output based on the pseudo random number, the first element, and the second element, wherein the output corresponds to the single probability distribution, and output memory to store the output.
Example 9 includes the computer readable storage medium of example 8, wherein the instructions cause the one or more processors to generate the first element by multiplying the mean value with the activation, and generate the second element by multiplying the variance value with the square of the activation.
Example 10 includes the computer readable storage medium of example 8, wherein the instructions cause the one or more processors to generate the output by generating a product by multiplying the second element by the pseudo random number, and adding the first element by the product.
Example 11 includes the computer readable storage medium of example 8, wherein the instructions cause the one or more processors to generate a plurality pseudo random numbers, the plurality of pseudo random numbers including the pseudo random number, generate a plurality of outputs, average the plurality of outputs, and generate the output based on the average.
Example 12 includes the computer readable storage medium of example 8, wherein the instructions cause the one or more processors to generate a plurality of outputs for a plurality of nodes, the plurality of nodes including the node.
Example 13 includes the computer readable storage medium of example 8, wherein the instructions cause the one or more processors to generate the pseudo random number by generating a pseudo random sequence using shift registers, adjust the pseudo random sequence by performing an exclusive OR (XOR) function to the pseudo random sequence, and convert the adjusted pseudo random sequence to a Gaussian pseudo random number sequence, the Gaussian pseudo random number sequence including the pseudo random number.
Example 14 includes the computer readable storage medium of example 8, wherein the instructions cause the one or more processors to output the output to input memory of a subsequent node.
Example 15 includes an apparatus to apply a plurality of weights in an artificial intelligence-based model, the apparatus comprising at least one memory, and processor circuitry including one or more of at least one of a central processing unit, a graphic processing unit or a digital signal processor, the at least one of the central processing unit, the graphic processing unit or the digital signal processor having control circuitry to control data movement within the processor circuitry, arithmetic and logic circuitry to perform one or more first operations corresponding to instructions, and one or more registers to store a result of the one or more first operations, the instructions in the apparatus, a Field Programmable Gate Array (FPGA), the FPGA including logic gate circuitry, a plurality of configurable interconnections, and storage circuitry, the logic gate circuitry and interconnections to perform one or more second operations, the storage circuitry to store a result of the one or more second operations, or Application Specific Integrate Circuitry including logic gate circuitry to perform one or more third operations, the processor circuitry to at least one of perform at least one of the first operations, the second operations or the third operations to generate (a) a first element by applying a mean value to an activation and (b) a second element by applying a variance value to a square of the activation, the mean value and the variance value corresponding to a single probability distribution, generate a pseudo random number, and generate an output based on the pseudo random number, the first element, and the second element, wherein the output corresponds to the single probability distribution, and output memory to store the output.
Example 16 includes the apparatus of example 15, wherein the processor circuitry is to generate the first element by multiplying the mean value with the activation, and generate the second element by multiplying the variance value with the square of the activation.
Example 17 includes the apparatus of example 15, wherein the processor circuitry is to generate the output by generating a product by multiplying the second element by the pseudo random number, and adding the first element by the product.
Example 18 includes the apparatus of example 15, wherein the processor circuitry is to generate a plurality pseudo random numbers, the plurality of pseudo random numbers including the pseudo random number, generate a plurality of outputs, average the plurality of outputs, and generate the output based on the average.
Example 19 includes the apparatus of example 15, wherein the processor circuitry is to generate a plurality of outputs for a plurality of nodes, the plurality of nodes including the node.
Example 20 includes the apparatus of example 15, wherein processor circuitry is to generate the pseudo random number by generating a pseudo random sequence using shift registers, adjust the pseudo random sequence by performing an exclusive OR (XOR) function to the pseudo random sequence, and convert the adjusted pseudo random sequence to a Gaussian pseudo random number sequence, the Gaussian pseudo random number sequence including the pseudo random number.
Example 21 includes the apparatus of example 15, wherein processor circuitry is to output the output to input memory of a subsequent node.
From the foregoing, it will be appreciated that example methods, apparatus and articles of manufacture have been disclosed that provide a Bayesian compute unit with reconfigurable sampler and methods and apparatus to operate the same. BDNNs introduce uncertainty information to overcome the problems of overfitting and sensitivity to malicious attacks. Instead of using fixed weights, BDNNs introduce weights associated with conditioned probability distribution (e.g., the output weight may be a value within a probability distribution defined by a mean and standard deviation). Because BDNNs introduce some amount of randomness, BDNNs can be trained with smaller training data without sacrificing accuracy. However, traditional BDNNs distribute different mean and variance value(s) corresponding to different probability distributions for every compute node in a layer. Therefore, such traditional BDNNs require a large amount of bandwidth and take time to access the multiple mean and variance values from system memory to generate weights that correspond to the multiple different probability distributions.
Examples disclosed herein generate multiple weights that correspond to a single probability distribution (e.g., a Gaussian distribution and/or a GMM distribution). Examples disclosed herein utilize the apply a mean and variance to input and/or activation data and apply a Gaussian sampler to the mean-based data and/or variance to apply to a plurality of different activations in a compute node of a AI-based model (e.g., a neural network, a machine learning model, a deep learning model, etc.). In this manner, only mean value(s) and variance value(s) corresponding to a single distribution are accessed from system memory to apply to multiple different activations. Thereby reducing the bandwidth and time needed to access probability distribution data to generate weighted outputs. Accordingly, the disclosed methods, apparatus and articles of manufacture are accordingly directed to one or more improvement(s) in the functioning of a neural network.
It is noted that this patent claims priority from Indian Provisional Patent Application Number 202141024844 which was filed on Jun. 4, 2021, and is hereby incorporated by reference in its entirety.
Although certain example methods, apparatus and articles of manufacture have been disclosed herein, the scope of coverage of this patent is not limited thereto. On the contrary, this patent covers all methods, apparatus and articles of manufacture fairly falling within the scope of the claims of this patent.
The following claims are hereby incorporated into this Detailed Description by this reference, with each claim standing on its own as a separate embodiment of the present disclosure.
| Number | Date | Country | Kind |
|---|---|---|---|
| 202141024844 | Jun 2021 | IN | national |
This patent arises from Indian Provisional Patent Application Number 202141024844 which was filed on Jun. 4, 2021. Indian Provisional Patent Application Number 202141024844 is hereby incorporated herein by reference in its entirety. Priority to Indian Provisional Patent Application Number 202141024844 is hereby claimed.
