Patent Application
20230035620
The present application relates to a neural network processor.
The present application further relates to a method of neural network processing.
Artificial Neural Networks (ANNs) show great performance in several data analysis tasks including visual and auditory applications. However, direct implementation of these algorithms without considering the sparsity of data requires high processing power, consumes vast amounts of energy and suffers from scalability issues. Yousefzadeh et al. address these concerns in “Conversion of Synchronous Artificial Neural Network to Asynchronous Spiking Neural Network using sigma-delta quantization”, Conference Paper March 2019 DOI: 10.1109/AICAS.2019.8771624. According to the approach described therein, asynchronous spikes are used to communicate the quantized output activations of the neural network processor elements (also denoted as neurons). In this paper it is recognized that direct quantization would result in an excessive firing activity of spiking neurons. The reason is that for quantizer input values X near the transition point of two quantization levels, small variations/oscillations in X may result in several big changes in the quantized Y which is not desirable. To mitigate this risk, the output of the “Hysteresis Quantizer” as proposed in the conference paper depends not only on the current input value but also on the previous value of its output. It is however a disadvantage of this known approach that it requires a relatively large storage space for each neuron, i.e. it requires that in addition to the current state also the quantized value of the previous state is stored. Accordingly, there is a need to a solution that maintains the advantages of this known approach, while mitigating the storage requirements.
In accordance with this need a neural network processor is provided as claimed in claim 1, and a method of neural network processing is provided as claimed in claim 12.
The claimed neural network processor comprises a plurality of mutually succeeding neural network processor layers and each neural network processor layer comprising a plurality of neural network processor elements having a respective state register for storing a state value X indicative for their state.
Neural network processor elements in a neural network processor layer are capable of transmitting differential event messages indicating a change of their state to neural network processor elements in a succeeding neural network processor layer in accordance with an activation function. A neural network processor element having a state value receiving at its input an input event message from another neural network processor element is configured to compute a new state value of its state in accordance with the input message. It selectively generates an output event message dependent on the change of its state value.
The neural network processor element comprises an additional state register for storing a value of a state value change indicator that is indicative of a direction of a previous state change exceeding a threshold value, wherein selectively generating an output event message is further dependent on the value of said indicator. Hence, contrary to the known approach, only one bit needs to be stored in addition to the state value. Therewith a substantial reduction of storage space is required, whereas an excessive “spiking” is avoided.
In an embodiment, a plurality of neural network processor elements is organized as a cluster, wherein the plurality of neural network processor elements shares a common computation unit. As the number of messages that is generated may be small, this optional further measure is particularly favorable. Due to the relatively low number of messages, the plurality of neural network processor elements can be efficiently kept updated by the common computation unit in a time-shared manner.
The plurality of neural network processor elements organized in the cluster may further share a common message buffer.
Also the plurality of neural network processor elements organized in the cluster may further share a common output unit.
Furthermore, the plurality of neural network processor elements organized in the cluster may share a common network interface.
In an embodiment a plurality of neural network processor elements organized in the cluster may form a neural network processor layer in the neural network processor. This is advantageous in that in practice, most message transfers will take place from a layer to a next layer. Therewith transmission of messages from a neural network processor element to its recipient neural network processor elements can take place efficiently. For example, a neural network processor element may send a single message to the cluster forming the next neural network processor layer, and the single message may be broadcasted inside the latter cluster to the recipient neural network processor elements.
In an embodiment a neural network processor element comprises a first computation subsection to compute a control signal indicative for a significant change of its state value, a second computation subsection to compute a modified control signal indicative of a significant change taking into account a previous significant change, and a third computation subsection to generate an output message. The neural network processor element may be configured such that the second computation section is only activated upon the condition that the control signal indicates a significant change of the state value, and such that the third computation section is only activated upon the condition that both the control signal indicates a significant change of the state value and the modified control signal indicates a significant change. Therewith the average computation load as well as the average computation time can be substantially reduced. The computation subsections are respective program modules in a program that causes a programmable processor to perform the computations. Alternatively, the computation subsections may be hardware modules in a dedicated computation unit.
The neural network processor may have an input for receiving sensor data, and an output to provide device control signals and be employed as a control system.
The control system may comprise a normalization unit to normalize the received sensor data, for example to accommodate for changing illumination conditions where image data is received as the sensor data.
The control system may further comprise an event generation unit to convert received sensor data into event data. Alternatively, a sensor may be used that provides its sensor data as event data.
A controlled device may include the control system in addition to at least one sensor unit to provide the control system with sensor data and a device to receive the device control signals.
These and other aspects are described in more detail with reference to the drawing. Therein:
Like reference symbols in the various drawings indicate like elements unless otherwise indicated.
i.e. if |R(t)−R(t−1)|>Thr
if |G(t)−G(t−1)|>Thg
if |B(t)−B(t−1)|>Thb
In this example the neural network processor has a total of 108K neural processor elements, and 1.6M parameters.
Parameters are the individual weights. The number of weights for a layer typically is the product of the number of channels in that layer, the number of channels of the previous layer and the size of the kernel. So for the first layer between EG and CV1 the following number NW of weights is computed:
NW=24(channel in the same layer)*(5*5)(kernel size)*3(channel of previous layer).
In an embodiment the weights may be determined by supervised learning with Stochastic Gradient Descent algorithm.
Specific data for the exemplary neural network processor is provided in the following table.
For example, the first convolutional layer CV1 uses a 5×5 convolution kernel to compute a feature map having a resolution of 31×98 pixels for 24-dimensional (NF) local feature vector. The first fully connected layer FC1 has 1104 neural network processor elements to each compute a respective global feature.
The neural network processor elements in the various layers have a respective state register for storing a state value (X) indicative for their state.
The neural network processor elements may be provided as fully autonomous units that are capable of storing their own state and of updating that state in response to received input messages and optionally as a function of time. Such autonomous functionality may be provided by dedicated hardware, by suitably programmed general purpose processing elements, by suitably configured reconfigurable elements or by a combination thereof. Alternatively part of their functionality may be shared. For example each neural network processor element may comprise a proper memory location in a memory unit and a plurality of neural network processor elements may share a common processor to update their states in response to received input messages and optionally as a function of time. The shared processor may be provided as dedicated hardware, by a suitably programmed general purpose processor, by a suitably configured reconfigurable processor or by a combination thereof. By way of example neural network processor elements in a layer may share such a processor.
The neural network processor is arranged as a differential value message network, in that at least part of neural network processor elements are configured to transmit differential event messages indicative for a change of their state to neural network processor elements in a succeeding neural network processor layer, and only if the magnitude of the state change is sufficiently large.
X′=X+Wi*ΔO
i,
wherein Wi, is a weight for said input i. Alternatively other functions may be contemplated to compute the updated state value X′.
Subsequently in step S3 a quantized difference ΔXq is computed as the difference between the quantized value └X′┘ of the updated state value X′ and the quantized value └X┘ of the state value X prevailing at the time of receipt of the differential value message ΔOi. These variables will further be denoted respectively as quantized updated state value and quantized original state value. The updated state value and original state value may for example be represented as a 16-bit signed integer and these values may be quantized by keeping only a number of most significant bits, e.g. the 5 most significant bits.
Therewith the quantized original state value └X′┘ is determined as:
└X′┘=X′>>m, wherein m is the number of least significant bits that is ignored,
and
└X┘=x>>m,
Hence, ΔXq=└X′┘−└X┘
In step S4 it is determined whether the quantized difference ΔXq is different from zero. If this is not the case, the updated state value X′ is denoted as the original state value X. If indeed the quantized difference ΔXq is different from zero, then in step S6 the activation function is applied to each of the quantized updated state value └X′┘ and the quantized original state value └X┘ and the difference value ΔO(t) is computed in step S7 to be transmitted in step S8 as a message Message ΔO(t). The value of the actuation function AF may for example be determined with look-up table indexed with the quantized difference ΔXq.
The binary state value Q is indicative for a polarity of a previous significant state change as will be explained in more detail below. The binary state value Q as well as the state value X may be initialized at an arbitrary value.
In the method of
If it is determined in sub-step S4A that the quantized difference ΔXq is non-zero, the updated binary state value Q′ is to indicate the polarity of the quantized difference ΔXq, for example, as is the case here, the value is set as Q′=1 in step S4b1 if the polarity is negative and the value is set as Q′=0 in step S4b2 if the polarity is positive. In sub-steps S4b1 or S4b2, whichever is executed in a particular case, the value of the modified quantized difference ΔX′q is determined, wherein:
ΔX′q=ΔXq+Q′−Q
Accordingly, in case of sub-step S4b1, this reduces in this example to:
ΔX′q=ΔXq+1−Q
And in case of sub-step S4b2, it reduces to:
ΔX′q=ΔXq−Q
Subsequent to a sub-step S4b1 it is determined in sub-step S4c1 whether or not the value of the modified quantized difference ΔX′q is zero. If that is the case, the procedure proceeds with step S5′. Analogously, subsequent to a sub-step S4b2 it is determined in sub-step S4c2 whether or not the value of the modified quantized difference ΔX′q is zero. Also in that case, the procedure proceeds with step S5′.
In case it is determined in one of sub-steps S4c1, S4c2 that the value of the modified quantized difference ΔX′q is non-zero, the procedure continues with a step S6′ which differs from step S6 of
In sub-step S6a1 a modified quantized state value Xq is computed from the quantized state value └X┘ and the binary state value Q as:
X
q
=└X┘+Q
In sub-step S6a2, which may be executed in parallel with sub-step S6a1, the modified quantized updated state value X′q is computed from the quantized updated state value X′q and the updated binary state value Q′ as:
X′
q
=└X┘+Q′.
Then in sub-step S6b1 the first actuation function value O is computed on the basis of the quantized state value └X┘. In particular it is computed as the value of the actuation function AF(•) for the modified quantized state value X′q i.e. the quantized state value └X┘ modified by addition of the binary state value Q. Analogously, in sub-step S6b2, which may be executed in parallel with sub-step S6b1, the second actuation function value O′ is computed on the basis of the quantized updated state value └X′┘. In particular it is computed as the value of the actuation function AF(•) for the modified quantized state value X′q i.e. the quantized updated state value └X′┘ modified by addition of the updated binary state value Q′.
The actuation function is typically a monotonous bilateral saturating function such as a sigmoid function.
In step S7 corresponding to step S7 of the procedure of
ΔOo=O′−O
In step S8, corresponding to step S8 in
An exemplary operation of the improved method as disclosed herein is now illustrated and described and compared with the method according to
It is presumed that the processor element has assumed a state X′=1.6 and a value Q′ at point in time 1. Responsive to an input event message at point in time 2, the state value of the processor element increases to 2.4. Accordingly, the quantized difference ΔXq indicating the difference between the quantized updated value └X′┘ and the quantized state value └X┘ of the recipient neural network processor element equals 1. Accordingly, in step S4b2, the value Q′ is set to 0 and the modified quantized difference ΔX′q is set to 1. Therewith the procedure continues with steps S6′, S7 and S8 to emit an output message ΔO*(2) conveying the value AF(X′q(2))−AF(Xq(2)). The “+” sign indicates a message indicative for a positive transition. The procedure of
Subsequently, at point in time 3, the processor element receives a new input event message as a result of which its state value increases from X=2.4 to X′=3.2. Again, the quantized difference ΔXq indicating the difference between the quantized updated value └X′┘ and the quantized state value └X┘ of the recipient neural network processor element equals 1. Accordingly, in step S4b2, the value Q′ is again set to 0 and the modified quantized difference ΔX′q is again set to 1. Therewith the procedure continues with steps S6′, S7 and S8 to emit an output message ΔO*(3) conveying the value AF(X′q(3))−AF(Xq(3)). The procedure of
At point in time 4, when the processor element receiving a new input event message its state value increases from X=3.2 to X′=3.6. It is determined in step S4a, that the quantized difference ΔXq indicating the difference between the quantized updated value └X′┘ and the quantized state value └X┘ of the recipient neural network processor element equals 0. Consequently, in step S4b3, the updated binary state value Q′ assumes the value of the previous binary state value Q, so that effectively, upon proceeding with S5′ the previous binary state value Q is not changed. As indicated by “NA”, no output message is issued. Also, no output message would be issued by the method of
At point in time 5, the processor element, receiving a new input event message, decreases its state value from X=3.6 to X′=2.8. This corresponds to a quantized difference ΔXq having a value −1. Accordingly, the procedure of
Subsequently, at point in time 6, the processor element receives a new input event message as a result of which its state value increases from X=2.8 to X′=3.2. Again, the quantized difference ΔXq indicating the difference between the quantized updated value └X′┘ and the quantized state value └X┘ of the recipient neural network processor element equals 1. Accordingly, the procedure of
ΔX′q=ΔXq−Q=1−1=0.
Hence, it is again avoided that an output message is issued. Instead, the procedure directly continues with step S5′, where the previous state value X and the previous binary state value Q are set equal to the updated state value X′ and the updated binary state value Q′ respectively. Hence, the previous binary state value now equals 0.
When comparing
The neural network processor element 1 has a respective state register 2 for storing a state value X indicative for its state. The neural network processor element 1 is capable of transmitting differential event messages indicative for a change of its state to neural network processor elements in a succeeding neural network processor layer in accordance with an activation function AFN. It is configured to compute upon receipt of an input message a new state value X′ which is stored in temporary storage register 3. The neural network processor element comprises an additional state register 4 for storing a value Q of a state value change indicator that is indicative for a direction of a previous state change exceeding a threshold value. As set out in more detail below, the neural network processor element 1 selectively generates an output event message ΔOo dependent on the change of its state value and further dependent on the value of state value change indicator.
In the exemplary embodiment the input message conveys an input value ΔOi, and the neural network processor element 1 uses multiplier 5 and adder 6 to compute the new state value X′ as:
X′=X+Wi*ΔOi
Quantization element 7 provides the quantized original state value └X┘ of the original state value X as:
└X┘=MSB(m,n,X)
In other words quantization element 7 selects the m most significant bits of the n bits representing the original state value X and passes the quantized value to comparator 9. Analogously, Quantization element 8 provides the quantized updated state value └X′┘ of the updated state value X′ as:
└X′┘=MSB(m,n,X′), and passes the quantized value to comparator 9.
The comparator 9 issues an output signal ΔXq which controls a selection element 10 that selects the value for updated binary state variable Q′ in storage element 11. The selection element 10 is controlled by the signal ΔXq, so that it selects the input for storage element 11 as follows:
Q′=1 if └X′┘−└X┘≤−1
Q′=Q if −1<└X′┘−└X┘<1
Q′=0 if └X′┘−└X┘≥1
Adder 12 computes a modified quantized original state value is Xq computed as:
Xq=└X┘+Q
Likewise Adder 13 computes a modified quantized updated state value is X′q computed as:
X′q=└X′┘+Q′
Comparator 14 compares these values and issues a control signal ΔX′q to control an output stage 15 with activation function elements 16, 17 and subtraction element that compute a differential output value ΔOo to be conveyed by an output message as:
ΔOo=AFN(X′q)−AFN(Xq), wherein AFN is an activation function, for example the ReLu function.
The output stage 15 is only activated by the control signal ΔX′q if the modified quantized updated state value Xq differs from the modified quantized original state value Xq.
Furthermore, the elements 12, 13, 14 are only activated if comparator 9 determines that either └X′┘−└X┘≤−1 or that └X′┘−└X┘≥1. If these elements are not activated than also the output stage is not activated.
Once processing is completed, i.e. if it is determined that no output message will be issued, or upon completion of issuing the output message, the value for the updated state value X′ is stored in the original state value register 2 via gate 19. Likewise, the value for the updated binary state value Q′ is stored in the original binary state value register 4 via gate 20.
In an embodiment, neural network processor elements 1 in a neural network layer of a neural network processor may be provided as fully autonomous units that are capable of storing their own state and to update that state in response to received input messages. Alternatively, all or part of their functionality may be shared. For example each neural network processor element may comprise a proper memory location in a shared memory unit to store its state X and its binary state Q, and a plurality of message based processor elements may share a common processor to update their states X+Q in response to received input messages.
An example thereof is shown in
The incoming messages are received in a message buffer 110 and provided as internal messages EVMj to individual neural network processor elements j in a core 130. In the embodiment shown the core 130 comprises a memory unit 132, wherein each addressable location 10_1, . . . , 10_j, . . . , 10_n, comprises a state Vn,j of a neural network processor element j at point in time n. The state Vn,j includes the state value X and the binary state value Q referred to above. The core 130 further comprises a shared computation unit 134 which is provided to update a state Vn,j stored in a location 10_j to updated state Vn+1,j in response to an input message EVM j and to control an output unit 120, to selectively generate an output event message EVMout conveying a difference ΔOo dependent on the change of state, dependent both the state value X and the binary state value Q as a result of said updating. The output unit 120 properly addresses the message to be transmitted by the message exchange network to one or more recipients, e.g. neural network processor elements in a subsequent neural network layer.
A neural network processor, e.g. the neural network processor 1000 shown in
It is noted that a shared computation unit 134 may have various embodiments, for example an embodiment in the form of dedicated hardware, for example an element as shown in
| Number | Date | Country | Kind |
|---|---|---|---|
| 19217091.8 | Dec 2019 | EP | regional |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/EP2020/086846 | 12/17/2020 | WO |
