Patent Application
20150120631
The present invention relates to the field of electronic circuits, particularly integrated pulsed-processing circuits that emulate the neurological processing of biological neurons.
In computational neuroscience, algorithms that mimic neural processing to perform some type of high-level function, such as the automatic object recognition in computer vision are studied. In these systems it is very common to use dynamic synapses. By using dynamic synapses, the effect that a nervous impulse from one neuron has on another is such that the internal status of the latter evolves in a manner similar to that indicated in
Each input pulse (14, 15 and 16) alters the status of the destination neuron, such that an initial area that increases to a certain speed is added, followed by a discharge at a slower speed. For example, the first input pulse (14) causes a transition (17) in the status of the neuron. The contributions of the successive pulses (15 and 16) added the corresponding contributions (18 and 19), giving rise to a total contribution (20) that is the sum of the parts (17, 18 and 19). Therefore, the impulse effect continues for a period of a few milliseconds. This is quite conventional in the current status of computational neuroscience, as described in: W. Gerstner, “Spiking Neurons,” Ch 1 in Pulsed Neural Networks, W. Maass and C M Bishop (Eds.), MIT Press, Cambridge Mass., 1999.
When neural network-based computer vision systems are designed for object recognition, hierarchical structures are commonly used, such as for example that shown in
In these networks, the vision sensor (21) sends information to the first processing layer (24). This first layer extracts very simple features from small areas of the field of vision. For example, it is very usual to use an array of Gabor filters, each of which identifies short “segments” at different orientations and scales. Therefore, each of the filters (22) of the first layer would activate those pixels where a small segment (of several pixels in length and thickness, depending on the scale) having a certain orientation is centred. These filters (such as 22), present in all the layers, are referred to as “feature map”. Each neuron in a feature map sends its pulses to “projective fields” (23) in feature maps of the destination layers. Thus, the information is transmitted to the feature maps of the next layer, which groups segments to identify “traces”. The next layer (26) groups “traces” to form and recognise more complex figures, and so on until recognising objects, regardless of their size and orientation.
In a visual sensing and pulsed-processing system, such as biological systems, each pixel (or neuron) in a layer receives information from neighborhoods of the preceding layer. This is illustrated in
For example, if the letter “A” is present at input, the pixels that detect oriented segments (35A, 35B, 35C) are activated in the first layer (32). The second layer (33) would identify, for example, characteristic intersections between segments. And the third layer (34) would group the latter to identify the letter “A” or rule out others by sending negative or inhibitory pulses. Throughout this process, each pixel (or neuron) in a layer receives successive pulses, both positive and negative, from various neighborhoods of the preceding layers and must decide whether or not to identify the presence of the feature that said pixel (neuron) defines. Thus, if the letter “A” is presented as a flash, the visual sensor sends a small number of electrical impulses from the pixels that detected the A. The pixels (neurons) of the following layers will receive positive and negative pulses from neighborhoods of pixels from the preceding layer that determine whether the feature to be detected by said neuron is present or not. For a neuron to determine whether the feature it represents is present or not, it must fulfil the contribution of a sure number of positive and negative pulses of the preceding layer. In order to prevent the first pulses arriving at the neuron from inducing the neuron to make an erroneous decision, nature uses the method described in
This type of mathematical modelling is commonly used among computational neuroscience researchers who develop software programs that emulate neuronal processes, such as W. Gerstner, “Spiking Neurons,” Ch 1 in Pulsed Neural Networks, W. Maass and CM Bishop (Eds.), MIT Press, Cambridge Mass., 1999; Masquelier, T., and Thorpe, S J (2007). Unsupervised learning of visual features through spike timing-dependent plasticity. PLoS Comput. Biol 3, e31.Masquelier, T., Guyonneau, R., and Thorpe, S J (2008). Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains. PLoS ONE 3 e1377 and, Masquelier, T., and Thorpe, S J (2010). “Learning to recognise objects using waves of spikes and spike timing-dependent plasticity,” in Proc. of the 2010 IEEE Int Joint Conf on Neural Networks, Barcelona.
However, in embodiments of hardware circuits it is very complex and expensive to create neural circuits with dynamic synapses that respond in the manner indicated in
In the event of using “Convolutional Neural Networks” (as in
a) shows the evolution over time of the status of the neuron when it receives positive and negative input pulses (performing instantaneous synaptic integration) until reaching the threshold and sending its own output pulse. Since each neuron must detect whether it receives a series of features within a time window, it is desirable to provide it with a mechanism to download its status, as shown in
However, the method shown in
Therefore, the technical problem posed in the state of the art is to find a way to make contributions as if dynamic synapses were used, but using the simplest possible neural circuits, as in the case of instantaneous integration synapses. Therefore, each neuron would be able to make the correct decision and would allow proper recognition by the neural network.
The present invention solves the technical problem addressed on the basis of
Consequently, dynamic synapses behaviour (non-instantaneous) is thus emulated and the neurons do not make a hasty decision based on the first arriving pulses. On the contrary, by allowing the interleaving of weaker pulse trains, it is possible to agree on the contribution from a larger number of neurons that send their impulses during the same milliseconds. The result achieved is the emulation of the biological behaviour of the dynamic synapses described in
The present invention solves the technical problem addressed by distributing the effect of the instantaneous contribution of an input pulse to an artificial neuron over a longer time interval.
To this end, in a first aspect of the present invention, an Event Scheduler block for converting a pulsed-processing neural network with instantaneous synapses into dynamic integration synapses is described. As mentioned earlier, there are neural networks in the prior art consisting of an Event Router block and a Neuron Array block interconnected to form an instantaneous synaptic integration device, wherein the Event Router receives a stream of pulses or events at input events entering a stream of pulses or events Ei from a source module.
The novel Event Scheduler block is disposed between the source module and the Event Router block. Depending on the embodiment of the invention, the Event Scheduler block may be included within the feature map extractor (comprised in the instantaneous synaptic integration device) or constitute a separate entity. In a preferred embodiment, the novel Event Scheduler block comprises: i) a memory module; ii) an Output Event Arbiter; iii) a finite state machine (FSM) comprising a pointer register, wherein the finite state machine writes the “p” parameters of the event to the memory module received in a “t” instant to be sent to the Output Event Arbiter in “r” future time instants; and iv) an Input Event Manager that directly sends the event to the Output Event Arbiter and, simultaneously, to the finite state machine, so that the Output Event Arbiter generates a signal arising from the arbitration [(i.e. when two events are simultaneous (that from input and that from reading the memory module), since they can only be sent to the next stage one by one, it is necessary to use an Arbiter to resolve the conflict. The Arbiter makes one wait while sending the other, and then sends the one waiting)] of each input event or pulse and the event or pulse retrieved from the finite state machine, the signal being sent to the Neuron Array block, where it is attenuated by a factor in the vicinity of “r”, via the Event Router block.
The Event Scheduler block of the present invention forms part of a processing system for Address Event Representation (AER). The converter device of the present invention, as described in the preceding paragraph, is capable of repeating in the time “r” times the contribution of a neural impulse in event- or pulse-driven processing systems.
In the preferred embodiment of the invention, the Finite State Machine (FSM) comprises a pointer register whose index points to a memory module position, increasing the index by one position after an At time increment of time units.
Likewise, and for the aforementioned preferred embodiment of the invention, the memory module is a circular register of Q records comprising a predetermined number of Q positions, each of which comprises a “busy” bit and capacity for storing the “p” parameters of the event. The finite state machine, prior to writing the “p” parameters of the event in “r” positions of the memory module, each of which will be chosen to be read at a tt+tn instant, detects whether the “busy” bit is activated, in which case the “p” parameters of the event will be written in the next or previous memory module whose “busy” bit is deactivated; otherwise, the “p” event parameters are written in the position of the memory module that will be read at the t,+tn instant and the “busy” bit of that memory position is activated. The finite state machine for the “t” time instant reads the “p” parameters of the event comprised in the position of the memory module pointed to by the index of the pointer register if the “busy” bit is activated, deactivates said “busy” bit and sends said “p” parameters to the Output Event Arbiter.
The Event Scheduler block of the present invention has an effect on the weights of the events that arrive at the Neuron Array block. In order to compensate the effect of the Event Scheduler block of the present invention, it is necessary to reprogram the weights in the Neuron Array block so that the effect of replacing each original event by a finite number “r” of events is equivalent.
Therefore, a second aspect of the invention is a feature map extractor connectable at input to a source module from which it receives pulses or events Ei. The novel feature map extractor comprises a conventional Event block. The feature map extractor of the present invention is characterised in that it additionally comprises the Event Scheduler block of the present invention and a modified Neuron Array block. The Neuron Array block is rescheduled in order for the effect of replacing each original event by a finite number “r” of events is equivalent. That is, the weights are weakened such that the effect of replacing each original event by a finite number “r” from the Event Scheduler block is equivalent.
In a third aspect of the present invention, a method for converting pulsed-processing neural network with instantaneous integration synapses into dynamic integration synapses is described. Therefore, the method of the present invention allows dynamic integration synapses behaviour using circuits whose behaviour is instantaneous synapses. The method consists of repeating the contribution of a neural pulse over time in event- or pulse-driven processing systems, causing the contribution of repeated events to be weaker than the original.
The method of the present invention comprises, for each input pulse or event Ei received in an instant “t” in an Address Event Representation (AER) processing system, repeating each pulse or event Ei in the future “r” instants while the effect of each pulse or event Ei is attenuated in the destination module (destination block).
The method for converting a pulsed-processing neural network with instantaneous integration synapses into dynamic integration synapses of the present invention comprises the following steps:
i) for each input event or pulse received at an instant “t”, store the values of their “p” parameters associated with each event or pulse to be retrieved in a finite number “r” of future instants t,+tn (n=1, 2, . . . r);
ii) for each input event or pulse received, additionally send the event or pulse to its destination (regardless of the storage process of the preceding step);
iii) the parameters of the event or pulse received to “r” positions of a memory module; the positions of this memory module are read one by one at a constant speed; the “r” positions in the memory module will be selected depending on the future instants ti+tn in which the “r” repetitions of the event or pulse are to be retrieved;
iv) retrieve, in a time-controlled manner, the “p” parameters of the event or pulse in the finite number “r” of future instants, wherein the time intervals are fixed or variable;
v) extract the event in one of the future “r” instants (where applicable), arbitrate [(i.e. when two events are simultaneous (that from input and that from reading the memory module), as they can only be sent to the following stage one by one, it is necessary to use an Arbiter to resolve the conflict. The Arbiter makes one of the events wait while it sends the other and then sends the waiting event)], by means of an Arbiter, the dispatch of the event retrieved from the memory by sending a new event being received (if it is being received) at the same instant of time ti; and
vi) weakening the weights in the destination module, reprogramming it with the values of the weakened weights, so that the effect of replacing each original event by a finite number “r” of events is equivalent.
In the preferred embodiment of the invention relating to the method and in accordance with the preferred embodiment of the invention relating to the system formed by the Event Scheduler and Feature Map Extractor, both of the present invention, each input pulse or event Ei received in an instant “t” in an Address Event Representation (AER) processing system is repeated in “r” future instants while the effect of said pulse or event Ei is attenuated by the Neuron Array. In step i), the Input Event Manager receives input pulses or events Ei, each of which arrives in its time t, instant. In step ii), each input pulse or event Ei received in its arrival instant t is sent directly to the Output Event Arbiter which, upon arbitrating it with the stored pulses or events that can be sent by the FSM, will send it to the Output Event Arbiter output. In step iii), for each input event E, the finite state machine (FSM) will write its “p” parameters to the “r” memory positions of the memory module to be read in future instants t,+tn (n=1, 2, . . . r). If a memory location has its “busy” bit activated, the event will be written in the nearest previous or next position whose “busy” bit is deactivated. In step iv), the FSM reads, with time steps At, the consecutive positions of the memory module. If its “busy” bit is activated, it is deactivated, the “p” parameters of the stored event are extracted (and optionally deleted) and sent to the Output Event Arbiter. In step v), the Output Event Arbiter will arbitrate the events from the FSM with the events from the Input Event Manager that may coincide in time. In step vi), in the destination module, which is the Neuron Array block, the weights must be attenuated by a factor in the vicinity of “r”. This is performed by reprogramming the weights comprised in the Neuron Array.
In a fourth aspect of the invention, the neuron status discharge mechanism can be suppressed until reaching a resting state, if an additional “r2” repetitions subsequent to the event are added to the train of “r” repetitions of the input event by changing the polarity of the event in said “r2” repetitions. To this end, the Event Scheduler of the present invention comprises that the finite state machine “FSM” (4) writes the “p” parameters of the event (changing its polarity) to the memory module, to be sent to the Output Event Arbiter in future “r2” time instants subsequent to the “r” time instants. This is detailed in
b) shows a discharge mechanism of the status of a neuron to detect whether a neuron receives a series of features within a time window.
FIG. 6Bi) shows the evolution of the status of a neuron receiving a pulse through dynamic synapses.
FIG. 6Bii) shows the method indicated thus far wherein in one embodiment with instantaneous synapses and neurons with discharge mechanism, each input pulse is replaced by a train of weaker “r” pulses.
FIG. 6Biii) shows a neuron that has no mechanism with instant download synapses, wherein the input pulse is replaced by two consecutive trains.
The present invention falls within the scope of an event-driven sensing and processing system such as that shown in
The event-driven sensing and processing system shown in
Contrarily, the present invention inserts a block in each AER processor module, which we will call Event Scheduler (1), between the AER input channel and the block that distributes the events to the Event Router neuron array (12). Therefore, a new feature map extractor (11b) is obtained, comprising the Event Scheduler (1) of the present invention, a conventional Event Router (12) and a Neuron Array block (13) modified to offset the effect of Event Scheduler on the weights of the events. The function of the Event Scheduler (1) is as follows: for each input event Ei received at an instant ti, the Event Scheduler (1) repeats the same event in “r” future instants t,+tn with n=1, 2, . . . r. The present invention adds new functionality to Neuron Array block (13), that of readapting the original weights dij (i.e. attenuation), because now every input event is mapped to r+1 events destined to each of the neurons of the projective fields.
The finite state machine (4) contains a “pointer” register (3) whose index points to a position of the memory module (5). The index increases continuously every time step Δï to point to the next position of the memory module (5). In the memory module (5), each memory position contains a “busy” bit (6) plus other bits to house the parameters inherent to a given event Ei. Whenever the Input Event Manager (7) receives a new event Ei at an instant ti, it sends it to the Output Event Arbiter (2) to send it to the output, but additionally the FSM (4) copies its parameters to “r” positions of the memory module (5) so that the “pointer” (3) read the positions in future instants ti+tn with n=1, 2, . . . r. Likewise, in these positions the “busy” bit (6) is activated. If a memory position to which we wish to write an event Ei is already busy, the event Ei will be written to the next (or previous) available position whose “busy” bit is not activated.
At the same time, the FSM (4) reads the register to which the “pointer” points (3) and, if its “busy” bit is activated, reads the event Ei, deactivates the “busy” bit (6), and sends the parameters Ei to the Output Event Arbiter (2).
The number Q of memory positions of the memory module (5) must be adequately dimensioned in accordance with the “r” parameter, the maximum rate of input events pmax, the time step At and the maximum future instant tn. Thus, as at each moment the memory module of Q positions must store information until a future time tn, it must be fulfilled that: Q▪ At=tn. Furthermore, if for a time tn events were to reach the maximum rate pmax, the register would be able to contain pmax r−tn events. Therefore, it must be fulfilled that Q>pmax−r−tn. Or, equivalently, it must be verified that pmax−r−At≦1.
The detail of the method described above with the components described in the embodiment of
i) for each input event E (8) that is received in the Input Event Manager (7) at an instant “t”, the “p” values of their parameters are stored in a memory module (for example, a circular register of Q records in
ii) whenever an input event E is received (8), additionally send the event to your destination, i.e. the Output Event Arbiter (2) (regardless of the storage process of the next step) which, upon arbitrating it with the stored events that can be sent by the FSM (4), it will send it to the Event Router block (12) through the Output Event Arbiter (2) output (9);
the “p” parameters of the input event E (8) are written in “r” positions of a memory module by the FSM (4); the positions of this memory module are read one by one at a constant speed, depending on the future instants t,+pn (n=I, 2, . . . r) in which you we wish to retrieve the “r” repetitions of the event and must therefore have to select the “r” positions in the memory module. If the “busy” bit of a memory position is activated, the event will be written in the nearest previous or subsequent position whose “busy” bit is deactivated. The “busy” bit of the memory position where the event is written will become activated;
read the positions of the memory module one by one and, in those where one of the “r” repetitions of an event is stored, retrieve its “p” parameters. The FSM (4) reads the consecutive positions of the memory module with time steps At. If their “busy” bit is activated, the “p” parameters of the stored event are extracted and sent to the Arbiter (2). Whenever one of the “r” repetitions of an event stored in the past is extracted, the Arbiter (2) will arbitrate the events from the FSM (4) with those from the Input Event Manager (7) that coincide in time; and
weaken the weights in the destination module, reprogramming it with the values of the weakened weights, so that the effect of replacing each original event by a finite number “r” of events is equivalent. The destination module is the Neuron Array (13), which attenuates the weights by a factor in the vicinity of “r”.
| Number | Date | Country | Kind |
|---|---|---|---|
| P201230702 | May 2012 | ES | national |
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/ES2013/070285 | 5/7/2013 | WO | 00 |
