Patent Application
20230128057
This application claims priority to German Application 10 2021 127 695.0, filed on Oct. 25, 2021. The content of the above-referenced Patent Application is hereby incorporated by reference in its entirety.
Various embodiments generally relate to artificial neural networks.
Currently, many artificial neural networks (ANNs) include dense layers, including many nodes or neurons. For example, AlexNet is one notable neural network used for classification that includes two dense hidden layers, each including 4096 neurons (e.g., 4096×4096 neurons from layer n−1 to layer n).
In the drawings, like reference characters generally refer to the same parts throughout the different views. The drawings are not necessarily to scale; emphasis is instead generally being placed upon illustrating the principles of the invention. In the following description, various embodiments of the invention are described with reference to the following drawings, in which:
The following detailed description refers to the accompanying drawings that show, by way of illustration, specific details and embodiments in which the invention may be practiced.
The word “exemplary” is used herein to mean “serving as an example, instance, or illustration.” Any embodiment or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments or designs.
The words “plurality” and “multiple” in the description or the claims expressly refer to a quantity greater than one. The terms “group (of)”, “set [of]”, “collection (of)”, “series (of)”, “sequence (of)”, “grouping (of)”, etc., and the like in the description or in the claims refer to a quantity equal to or greater than one, i.e., one or more. Any term expressed in the plural form that does not expressly state “plurality” or “multiple” likewise refers to a quantity equal to or greater than one. The terms “proper subset”, “reduced subset”, and “lesser subset” refer to a subset of a set that is not equal to the set, i.e., a subset of a set that contains fewer elements than the set.
The terms “at least one” and “one or more” may be understood to include a numerical quantity greater than or equal to one (e.g., one, two, three, four, [ . . . ], etc.).
As used herein, unless otherwise specified, the use of the ordinal adjectives “first”, “second”, “third”, etc., to describe a common object merely indicate that different instances of like objects are being referred to, and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.
The term “data” as used herein may be understood to include information in any suitable analog or digital form, e.g., provided as a file, a portion of a file, a set of files, a signal or stream, a portion of a signal or stream, a set of signals or streams, and the like. Further, the term “data” may also be used to mean a reference to information, e.g., in the form of a pointer. However, the term data is not limited to the aforementioned examples and may take various forms and represent any information as understood in the art.
The term “processor” or “controller” as, for example, used herein may be understood as any kind of entity that allows handling data, signals, etc. The data, signals, etc., may be handled according to one or more specific functions executed by the processor or controller.
A processor or a controller may thus be or include an analog circuit, digital circuit, mixed-signal circuit, logic circuit, processor, microprocessor, Central Processing Unit (CPU), Neuromorphic Computer Unit (NCU), Graphics Processing Unit (GPU), Digital Signal Processor (DSP), Field Programmable Gate Array (FPGA), integrated circuit, Application Specific Integrated Circuit (ASIC), etc., or any combination thereof. Any other kind of implementation of the respective functions, which will be described below in further detail, may also be understood as a processor, controller, or logic circuit. It is understood that any two (or more) of the processors, controllers, or logic circuits detailed herein may be realized as a single entity with equivalent functionality or the like, and conversely that any single processor, controller, or logic circuit detailed herein may be realized as two (or more) separate entities with equivalent functionality or the like.
A “circuit” as used herein is understood as any kind of logic-implementing entity, which may include special-purpose hardware or a processor executing software. A circuit may thus be an analog circuit, digital circuit, mixed-signal circuit, logic circuit, processor, microprocessor, signal processor, Central Processing Unit (“CPU”), Graphics Processing Unit (“GPU”), Neuromorphic Computer Unit (NCU), Digital Signal Processor (“DSP”), Field Programmable Gate Array (“FPGA”), integrated circuit, Application Specific Integrated Circuit (“ASIC”), etc., or any combination thereof. Any other kind of implementation of the respective functions, which will be described below in further detail, may also be understood as a “circuit.” It is understood that any two (or more) of the circuits detailed herein may be realized as a single circuit with substantially equivalent functionality. Conversely, any single circuit detailed herein may be realized as two (or more) separate circuits with substantially equivalent functionality. Additionally, references to a “circuit” may refer to two or more circuits that collectively form a single circuit.
As utilized herein, terms “module”, “component,” “system,” “circuit,” “element,” “interface,” “slice,” “circuitry,” and the like are intended to refer to a set of one or more electronic components, a computer-related entity, hardware, software (e.g., in execution), and/or firmware. For example, circuitry or a similar term can be a processor, a process running on a processor, a controller, an object, an executable program, a storage device, and/or a computer with a processing device. By way of illustration, an application running on a server and the server can also be circuitry. One or more circuits can reside within the same circuitry, and circuitry can be localized on one computer and/or distributed between two or more computers. A set of elements or a set of other circuits can be described herein, in which the term “set” can be interpreted as “one or more.”
As used herein, a “signal” may be transmitted or conducted through a signal chain in which the signal is processed to change characteristics such as phase, amplitude, frequency, and so on. The signal may be referred to as the same signal even as such characteristics are adapted. In general, so long as a signal continues to encode the same information, the signal may be considered as the same signal.
As used herein, a signal that is “indicative of” a value or other information may be a digital or analog signal that encodes or otherwise communicates the value or other information in a manner that can be decoded by and/or cause a responsive action in a component receiving the signal. The signal may be stored or buffered in a computer-readable storage medium before its receipt by the receiving component. The receiving component may retrieve the signal from the storage medium. Further, a “value” that is “indicative of” some quantity, state, or parameter may be physically embodied as a digital signal, an analog signal, or stored bits that encode or otherwise communicate the value.
It will be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be physically connected or coupled to the other element such that current and/or electromagnetic radiation (e.g., a signal) can flow along a conductive path formed by the elements. Intervening conductive, inductive, or capacitive elements may be present between the element and the other element when the elements are described as being coupled or connected to one another. Further, when coupled or connected to one another, one element may be capable of inducing a voltage or current flow or propagation of an electromagnetic wave in the other element without physical contact or intervening components. Further, when a voltage, current, or signal is referred to as being “applied” to an element, the voltage, current, or signal may be conducted to the element by way of a physical connection or by way of capacitive, electromagnetic, or inductive coupling that does not involve a physical connection.
As used herein, “memory” is understood as a non-transitory computer-readable medium where data or information can be stored for retrieval. References to “memory” included herein may thus be understood as referring to volatile or non-volatile memory, including random access memory (RAM), read-only memory (ROM), flash memory, solid-state storage, magnetic tape, hard disk drive, optical drive, etc., or any combination thereof. Furthermore, registers, shift registers, processor registers, data buffers, etc., are also embraced herein by the term memory. A single component referred to as “memory” or “a memory” may be composed of more than one different type of memory and thus may refer to a collective component comprising one or more types of memory. Any single memory component may be separated into multiple collectively equivalent memory components and vice versa. Furthermore, while memory may be depicted as separate from one or more other components (such as in the drawings), memory may also be integrated with other components, such as on a common integrated chip or a controller with an embedded memory.
The term “software” refers to any type of executable instruction, including firmware.
Unless specifically stated otherwise, discussions herein using words such as “processing,” “computing,” “calculating,” “determining,” “presenting,” “displaying,” or the like may refer to actions or processes of a machine (e.g., a computer/processor/etc.) that manipulates or transforms data represented as physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or a combination thereof), registers, or other machine components that receive, store, transmit, or display information.
Exemplary embodiments of the present disclosure may be realized by one or more computers (or computing devices) reading out and executing computer-executable instructions recorded on a storage medium (e.g., non-transitory computer-readable storage medium) to perform the functions of one or more of the herein-described embodiment(s) of the disclosure. The computer(s) may comprise one or more of a central processing unit (CPU), a microprocessing unit (MPU), or other circuitry, and may include a network of separate computers or separate computer processors. The computer-executable instructions may be provided to the computer, for example, from a network or a non-volatile computer-readable storage medium. The storage medium may include, for example, one or more of a hard disk, a random-access memory (RAM), a read-only memory (ROM), a storage of distributed computing systems, an optical drive (such as a compact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD), a flash memory device, a memory card, and the like. By way of illustration, specific details and embodiments in which the invention may be practiced.
As used herein, unless otherwise specified the use of the ordinal adjectives “first”, “second”, “third” etc., to describe a common object, merely indicate that different instances of like objects are being referred to, and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.
Presently, the execution of neural networks, such as the trained neural network 100, includes performing operations or calculations involving “zero multiplications”. The execution can include multiplication involving values that are or are close to zero value and thus do not impact the result of the trained neural network and thus are ultimately unnecessary. Further, performing such calculations or operations results in unnecessary data movement between memories. As the neural networks get larger or more complex, e.g., more interconnected neurons, this leads to more unnecessary calculations and a reduction in energy-efficiency, among other things.
Accordingly, neural networks can be characterized by sparsity, which defines the amount of inclusion of zeros or very small value of weights and (hidden-layer) neurons that do not impact the final calculation of a trained neural network. Existing solutions for exploiting or reducing the sparsity of a trained neural network include removing redundant parameters, such as, for example, by removing weights or certain layers. This pruning or paring of the neural network can decrease the memory footprint of a neural network and help avoid extra calculations during inference.
The disadvantages of this approach are that it requires estimations and assumptions as to the weights and hidden layers that would be sparse or involve zero or near zero calculations. Moreover, pruning is static because it is applied to all inputs applied to the neural network, regardless of the pruned weights or hidden layers would qualify as sparse.
The section 300 of the neural network includes a hidden layer h which includes nodes h1-h4. The nodes h1-h4 are interconnected or related by weighted connections or weights w to the nodes ha1-ha3 of the next hidden layer ha. The data indicating or reflecting the relationship between nodes, e.g., the values of nodes, the weights, etc. may be stored or maintained in a suitable memory or storage device.
The matrix equation 350 mathematically represents the relationship between the hidden layer h and the hidden layer ha using the weights w, w11-w43. As shown, the values of nodes of the hidden layer h can be represented as a vector that is to be multiplied by the weights, which is represented in a matrix form.
An activation function (Act Func) is applied to the outputs of the multiplication between the hidden layer node h (the node values) and the weights w (w11-w43) to produce node/neuron values. The activation function to be applied may be any suitable function for neural networks, including, for example, a sigmoid function or a Rectified Linear Unit (ReLU) function. Examples of the sigmoid function and the ReLU function are shown in
Referring back to
In exemplary embodiments of the present disclosure, additional information may be present and used to execute a trained neural network more efficiently and avoid many calculations. Data indications may be used to denote or convey the status of the nodes of hidden layers. In one case, the indications may be in mask bits, but other types of indications or designations may be used in other instances.
In various exemplary embodiments of the present disclosure, values of nodes determined during execution (inference) of a trained neural network can be compared to a threshold. For example, an indication, e.g., a mask bit value, can be assigned based on the comparison. As such, the determined mask bit value can indicate or inform nodes that have a node value large enough (e.g., larger than a threshold) to be needed or significant for further calculations in the execution of the trained neural network. Such nodes can be considered as “strong nodes”.
Moreover, a mask bit can indicate or inform nodes that have a “low” enough (e.g., lower than a threshold) value to be considered unnecessary to perform calculations with the node to execute the neural network. Such nodes may be considered as “weak nodes”. In some embodiments described herein, the weak nodes may not be used or bypassed in executing a neural network.
Specifically, in the example of
In the example of
As shown in
As shown in
The system 500 further includes one or more load/store units (LSU) 520, at least one control unit circuitry 530, one or more multiply and accumulate (MAC) units 540, and one or more activation circuitries 550. Connections, e.g., physical connections such as buses or other data lines, between the components may not be shown.
The LSU or vector LSU 520 can be configured to access, retrieve, and provide data between the components of the system 500. For example, the LSU 520 may retrieve or access the weights or node values, e.g., from memory 510, and provide them to other components of the system 500.
The control unit circuitry 530 may include hardware or processing circuitry, e.g., one or more processors programmed or configured to control one or more aspects of the system 500. In one instance, the control unit circuitry 530 can control which parameters or which operations are to be performed during the execution of a trained neural network. In
The MAC units 540 are each a computational component that performs various calculations or mathematical operations. Each MAC unit 540 can include one or more registers R for storing data received by or produced by the MAC units. Further, each of the MAC units 540 can include a neuron control component 530a that, when enabled, can allow the respective MAC unit 540 to perform calculations or operations for multiple neurons, e.g., during execution or implementation of a trained neuronal network or be triggered to only allow processing for a single node. In the examples herein, one or more of the MAC units 540 may perform calculations for more than one neuron at a time by obtaining weights for more than one node of the hidden layer to be determined.
Further, each MAC unit 540 can include the MAC components (MACs) that perform multiplication and accumulations based on inputs received by the respective MAC unit 540.
The activation circuitries 550 may perform operations related to applying an activation function to inputs, e.g., values calculated by the one or more MAC units 540. The activation circuitries 550 can also include one or more R registers for receiving and storing outputs. Each of the activation circuitries 550 include an activation component A. The activation component A may be a hardware component, such as processing circuitry, that applies an activation function to inputs, which in examples herein can be outputs of the MAC units 540. An output of the activation component A can be the node value for a particular node.
Further, the activation circuitry 550 can be configured to compare the output of the activation component A to a (predefined or predetermined) threshold value. This comparison can be used to producing or assigning indications, e.g., mask bits to nodes. Each of the activation circuitries 550 can include a comparator 550a that compares the output of the activation component A to a threshold T and produces an output, e.g., a mask bit Mb.
In the example of
As mentioned, memory 510 stores the neural network configuration data, including data indicating the relationships between nodes, including the weights of interconnected nodes. The w weights (w11-w43) connecting the hidden layer h and hidden layer ha, and the node values of the hidden layer h, h1-h4 are shown stored in the memory 510. Further, the memory 510 may store the mask bits M for the hidden layer h. The memory 510 does not yet have node values for the nodes ha1-ha3 of hidden layer ha. Hence, the trained neural network has been partially executed.
During the trained neural network implementation, the system 500 can be configured to avoid or skip certain computations. Referring back to
Therefore, for system 500 to determine the values of the nodes for the hidden layer ha, data indicating the state of the nodes (e.g., mask bits) are transferred or sent to the control unit circuitry 530. The control unit circuitry 530 can then read and react to the state of the nodes.
As shown starting in
After loading node value of h1 and weights w11 and w12, the MAC unit 540 can perform mathematical operations or calculations to determine the multiplicative product of h1 and w11 (for ha1) and the multiplicative product of h1 and w12 (for ha2). These results (multiplicative products) can each be stored, for example, in the registers R coupled to the output of the MAC components.
Subsequently, the control unit circuitry 530 can be configured further cause the LSU to load into the same MAC unit 540, the next parameters for determining the first and second nodes, ha1 and ha2. Before any loading command, the control unit circuitry 530 consider using the node (and its associated node value and associated weights). However, in this case, the control unit circuitry 530 can determine that the next node, h2, is weak because its mask bit value is “0”.
Therefore, instead of causing the node value of h2 and associated weights for the first and second nodes of the next hidden layer ha to be loaded, the control unit circuitry 530 proceeds to consider the next node h3 in the execution of the trained artificial neural network. In doing so, the control unit circuitry 530 effectively causes the system 500 to skip performing operations, determinations, or calculations involving the node h2. No operations involving the node h2 and its associated node value and weights are performed for layer ha. Accordingly,
Subsequently, in
In
The multiple neuron control component 530a of the control unit circuitry 530 can again enable (e.g., send a signal to enable) the multi-neuron functionality of the MAC unit 540. The MAC unit 540 determines the multiplicative product of h4 and w41 and h4 and w42, respectively, for the nodes ha1 and ha2. The products are respectively accumulated with the previously determined multiplicative products for ha1 and ha2. Each accumulation can be stored, e.g., in respective a register of the MAC unit 540 for further use or processing.
Since node h4 is the last node for layer h, accumulation produces the preliminary node values for ha1 and ha2. Thus, the control unit circuitry 530 can be configured to activate or prompt one of the activation circuitries 550 to obtain the calculated values for ha1 and ha2. The activation circuitry 550, using the activation component A, applies the activation function to the calculated node values to produce the respective node values for ha1 and ha2, which can be stored in registers R.
Further, the activation circuitry 550 can then apply the comparator circuitry or comparator 550a to each node's values determined by the activation component A. The comparator 550a can determine whether the node value generated by the activation component A is greater than a threshold and generate a corresponding indicator, e.g., a mask bit based on the comparison result.
For example, if the node value is greater or equal to the threshold T, a mask bit value (Mb) of “1” can be generated. If the node value is less than the threshold T, a mask bit value (Mb) of “0” can be generated.
The generated node values and the corresponding mask bits Mb from the activation circuitry 550 can be retrieved, e.g., by the LSU 520, and provided into the memory 510. They can be used to determine other nodes, e.g., of the next layer of the trained ANN.
Further, another MAC unit 540 may independently generate the node values for ha1 and ha2.
In
After loading the parameters (h1, w31, h3, and w33), the second MAC unit 540 can perform operations/calculations. The single neuron component 530b of the control unit circuitry 530 can enable the second MAC unit 540 to function in a single neuron mode. The product, h1 and w31, and the product h3 and w33 produced by the second MAC unit can be accumulated or summed together, in this case, in a single intermediate data, ha3_t1.
After generating the intermediate node data ha3_t1, the control unit circuitry 530 can cause the LSU 520 to retrieve and load into the second MAC unit 540 the parameters of the fourth node of the layer h, for producing the third node ha3 of the layer ha. Namely, the parameters of the fourth node h4 for determining the third node of the next layer ha (h4 and w43) are loaded into the second MAC unit as shown in
The second MAC unit 540, as shown, can produce or generate the product of h4 and w43, and the intermediate product ha3_t2. The second MAC unit 540 can accumulate the intermediate products ha3_t1 and ha3_t2 to produce a final node calculation for ha3.
After producing a node calculation for ha3 in the second MAC unit 540, the control unit circuitry 530 can trigger at least one activation circuitry 550 to operate. One activation circuitry 550 obtains the node calculation for ha3 and applies the activation function using the activation component A. The activation component A produces the node value for ha3. Further, the comparator 550a compares the node value ha3 to the threshold T to produce a status or group indication, e.g., a mask bit Mb, depending on the comparison result by the comparator 550a. The produced node value ha3 and the corresponding mask bit Mb can be sent, e.g., via the LSU 520, to the memory 510 for storage and later for further use in the execution of the trained ANN.
Then system 500 can continue to execute the trained neural network by determining the next hidden layer hb (not shown) based on the produced nodes values and mask bits determined for hidden layer ha. The same type of operations and actions can be performed again by the memory 510, the LSU 520, the control unit circuitry 530, the one or more MAC units 540, and the one or more activation circuitries 550 to determine the values of the next layer. In doing so, calculations or operations nodes have been determined to be in the group of weak nodes, e.g., have a mask bit value of “0” can be skipped or avoided. Instead, only calculations or operations involving strong nodes, e.g., having a mask bit value of “1” can be performed.
The following examples pertain to further aspects of this disclosure:
Example 1 is a system for executing a pre-trained artificial neural network including a plurality of interconnected nodes, wherein the system includes: at least one memory including weight values of the pre-trained neural network and configured to a store node value and a mask bit value for each of the plurality of nodes of the pre-trained neural network; one or more multiply and accumulate (MAC) units configured to perform operations for determining node values; a control unit circuitry configured, during execution of the neural network, to dynamically control operations of the one or more MAC units so as to cause a reduction in a number of calculations to be performed by the one or more MAC units including to cause the one or more MAC units to perform operations involving a subset of the plurality of nodes and to cause the one or more MAC units to avoid performing operations involving nodes of the plurality nodes that are outside of the subset that the respective MAC units would otherwise perform if the respective nodes were identified as being in the subset.
Example 2 is the subject matter of Example 1, wherein the control unit circuitry can be configured, during execution of the neural network, to identify nodes having node values within a predefined threshold range as belonging outside the subset to identify nodes having node values are outside the predefined threshold range as belonging in the subset.
Example 3 is the subject matter of Example 2, wherein the control unit circuitry configured to identify nodes having node values that are within the predefined threshold range as belonging outside of the subset or to identify nodes having node values that are outside the predefined threshold range as belonging in the subset can include the control unit circuitry being configured to generate a corresponding mask bit value to identifying nodes as belonging outside or in the subset.
Example 4 is the subject matter of Example 3, wherein the control unit circuitry to cause the one or more MAC units to avoid performing operations involving nodes of the plurality nodes that are outside of subset can further include the control unit circuitry to cause the one or more MAC units to avoid performing operations using the respective node values from nodes identified by the corresponding mask bit values as being outside of the subset.
Example 5 is the subject matter of Example 3 or 4, which can further include one or more activation circuitries configured to apply an activation function to respective outputs of the one or more MAC units.
Example 6 is the subject matter of Example 5, wherein the one or more activation circuitries can be further configured to compare respective outputs of the activation function to the predefined threshold range, and generate the mask bit values for nodes based the respective comparisons.
Example 7 is the subject matter of any of Examples 1 to 6, which can further include: one or more vector load/store units configured to transfer data between the at least one memory unit, the one or more MAC units, and/or the control unit circuitry, wherein during execution of the neural network, the control unit circuitry can be further configured to dynamically cause the vector load/store unit to retrieve from the at least one memory and load into the one or more MAC units, node values from and weights associated with the subset of nodes and configured to avoid causing the vector/load store unit to retrieve from the at least one memory and load into the one or more MAC units, node values from and weights associated with nodes outside the subset of nodes.
Example 8 is the subject matter of any of Examples 1 to 6, wherein the predefined threshold range can be a range including a value of zero.
Example 9 is the subject matter of any of Examples 1 to 8, wherein the trained artificial neural network can be a trained feedforward artificial neural network including a plurality of layers, the plurality of layer including an input layer, one or more hidden layers, and an output layer, wherein each layer comprises one or more of the plurality of nodes, and wherein the nodes of neighboring layers are related by one or more the weighted connections.
Example 10 is the subject matter of any of Examples 1 to 8, wherein the artificial neural network can be a convolutional neural network.
Example 11 is the subject matter of any of Examples 1 to 8, wherein the artificial neural network can be a recurrent neural network.
Example 1 A is a method for facilitating execution of a trained artificial neural network comprising a plurality of interconnected nodes, the method including: obtaining input data for the trained neural network; applying the input data to the trained neural network; and determining node values for the plurality of interconnected nodes. Determining the node values for the plurality of interconnected nodes includes identifying nodes respectively having previously determined node values greater than or equal to a predefined threshold value; and determining node values for nodes without a determined node value using only the nodes identified as respectively having determined node values greater than the predefined threshold value.
Example 2A is the subject matter of Example 1A, wherein determining node values for the plurality of interconnected nodes may further include identifying, nodes having respectively determined node values less than the predefined threshold value.
Example 3A is the subject matter of Example 1A or 2A, wherein determining node values of one or more nodes without any determined node values may further include performing, using one or more multiply-accumulate (MAC) units, one or more calculations using the node values and associated weight values from one or more of the nodes identified as having respectively determined node values greater than or equal to the predefined threshold value.
Example 4A is the subject matter of Example 3A, wherein determining node values of one or more nodes without any determined node values may further include causing the one or more MAC units to avoid performing any calculations using the node values and associated weight values of the nodes identified as having respectively determined node values less than the predefined threshold value.
Example 5A is the subject matter of Example 3A or 4A, wherein determining node values of one or more nodes without any determined node values using only the nodes identified as having respective determined node values greater than the predefined threshold value may further include loading into the one or more MAC units, at one or more instances, one or more node values and one or more weight values only for nodes identified as having respective determined node values greater than the predefined threshold.
Example 6A is the subject matter of any of Examples 1 A to 5A, which may further include determining an identifier for nodes having a determined node values, the identifier indicating whether the node has a respective node value greater than or equal to the predefined threshold or has a respective node value less than the predefined threshold.
Example 7A is the subject matter of Example 3A, wherein a control unit circuitry coupled to the one or MAC units may cause the one or more MAC units to perform one or more calculations using only the node values and associated weight values from the nodes identified as having respective determined node values greater than or equal to the predefined threshold value.
Example 8A is the subject matter of Example 3A, wherein determining node values of one or more nodes without any determined node values may further include applying an activation functions to one or more outputs of the one or MAC units.
Example 9A is the subject matter of any of Examples 1 A to 8A, wherein the artificial neural network can be a feedforward neural network including a plurality of layers, the plurality of layer including an input layer, one or more hidden layers, and an output layer, wherein each layer comprises one or more of the plurality of nodes, and wherein the nodes of neighboring layers are related by one or more the weighted connections.
Example 10A is the subject matter of any of Examples 1 A to 8A, wherein the artificial neural network can be a convolutional neural network.
Example 11A is the subject matter any of Examples 1 A to 8A, wherein the artificial neural network can be a recurrent neural network.
It should be noted that one or more of the features of any of the examples above may be suitably or appropriately combined with any one of the other examples or with embodiments disclosed herein.
The foregoing description has been given by way of example only and it will be appreciated by those skilled in the art that modifications may be made without departing from the broader spirit or scope of the invention as set forth in the claims. The specification and drawings are therefore to be regarded in an illustrative sense rather than a restrictive sense.
The scope of the disclosure is thus indicated by the appended claims and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced.
It is appreciated that implementations of methods detailed herein are demonstrative in nature, and are thus understood as capable of being implemented in a corresponding device. Likewise, it is appreciated that implementations of devices detailed herein are understood as capable of being implemented as a corresponding method. It is thus understood that a device corresponding to a method detailed herein may include one or more components configured to perform each aspect of the related method.
All acronyms defined in the above description additionally hold in all claims included herein.
| Number | Date | Country | Kind |
|---|---|---|---|
| 10 2021 127 695.0 | Oct 2021 | DE | national |
