Patent Application
20230214665
Aspects of the present invention generally relate to a neural network system for distributed neural network boosting for a programmable logic controller (PLC) with a plurality of processing units.
Artificial intelligence (AI) is set to play a key role in reducing the programming and engineering effort required to create automation solutions. The first important step for integrating AI into the automation environment is a Neural Processing Unit (NPU) module. A common challenge when working with an edge device such as a Technology Module (TM) NPU is its resource limitation. Processing speed, memory size, and storage can constrict the applicability of the device. The NPU specializes in computation of feedforward neural networks (NNs). It takes a tensor input and performs a series of tensor operations to calculate for the output. Using image detection as an example application, the input is an image while the outputs are bounding boxes and classifications of the objects detected. Such a large input space (number of pixels) and output space (number of possible bounding boxes and classes) usually require the NN model to have a high Vapnik-Chervonenkis (VC) dimension to effectively capture an underlying distribution in the training data.
To support a high VC dimension, the NN must have a large number of nodes and layers in its computational graph. However, there are three potential problems associated with having a large graph. First, the model can easily overfit to training data, rendering it less accurate and less generalizable. Secondly, the speed of the computation may not be enough for a real-time application due to the limit of processing speed. Lastly, an edge device may not have sufficient computation resource to even process the graph. The bottleneck is often the volatile memory capacity needed to store the graph at runtime. As a result, the maximum VC dimension of a deep neural network is often limited by the edge device’s available memory.
Presently, the NPU is limited to only running a single neural network to perform a specific application. This neural network model is often purposefully designed to have just the right number of VC dimension for the application. Therefore, scalability is not an option under the computational resource constraint. Using image detection as an example again, such limitation effectively means that the NN model can only identify up to a limited number of classes with limited accuracy. The Mobilenets is the current state-of-the-art pre-trained model that performs object detection in edge devices (i.e., mobile phone). It is trained on the ImageNet with up to 1000 different classes. It presents global hyperparameters for users to adjust the trade-off between accuracy and latency due to resource limitation.
Therefore, there is a need for a learning method which enables a PLC to use more than one NPU to scale an application.
Briefly described, aspects of the present invention relate to a distributed ensemble learning method which enables a PLC to use more than one Neutral Processing Unit (NPU) to scale an application. It is called Adaptive Boosting (AdaBoost). The high-level idea of this algorithm is to train multiple weak learners, which are then used to “vote” for a final hypothesis. Weak learners are machine learning models that do not capture the entire data distribution and are purposefully designed to predict with lower accuracy. In this case, neural networks are used as weak learners. The advantage of the invention is to leverage a distributed computing method and a machine learning boosting algorithm to scale applications by using more than one NPU. The image detection, used as an example, has high dimensional inputs and outputs. The art solution, Mobilenets, gives users the flexibility to modify a model’s size so it fits into the edge device’s resource limitation, but it is trading the model accuracy for smaller model size. Its accuracy drops when the model’s complexity is reduced. The proposed invention makes no such trade-off. The AdaBoost combines multiple distributed neural network models to create a more complex model without reaching the resource limitation.
In accordance with one illustrative embodiment of the present invention, a computer-implemented method of distributed neural network boosting performed by a neural network system is provided. The method comprises through operating at least one processor a step of providing a boosting algorithm that distributes a model among a plurality of processing units each being a weak learner of multiple weak learners that can perform computations independent from one another yet process data concurrently. The method further comprises enabling a distributed ensemble learning which enables a programmable logic controller (PLC) to use more than one of the processing units of the plurality of processing units to scale an application. The method further comprises training the multiple weak learners using the boosting algorithm. The multiple weak learners are machine learning models that do not capture an entire data distribution and are purposefully designed to predict with a lower accuracy. The method further comprises using the multiple weak learners to vote for a final hypothesis based on a feed forward computation of neural networks.
In accordance with another illustrative embodiment of the present invention, a neural network system is provided for distributed neural network boosting. The system comprises a processor and an accessible memory storing a neural program comprising software instructions that when executed by the processor are configured to provide a boosting algorithm that distributes a model among a plurality of processing units each being a weak learner of multiple weak learners that can perform computations independent from one another yet process data concurrently. The software instructions enable a distributed ensemble learning which enables a programmable logic controller (PLC) to use more than one of the processing units of the plurality of processing units to scale an application. The software instructions train the multiple weak learners using the boosting algorithm. The multiple weak learners are machine learning models that do not capture an entire data distribution and are purposefully designed to predict with a lower accuracy. The software instructions use the multiple weak learners to vote for a final hypothesis based on a feed forward computation of neural networks.
In accordance with another illustrative embodiment of the present invention, a non-transitory computer-readable storage medium encoded with executable instructions is provided. Instructions, when executable by at least one processor to operate one or more neural network systems. The instructions comprise providing a boosting algorithm that distributes a model among a plurality of processing units each being a weak learner of multiple weak learners that can perform computations independent from one another yet process data concurrently. The instructions comprise enabling a distributed ensemble learning which enables a programmable logic controller (PLC) to use more than one of the processing units of the plurality of processing units to scale an application. The instructions comprise training the multiple weak learners using the boosting algorithm. The multiple weak learners are machine learning models that do not capture an entire data distribution and are purposefully designed to predict with a lower accuracy. The instructions comprise using the multiple weak learners to vote for a final hypothesis based on a feed forward computation of neural networks.
To facilitate an understanding of embodiments, principles, and features of the present invention, they are explained hereinafter with reference to implementation in illustrative embodiments. In particular, they are described in the context of a neural network system for distributed neural network boosting for a programmable logic controller (PLC) with a plurality of processing units such as Neural Processing Units (NPUs). Distributed neural network boosting is provided in an NPU. An AdaBoost algorithm is used to boost multiple neural networks as weak learners, each of which can reside in a separate NPU so they can utilize resources independently and perform computation concurrently. With the AdaBoost training, each weak learner focuses on a resampled subset of the dataset. Combining all the outputs allows the overall model to expand the Vapnik-Chervonenkis (VC) dimension, effectively covering a larger underlying distribution of the data. Currently, the VC dimension of a model implemented in the NPU is limited by its computational resource. And such limitation directly equates to narrower applicability of the device as the model has limited capacity to learn a complicated distribution. The invention eliminates the limitation, enabling the NPU to scale the applications. Embodiments of the present invention, however, are not limited to use in the described devices or methods.
The components and materials described hereinafter as making up the various embodiments are intended to be illustrative and not restrictive. Many suitable components and materials that would perform the same or a similar function as the materials described herein are intended to be embraced within the scope of embodiments of the present invention.
These and other embodiments of a neural network system according to the present disclosure are described below with reference to
Consistent with one embodiment of the present invention,
The distributed neural network boosting is a novel approach to divide computations of a machine learning model into multiple computers or hardware devices. This approach leverages the boosting technique to train more than one model (aka weak learners), which will produce a final prediction altogether. A model is a machine learning model that can be trained to map input data to output prediction. A weak learner is a machine learning model that is purposely designed to be lightweight and simple. It is expected to learn only from the subset of the training data, so it won’t be a strong predictor by itself.
The neural network system 105 comprises a processor 112 and an accessible memory 115 storing a neural program 117 comprising software instructions 120 that when executed by the processor 112 are configured to provide a boosting algorithm 122 that distributes a model 125 among the plurality of processing units 110(1-x) each being a weak learner of multiple weak learners (WLs) 127(1-x) that can perform computations independent from one another yet process data concurrently.
The boosting algorithm 122 is one of ensemble algorithms in supervised learning that is used to reduce a bias 162 and variance 160. It involves sequentially training the weak learners (WLs) 127(1-x) with varying weight distributions in a sample dataset. And then all the trained weak learners together can produce the final prediction. The variance 160 in probability theory and statistics is the expectation of the squared deviation of a random variable from its mean. Informally, it measures how far a set of numbers are spread out from their average value. The bias 162 is a feature of a statistical technique or of its results whereby the expected value of the results differs from the true underlying quantitative parameter being estimated.
The software instructions 120 enable a distributed ensemble learning which enables the programmable logic controller (PLC) 107 to use more than one processing units of the plurality of processing units 110(1-x) to scale an application 130. The distributed ensemble learning is a family of algorithms in supervised learning where multiple weaker learners (WLs) 127(1-x) were trained individually and then all of which will be combined to produce a prediction.
The software instructions 120 train the multiple weak learners (WLs) 127(1-x) using the boosting algorithm 122. The multiple weak learners (WLs) 127(1-x) are machine learning models (MODs) 135(1-x) that do not capture an entire data distribution 137 and are purposefully designed to predict with a lower accuracy 140. The entire data distribution 137 is a function or a listing which shows all the possible values (or intervals) of the data. It also (and this is important) tells how often each value occurs. Accuracy is the number of correct predictions divided by the total number of predictions. A weak learner doesn’t need to have a high accuracy. The lower accuracy 140 allows them to be lightweight and fast.
The software instructions 120 use the multiple weak learners (WLs) 127(1-x) to vote for a final hypothesis 142 based on a feed forward computation 144 of neural networks (NNs) 150(1-x). The final hypothesis 142 is a final prediction that a machine learning model outputs. In one embodiment, neural networks (NNs) 150(1-x) are used as the multiple weak learners (WLs) 127(1-x). The feed forward computation 144 is a matrix computation of a neural network during the prediction phase (antonym of back propagation).
The NPU 110 is an edge device designed to be used in the PLC system. An edge device is a hardware device that can locally perform optimized matrix operations rather than relying on connectivity such as a cloud computation.
For example, the boosting algorithm 122 may be applied to image detection. A single shot detector (SSD) may be used as a weak learner 127 while at least two hyperparameters 152(1-2) are used to intentionally render the model 125 weak. The two hyperparameters 152 are adjustable to achieve different performance or result in the machine learning model 125. The two hyperparameters 152 include a width multiplier 152(1) that thins the neural network system 105 at each layer and a resolution multiplier 152(2) that reduces an input image resolution. Accuracy of the model 125 and speed is adjusted with the two hyperparameters 152. Accuracy is the number of correct predictions divided by the total number of predictions. In image detection, it could also be a metric measured by the intersection over union (IOU) of the bounding boxes of the detected objects. Speed is the computation speed of the machine learning model 125 (i.e. how fast the model 125 produces a final prediction).
Each model of models (MODs) 135(1-x) being a WL 127 of the multiple weak learners (WLs) 127(1-x) returns a list of output bounding boxes 155 and their respective classes 157. The bounding boxes 155 capture the object of interest on an image. The boosting algorithm 122 may be used for a typical regression task or a classification task. In classification tasks, the machine learning model 125 is used to identify the class 157 of an object. For example, an image detection model trained to identify dogs and cats on images will produce bounding boxes labeled with the respective classes, dogs and cats. In a regression task, the machine learning model 125 is used to predict a continuous sample variable. For example, an image detection model trained to find human will calculate the center, width and height of a bounding box which are a continuous variable (as opposed to a discrete class).
The boosting algorithm 122 guarantees a reduction in the variance 160 without increasing the bias 162 thus making the model 125 more generalizable. The boosting algorithm 122 combines multiple distributed neural network models (MODs) 135(1-x) to create a more complex model without reaching a resource limitation. For example, computation resources of a computer or a hardware device such as the processing power and memory are limited. Such resource limitation can place an upper bound on the speed as well as the accuracy of the model predictions.
The boosting algorithm 122 combines all outputs to allow the model 125 to expand a Vapnik-Chervonenkis (VC) dimension 165 effectively covering a larger underlying distribution of training data. The Vapnik-Chervonenkis (VC) dimension 165 is a measure of the capacity (complexity, expressive power, richness, or flexibility) of a space of functions that can be learned by the machine learning model 125. With the boosting algorithm 122 training, each weak learner of the multiple weak learners (WLs) 127(1-x) focuses on a resampled subset of a dataset.
In order to apply the boosting algorithm 122 to image detection, there are two major modifications required. It can use the Mobilenets Single Shot Detector (SSD) as the weak learner. There are two hyperparameters 152(1-2) that can be used to intentionally render the model “weak”— the width multiplier 152(1) that thins a neural network at each layer and the resolution multiplier 152(2) that reduces the input image resolution. Based on the Table 1 and 2 below, the model accuracy and speed can be easily adjusted with these two hyperparameters 152(1-2).
The boosting algorithm 122 can train the weak learners (WLs) 127(1-x) as described below in Table 3 of the Boosting Neural Networks. However, the voting of the final hypothesis (the “Output” section) will need special care depending on applications.
For the typical regression or classification tasks, the boosting algorithm 122 detailed above can be used directly. For image detection tasks, the final output hypothesis cannot be a simple weighted sum as each model 135(1-x) of the weak learners (WLs) 127(1-x) (i.e. SSD) returns a list of bounding boxes 155 and their respective classes 157. To appropriately combine the weak learners’ results, all the output bounding boxes 155 and classes 157 shall be grouped together into a set. This set can contain many low-confidence predictions and duplicates. Therefore, it is important to discard the low confidence predictions by a threshold 170 and then apply the Non-Maximum Suppression to reduce the duplicates.
The boosting algorithm 122 can guarantee a reduction in the variance 160 without increasing the bias 162, making the model 125 more generalizable. This benefit solves the first overfitting problem. Another advantage is the convenience of distributing the model 125 among multiple processing units 110(1-x). Since each weak learner 127 can perform computations independent from one another, they can reside in separate NPU’s. With multiple NPU’s 110(1-x) processing the data concurrently, it will solve the two problems related to computational speed and memory limitation. This method will improve the scalability and flexibility of the NPU 110, making it applicable to a broader group of tasks.
Referring to
Turning now to
As seen in
As shown in
The method 600 performed by a neural network system comprises a step 605 of using the boosting algorithm 122 for an image detection task. Each model of the multiple weak learners (WLs) 127(1-x) returns a list of output bounding boxes 155 and their respective classes 157. The method 600 further comprises a step 610 of grouping together all the output bounding boxes 155 and all the classes 157 into a set such that the set contains many low-confidence predictions and duplicates. The method 600 further comprises a step 615 of discarding the many low-confidence predictions by using the threshold 170. The method 600 further comprises a step 620 of applying non-maximum suppression to reduce the duplicates.
In
The method 700 comprises through operating at least one processor a step 705 of providing the boosting algorithm 122 that distributes the model 125 among the plurality of processing units 110(1-x) each being a weak learner of multiple weak learners (WLs) 127(1-x) that can perform computations independent from one another yet process data concurrently. The method 700 further comprises a step 710 of enabling a distributed ensemble learning which enables a programmable logic controller (PLC) to use more than one processing units of the plurality of processing units to scale an application. The method 700 further comprises a step 715 of training the multiple weak learners using the boosting algorithm. The multiple weak learners are machine learning models that do not capture an entire data distribution and are purposefully designed to predict with a lower accuracy. The method 700 further comprises a step 720 of using the multiple weak learners to vote for a final hypothesis based on a feed forward computation of neural networks.
While a programmable logic controller (PLC) is described here a range of one or more other industrial controllers or other forms of industrial controllers are also contemplated by the present invention. For example, other types of industrial controllers may be implemented based on one or more features presented above without deviating from the spirit of the present invention.
The techniques described herein can be particularly useful for feed forward computation for prediction. While particular embodiments are described in terms of the feed forward computation, the techniques described herein are not limited to feed forward computation but can also be used with other neural networks.
With regard to
As shown in
The computer system 810 also includes a system memory 830 coupled to the bus 821 for storing information and instructions to be executed by processors 820. The system memory 830 may include computer readable storage media in the form of volatile and/or nonvolatile memory, such as read only memory (ROM) 831 and/or random access memory (RAM) 832. The system memory RAM 832 may include other dynamic storage device(s) (e.g., dynamic RAM, static RAM, and synchronous DRAM). The system memory ROM 831 may include other static storage device(s) (e.g., programmable ROM, erasable PROM, and electrically erasable PROM). In addition, the system memory 830 may be used for storing temporary variables or other intermediate information during the execution of instructions by the processors 820. A basic input/output system (BIOS) 833 containing the basic routines that helps to transfer information between elements within computer system 810, such as during start-up, may be stored in ROM 831. RAM 832 may contain data and/or program modules that are immediately accessible to and/or presently being operated on by the processors 820. System memory 830 may additionally include, for example, operating system 1034, application programs 835, other program modules 836 and program data 837.
The computer system 810 also includes a disk controller 840 coupled to the bus 821 to control one or more storage devices for storing information and instructions, such as a hard disk 841 and a removable media drive 842 (e.g., floppy disk drive, compact disc drive, tape drive, and/or solid state drive). The storage devices may be added to the computer system 810 using an appropriate device interface (e.g., a small computer system interface (SCSI), integrated device electronics (IDE), Universal Serial Bus (USB), or FireWire).
The computer system 810 may also include a display controller 865 coupled to the bus 821 to control a display 866, such as a cathode ray tube (CRT) or liquid crystal display (LCD), for displaying information to a computer user. The computer system includes an input interface 860 and one or more input devices, such as a keyboard 862 and a pointing device 861, for interacting with a computer user and providing information to the processor 820. The pointing device 861, for example, may be a mouse, a trackball, or a pointing stick for communicating direction information and command selections to the processor 820 and for controlling cursor movement on the display 866. The display 866 may provide a touch screen interface which allows input to supplement or replace the communication of direction information and command selections by the pointing device 1061.
The computer system 810 may perform a portion or all of the processing steps of embodiments of the invention in response to the processors 820 executing one or more sequences of one or more instructions contained in a memory, such as the system memory 830. Such instructions may be read into the system memory 830 from another computer readable medium, such as a hard disk 841 or a removable media drive 842. The hard disk 841 may contain one or more datastores and data files used by embodiments of the present invention. Datastore contents and data files may be encrypted to improve security. The processors 820 may also be employed in a multi-processing arrangement to execute the one or more sequences of instructions contained in system memory 830. In alternative embodiments, hard-wired circuitry may be used in place of or in combination with software instructions. Thus, embodiments are not limited to any specific combination of hardware circuitry and software.
As stated above, the computer system 810 may include at least one computer readable medium or memory for holding instructions programmed according to embodiments of the invention and for containing data structures, tables, records, or other data described herein. The term “computer readable medium” as used herein refers to any medium that participates in providing instructions to the processor 820 for execution. A computer readable medium may take many forms including, but not limited to, non-volatile media, volatile media, and transmission media. Non-limiting examples of non-volatile media include optical disks, solid state drives, magnetic disks, and magneto-optical disks, such as hard disk 841 or removable media drive 842. Non-limiting examples of volatile media include dynamic memory, such as system memory 830. Non-limiting examples of transmission media include coaxial cables, copper wire, and fiber optics, including the wires that make up the bus 821. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.
The computing environment 800 may further include the computer system 810 operating in a networked environment using logical connections to one or more remote computers, such as remote computer 880. Remote computer 880 may be a personal computer (laptop or desktop), a mobile device, a server, a router, a network PC, a peer device or other common network node, and typically includes many or all of the elements described above relative to computer system 810. When used in a networking environment, computer system 810 may include modem 872 for establishing communications over a network 871, such as the Internet. Modem 872 may be connected to bus 821 via user network interface 870, or via another appropriate mechanism.
Network 871 may be any network or system generally known in the art, including the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a direct connection or series of connections, a cellular telephone network, or any other network or medium capable of facilitating communication between computer system 810 and other computers (e.g., remote computer 880). The network 871 may be wired, wireless or a combination thereof. Wired connections may be implemented using Ethernet, Universal Serial Bus (USB), RJ-11 or any other wired connection generally known in the art. Wireless connections may be implemented using Wi-Fi, WiMAX, and Bluetooth, infrared, cellular networks, satellite or any other wireless connection methodology generally known in the art. Additionally, several networks may work alone or in communication with each other to facilitate communication in the network 871.
In some embodiments, the computer system 810 may be utilized in conjunction with a parallel processing platform comprising a plurality of processing units. This platform may allow parallel execution of one or more of the tasks associated with optimal design generation, as described above. For the example, in some embodiments, execution of multiple product lifecycle simulations may be performed in parallel, thereby allowing reduced overall processing times for optimal design selection.
The embodiments of the present disclosure may be implemented with any combination of hardware and software. In addition, the embodiments of the present disclosure may be included in an article of manufacture (e.g., one or more computer program products) having, for example, computer-readable, non-transitory media. The media has embodied therein, for instance, computer readable program code for providing and facilitating the mechanisms of the embodiments of the present disclosure. The article of manufacture can be included as part of a computer system or sold separately.
While various aspects and embodiments have been disclosed herein, other aspects and embodiments will be apparent to those skilled in the art. The various aspects and embodiments disclosed herein are for purposes of illustration and are not intended to be limiting, with the true scope and spirit being indicated by the following claims.
An executable application, as used herein, comprises code or machine readable instructions for conditioning the processor to implement predetermined functions, such as those of an operating system, a context data acquisition system or other information processing system, for example, in response to user command or input. An executable procedure is a segment of code or machine readable instruction, sub-routine, or other distinct section of code or portion of an executable application for performing one or more particular processes. These processes may include receiving input data and/or parameters, performing operations on received input data and/or performing functions in response to received input parameters, and providing resulting output data and/or parameters.
A graphical user interface (GUI), as used herein, comprises one or more display images, generated by a display processor and enabling user interaction with a processor or other device and associated data acquisition and processing functions. The GUI also includes an executable procedure or executable application. The executable procedure or executable application conditions the display processor to generate signals representing the GUI display images. These signals are supplied to a display device which displays the image for viewing by the user. The processor, under control of an executable procedure or executable application, manipulates the GUI display images in response to signals received from the input devices. In this way, the user may interact with the display image using the input devices, enabling user interaction with the processor or other device.
The functions and process steps herein may be performed automatically or wholly or partially in response to user command. An activity (including a step) performed automatically is performed in response to one or more executable instructions or device operation without user direct initiation of the activity.
The system and processes of the figures are not exclusive. Other systems, processes and menus may be derived in accordance with the principles of the invention to accomplish the same objectives. Although this invention has been described with reference to particular embodiments, it is to be understood that the embodiments and variations shown and described herein are for illustration purposes only. Modifications to the current design may be implemented by those skilled in the art, without departing from the scope of the invention. As described herein, the various systems, subsystems, agents, managers and processes can be implemented using hardware components, software components, and/or combinations thereof.
Computer readable medium instructions for carrying out operations of the present disclosure may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user’s computer, partly on the user’s computer, as a stand-alone software package, partly on the user’s computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user’s computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.
Aspects of the present disclosure are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, may be implemented by computer readable medium instructions.
It should be appreciated that the program modules, applications, computer-executable instructions, code, or the like depicted in
It should further be appreciated that the computer system 810 may include alternate and/or additional hardware, software, or firmware components beyond those described or depicted without departing from the scope of the disclosure. More particularly, it should be appreciated that software, firmware, or hardware components depicted as forming part of the computer system 810 are merely illustrative and that some components may not be present or additional components may be provided in various embodiments. While various illustrative program modules have been depicted and described as software modules stored in system memory, it should be appreciated that functionality described as being supported by the program modules may be enabled by any combination of hardware, software, and/or firmware. It should further be appreciated that each of the above-mentioned modules may, in various embodiments, represent a logical partitioning of supported functionality. This logical partitioning is depicted for ease of explanation of the functionality and may not be representative of the structure of software, hardware, and/or firmware for implementing the functionality. Accordingly, it should be appreciated that functionality described as being provided by a particular module may, in various embodiments, be provided at least in part by one or more other modules. Further, one or more depicted modules may not be present in certain embodiments, while in other embodiments, additional modules not depicted may be present and may support at least a portion of the described functionality and/or additional functionality. Moreover, while certain modules may be depicted and described as sub-modules of another module, in certain embodiments, such modules may be provided as independent modules or as sub-modules of other modules.
Although specific embodiments of the disclosure have been described, one of ordinary skill in the art will recognize that numerous other modifications and alternative embodiments are within the scope of the disclosure. For example, any of the functionality and/or processing capabilities described with respect to a particular device or component may be performed by any other device or component. Further, while various illustrative implementations and architectures have been described in accordance with embodiments of the disclosure, one of ordinary skill in the art will appreciate that numerous other modifications to the illustrative implementations and architectures described herein are also within the scope of this disclosure. In addition, it should be appreciated that any operation, element, component, data, or the like described herein as being based on another operation, element, component, data, or the like can be additionally based on one or more other operations, elements, components, data, or the like. Accordingly, the phrase “based on,” or variants thereof, should be interpreted as “based at least in part on.”
Although embodiments have been described in language specific to structural features and/or methodological acts, it is to be understood that the disclosure is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as illustrative forms of implementing the embodiments. Conditional language, such as, among others, “can,” “could,” “might,” or “may,” unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain embodiments could include, while other embodiments do not include, certain features, elements, and/or steps. Thus, such conditional language is not generally intended to imply that features, elements, and/or steps are in any way required for one or more embodiments or that one or more embodiments necessarily include logic for deciding, with or without user input or prompting, whether these features, elements, and/or steps are included or are to be performed in any particular embodiment.
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.
While embodiments of the present invention have been disclosed in exemplary forms, it will be apparent to those skilled in the art that many modifications, additions, and deletions can be made therein without departing from the spirit and scope of the invention and its equivalents, as set forth in the following claims.
Embodiments and the various features and advantageous details thereof are explained more fully with reference to the non-limiting embodiments that are illustrated in the accompanying drawings and detailed in the following description. Descriptions of well-known starting materials, processing techniques, components and equipment are omitted so as not to unnecessarily obscure embodiments in detail. It should be understood, however, that the detailed description and the specific examples, while indicating preferred embodiments, are given by way of illustration only and not by way of limitation. Various substitutions, modifications, additions and/or rearrangements within the spirit and/or scope of the underlying inventive concept will become apparent to those skilled in the art from this disclosure.
As used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having” or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a process, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but may include other elements not expressly listed or inherent to such process, article, or apparatus.
Additionally, any examples or illustrations given herein are not to be regarded in any way as restrictions on, limits to, or express definitions of, any term or terms with which they are utilized. Instead, these examples or illustrations are to be regarded as being described with respect to one particular embodiment and as illustrative only. Those of ordinary skill in the art will appreciate that any term or terms with which these examples or illustrations are utilized will encompass other embodiments which may or may not be given therewith or elsewhere in the specification and all such embodiments are intended to be included within the scope of that term or terms.
In the foregoing specification, the invention has been described with reference to specific embodiments. However, one of ordinary skill in the art appreciates that various modifications and changes can be made without departing from the scope of the invention. Accordingly, the specification and figures are to be regarded in an illustrative rather than a restrictive sense, and all such modifications are intended to be included within the scope of invention.
Although the invention has been described with respect to specific embodiments thereof, these embodiments are merely illustrative, and not restrictive of the invention. The description herein of illustrated embodiments of the invention is not intended to be exhaustive or to limit the invention to the precise forms disclosed herein (and in particular, the inclusion of any particular embodiment, feature or function is not intended to limit the scope of the invention to such embodiment, feature or function). Rather, the description is intended to describe illustrative embodiments, features and functions in order to provide a person of ordinary skill in the art context to understand the invention without limiting the invention to any particularly described embodiment, feature or function. While specific embodiments of, and examples for, the invention are described herein for illustrative purposes only, various equivalent modifications are possible within the spirit and scope of the invention, as those skilled in the relevant art will recognize and appreciate. As indicated, these modifications may be made to the invention in light of the foregoing description of illustrated embodiments of the invention and are to be included within the spirit and scope of the invention. Thus, while the invention has been described herein with reference to particular embodiments thereof, a latitude of modification, various changes and substitutions are intended in the foregoing disclosures, and it will be appreciated that in some instances some features of embodiments of the invention will be employed without a corresponding use of other features without departing from the scope and spirit of the invention as set forth. Therefore, many modifications may be made to adapt a particular situation or material to the essential scope and spirit of the invention.
Respective appearances of the phrases “in one embodiment,” “in an embodiment,” or “in a specific embodiment” or similar terminology in various places throughout this specification are not necessarily referring to the same embodiment. Furthermore, the particular features, structures, or characteristics of any particular embodiment may be combined in any suitable manner with one or more other embodiments. It is to be understood that other variations and modifications of the embodiments described and illustrated herein are possible in light of the teachings herein and are to be considered as part of the spirit and scope of the invention.
In the description herein, numerous specific details are provided, such as examples of components and/or methods, to provide a thorough understanding of embodiments of the invention. One skilled in the relevant art will recognize, however, that an embodiment may be able to be practiced without one or more of the specific details, or with other apparatus, systems, assemblies, methods, components, materials, parts, and/or the like. In other instances, well-known structures, components, systems, materials, or operations are not specifically shown or described in detail to avoid obscuring aspects of embodiments of the invention. While the invention may be illustrated by using a particular embodiment, this is not and does not limit the invention to any particular embodiment and a person of ordinary skill in the art will recognize that additional embodiments are readily understandable and are a part of this invention.
It will also be appreciated that one or more of the elements depicted in the drawings/figures can also be implemented in a more separated or integrated manner, or even removed or rendered as inoperable in certain cases, as is useful in accordance with a particular application.
Benefits, other advantages, and solutions to problems have been described above with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any component(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as a critical, required, or essential feature or component.
| Filing Document | Filing Date | Country | Kind |
|---|---|---|---|
| PCT/US2020/028694 | 4/17/2020 | WO |
