The present disclosure generally relates to computer systems, and more particularly, to improved computational efficiency by neural estimation.
AI is a data-driven technology that takes unique datasets as input to train AI computer models on appropriately configured computing devices. Once trained, an AI computer model may take new data as input to predict, classify, or otherwise output results for use in a variety of applications. In machine learning, a widely used AI technique in computing systems, involves a process that uses data and algorithms to train (or teach) computer models. The trained model allows the computing device to make decisions without the need for explicit or rule-based programming. In particular, machine learning algorithms build a model on training data to identify and extract patterns from the data, and therefore acquire (or learn) unique knowledge that can be applied to new data sets.
Today, computing devices often use mutual information (MI) in data sets, which provides a ubiquitous measure of dependency between a pair of random variables, and is one of the corner stones of information theory. In machine learning, the information maximization principle for learning representation from unlabeled data through self-supervision motivated the development of many MI estimators and applications. The information bottleneck is another principle that triggered recent interest in mutual information estimation. MI is also used to understand the information flow in neural networks, in learning clusters and in regularizing the training of Generative Adversarial Networks (GANs).
According to various embodiments, a computing device, a non-transitory computer readable storage medium, and a method are provided for determining whether data is related. A data X having N samples and a data Y having N samples is received. A function f(x,y) is defined to be a trainable neural network based on the data X and the data Y. Variable η is defined to be trainable and scalar. A permuted version of the data Y is created. A loss mean based on the trainable neural network f(x,y), the permuted version of the sample data Y, and the trainable scalar variable η is computed. A loss with respect to the trainable neural network f(x,y) and the scalar variable η is minimized. It is determined whether the trainable neural network f(x,y) having undergone the minimization of the loss provides an output that meets one or more predetermined criteria. Upon determining that the trainable neural network f(x,y) has not met the one or more predetermined criteria, the process returns to the act of creating a permuted version of the second sample data Y and the iterative process continues. However, upon determining that the trainable neural network f(x,y) has met the one or more predetermined criteria, test data XT and YT is received. A mutual information (MI) between the test data XT and YT is estimated. Upon determining that the estimated MI is at or above a predetermined threshold, the test data XT and YT is deemed to be dependent. However, upon determining that the estimated MI is not at or above the predetermined threshold, the test data XT and YT is deemed to be independent.
In one embodiment, the one or more predetermined criteria include at least one of: a loss with respect to the trainable neural network f(x,y) and the scalar variable η is at or below a predetermined threshold; or a maximum number of iterations of minimizing the loss with respect to the trainable neural network f(x,y) and the scalar variable η is reached.
In one embodiment, a similarity between the data X and the data Y is determined by way of KL divergence for various bounds.
In one embodiment, the determination of the similarity between the data X and the data Y does not include a logarithm term.
In one embodiment, the determination of the similarity between the data X and the data Y includes: upon determining that the estimated MI is at or above the predetermined threshold, generating a positive scalar output; and upon determining that the estimated MI is not at or above the predetermined threshold, generating a negative scalar output.
In one embodiment, the permuted version of the data Y includes a random distribution of the N samples of the data Y with respect to the N samples of the data X.
In one embodiment, the loss mean is f(x_i, y_i)−exp(-eta) [mean exp[f(x_i, y0_i)]]+η−1.
In one embodiment, the estimation of the MI is based on I(xt, yt)>=mean f(xt_i, yt_i)−log [mean exp [f(xt_i, yt_i)]].
In one embodiment, η plays a role of a Lagrange multiplier that ensures normalization of a likelihood ratio.
In one embodiment, the estimation of the mutual information (MI) between the test data XT and YT includes a linear witness function.
In one embodiment, the estimated mutual information (MI) is used in the context of a generative adversarial network (GAN).
These and other features will become apparent from the following detailed description of illustrative embodiments thereof, which is to be read in connection with the accompanying drawings.
The drawings are of illustrative embodiments. They do not illustrate all embodiments. Other embodiments may be used in addition or instead. Details that may be apparent or unnecessary may be omitted to save space or for more effective illustration. Some embodiments may be practiced with additional components or steps and/or without all the components or steps that are illustrated. When the same numeral appears in different drawings, it refers to the same or like components or steps.
In the following detailed description, numerous specific details are set forth by way of examples to provide a thorough understanding of the relevant teachings. However, it should be apparent that the present teachings may be practiced without such details. In other instances, well-known methods, procedures, components, and/or circuitry have been described at a relatively high-level, without detail, to avoid unnecessarily obscuring aspects of the present teachings.
The present disclosure generally relates to improvements to computer-related technology. More particularly, it relates to improved computational efficiency and accuracy in computing devices that perform machine learning and artificial intelligence. In many machine learning applications and other scientific fields, a computing device performing the machine learning application estimates MI given samples from the joint distribution of high dimensional random variables. This is a challenging problem and many methods have been devised to address it. Since MI is defined as the Kullback-Leibler (KL) divergence between the joint distribution and the product of marginals, a computing device can leverage non-parametric estimators of f-divergences. Specifically of interest to us is the Donsker-Varadhan (DV) representation of the KL divergence that can be used with neural networks estimators. Other approaches to estimating the MI are through finding lower bounds using variational Bayesian methods, as well as through geometric methods like binning.
What is provided herein is a new estimator of MI that can be used in direct MI maximization or as a regularizer, thanks to its unbiased gradients. Our starting point is the DV lower bound of the KL divergence that is represented equivalently via a joint optimization that is referred to herein as η-DV on a witness functionf and an auxiliary variable η.
In one aspect what is also shown is that when the witness function f is learned in a Reproducing Kernel Hilbert Space (RKHS) the η-DV problem is jointly convex in bothf and The dual of this problem sheds the light on this estimator as a constrained ratio estimation where η plays the role of a Lagrange multiplier that ensures proper normalization of the likelihood ratio of the joint and the product of marginal probability distributions. What is also discussed herein is that the witness function can be estimated as a neural network. In one aspect, the estimator for MI is specified, and it is shown how it compares to known alternatives. Reference now is made in detail to the examples illustrated in the accompanying drawings and discussed below.
A calculation of the MI of the pair of random variables/features X and Y may be too computationally complex for a computing device. Thus, a direct calculation of the MI of a pair of random variables/features is typically impractical. Rather, what is performed is an estimation of the MI by way of using lower bounds, discussed in more detail below. In this regard, reference is made to
Consider two probability distributions P and Q, where P is absolutely continuous with regard to Q. Let p and q be their respective densities defined on X⊂Rd. Their Kullback-Leibler
We are interested in the MI between two random variables X, Y where Xis defined on X⊂Rd
I(X;Y)=KL(pxy, pxpy), (Eq. 2)
Equation 2 above provides the KL divergence between the joint density and the product of marginals. Non-parametric estimation of MI from samples is a salient consideration in science and machine learning. Below, the variational lower bounds on KL to enable such estimation are discussed.
To determine the variational characterization of KL divergence, we let H be any function space mapping to R.
KL()≥=sup {f(x)−log ((x)):f∈} (Eq. 3)
where the equality holds if and only if (iif):
f*=log (p/q)∈ (Eq. 4)
The bound of equation 3 is referred to herein as the DV bound. The second variational representation is provided by the equation 5 below:
KL()≥()=1+sup{f(x)−f(x):f∈(} (Eq. 5)
with equality iif:
f*=log(p/q)∈ (Eq. 6)
The bound provided by equation 5 above is referred to herein as thef-div bound (as in f-divergence). From equations 3 and 5 indicate that the variational bounds estimate the log-likelihood ratio f*=log(p/q), and the tightness of the bound depends on the representation power of H. In order to compare these two lower bounds, observe that log(t)≤t−1, t>0.
Therefore,
log (f(x))≤f(x)−1 (Eq. 7)
KL()≥()≥() ((Eq. 8)
Equation 8 indicates that the DV bound is tighter than the f-div bound.
Now, given samples {xi, i=1 . . . N, P}, {yi, i=1 . . . N, yi≈Q}, in one embodiment, the KL divergence can be estimated by computing the variational bound from Monte-Carlo simulation. Specifically, in one embodiment, the following estimator can be used for the DV bound:
Note that the Mutual Information Neural Estimator (MINE) considered the above formulation with the hypothesis class H being a neural network. For the f-div bound we have similarly the following estimator:
While DV bound is tighter than the f-div bound, in order to learn the function f using stochastic gradient optimization, f-div is a better fit because the cost function is linear in the expectation, whereas in the DV bound we have a log non-linearity being applied to the expectation. This non-linearity can introduce biases in the mini-batch estimation of the cost function. In one aspect, what is discussed herein is how to alleviate this problem and remove the non-linearity at the cost of an additional auxiliary variable that will enable better optimization for the DV bound, sometimes referred to herein as an η-trick for the DV bound.
We start with the following Lemma, that gives a variational characterization of the log.
Lemma 1. Let x>0, we have: log(x)=minηe−ηx+η−1.
Using Lemma 1 we can now linearize the log in the DV bound.
Lemma 2. Let H be any function space mapping X to R.
KL()≥()=−inf{L(f, η):f∈,η∈} (Eq. 11)
L(f, η)=e−ηf(x)−f(x)+η−1 (Eq. 12)
Equation 11 is referred to herein as the η-DV bound. Note that for η=0 we recover the f-div bound. Using Lemma 2, we can now rewrite the estimator for the DV bound in Eq. 9 as follows:
Equation 13 enables unbiased stochastic gradient optimization of the function f.
In our discussion above it was established that the DV bound is tighter than the f-div and derived ri-DV as an alternative to DV, enabling unbiased stochastic optimization. However, the role of η in improving the lower bound remains unclear. To highlight its role, we consider the η-DV bound from Eq. 11, and restrict the hypothesis function class to an RKHS. We can compare η-DV estimator to the f-div estimator and further understand this hierarchy of lower bounds. In this regard, reference now is made to
={f|f(x)=w, Φ(x), Φ:X→m, w∈m} (Eq. 15)
Now for f∈H , the loss given in Eq. 12 for η-DV can be rewritten as follows:
L(f, η)L(w, η)=e−η−w, Φ(x)+η−1 (Eq. 16)
We consider the following regularized loss: L(w, η) =L(w, η)+Ω(w), and the corresponding sample-based formulation (w, η)=L(w, η)+Ω(w), where Ω(w) is a convex regularizer, e.g.,
The η-DV primal problem can now be defined as:
The discussion below explains that L(w, η) is jointly convex in (w, η) and derive its dual, which will shed light on the nature of the likelihood ratio estimate as well as the role of η. In Lemma 3 we first establish the convexity of the η-DV loss function.
Lemma 3. and are jointly convex in w and η.
(u, η)=L(u, η)+Ω(u)+ϵe−η (Eq. 18)
Accordingly, the KL estimate is therefore given as follows:
()=−L(w*, η*) (Eq. 19)
Where (w*, η*)=arg min(u, η)(u, η) (Eq. 20)
In another embodiment, a way of estimating the KL divergence is through a likelihood ratio estimation. Given a ratio estimate {circumflex over (r)} of p/q, the KL divergence can be computed as EQ{circumflex over (r)} log({circumflex over (r)}), which can be easily estimated using samples from Q.
Theorem 1 below shows that the dual problem (denoted D), corresponding to the primal minimization problem (denoted P) of η-DV, reduces to a constrained likelihood ratio estimation problem (denoted C).
Theorem 1: Let Ω*(.) be the Fenchel conjugate of Ω(.). The η-DV bound restricted to an RKHS, amounts to the following regularized convex minimization: P=(minw,ηL(w, η)+106 (w)), that has the following dual form:
Noticing that η−1 plays the role of a Lagrangian multiplier, this is equivalent to the following likelihood ratio estimation problem:
Therefore, we have P=D=C. Let (w*, η*X an optimizer of P. Let r* be an optimizer of D, then KL estimate is given by the equation below:
()=∫Xr*(y)log(r*(y))q(y)dy=L(w*, η*) (Eq. 23)
The regularizer
can now be given the following interpretation based on the results of Theorem 1. It is noted that the definition of the MMD between distributions using an RKHS: MMDΦ(P, Q)=∥Ex−PΦ(x)−EyQΦ(y)∥. Replacing the Fenchel conjugate Q*(.) by its expression in Theorem 1, we see that η-Dy is equivalent to the following dual (η-DV-D):
Equation 24 can be written as a constrained ratio estimation problem (η-DVC):
Accordingly, the foregoing theorem demonstrates that the rj-Dy optimization problem is equivalent to the constrained likelihood ratio estimation problem r given in Eq. 25, where the ratio is estimated using the MMD distance in the RKHS between rq and p. It is also noted that p/q is a feasible point and for the universal feature map MMDΦ(rq, p)=0 iif r=p/q. Therefore, for a universal kernel, p/q is optimal and we recover the KL divergence for r=p/q. When comparing η-DVD from Eq. 24 and η-DV-C from Eq. 25, we see that η−1 plays the role of a Lagrangian that ensures that rq is indeed a normalized distribution. In practice, we solve the primal problem (P), while the dual problem (D) and its equivalent constrained form (C) explain why this formulation estimates the KL divergence and the role of η as a Lagrangian multiplier that enforces a normalization constraint. Let r* be the solution, then we have
()=∫xr*(y)log(r*(y))q(y)dy=−L(w*, η*) (Eq. 26)
For comparison, the f-div bound, restricted to an RKHS, is equivalent to the following ratio estimation (which follow from the proof of Theorem 1 by eliminating max on η, and setting η=0).
Let r* be the optimum, then the KL divergence can be estimated as follows:
()=∫Xr*(y)log(r*(y))q(y)dy+(1−17 Xr*(y)q(y)dy) (Eq. 28)
Comparing () and () it is noted that they both have a relative entropy term, but the f-div bound does not impose the normalization constraint on the ratio, which biases the estimate. In Theorem 1, if the loss L is replaced by its empirical counterpart L from equation 13 hereinabove, the equivalent Dual takes the following form:
The KL estimate is given by the expression below:
One shortcoming of the RKHS approach is that the method depends on the choice of feature map Φ. The teachings herein propose to learn Φ as a deep neural network as in MINE. Given samples xi from P, yi from Q, the KL estimation problem becomes:
Eq. 31 can be solved using BCD on (w, η, Φ). Note that if Φ(⋅) is known and fixed, then optimization problem in Eq. 31 becomes convex in η and w. We refer to this bound as η-DV-convex.
The following observations can be made. First, regarding ratio estimation via feature matching, KL divergence estimation with variational bounds is a ratio estimation using a form of MMD matching. Second, the choice of the feature space or the architecture of the neural network introduces a bias in the ratio estimation. Third, regarding Ratio Normalization, η-Dy bound introduces a normalization constraint on the ratio, that provides a better consistent estimate. Fourth, the choice of the regularizer dictates the choice of the metric.
We now specify the KL estimators (discussed above) for the MI estimation problem. Given a function space H defined on X×Y,
Equivalently, with the η-trick we have:
Accordingly, equation 34 demonstrates that the log term of Eq. 32 is removed, thereby substantially reducing the computational complexity of the computing device performing the MI calculation. Indeed, given iid samples from marginals xi˜px and {tilde over (y)}i˜py, i=1 . . . N, and samples from the joint (x1, yi)˜pxy, i=1 . . . N, the MI can be estimated as follows:
When H is an RKHS, η can be seen as a Lagrangian ensuring that the ratio estimation r(x, y) of
is normalized when integrated on pxpy, i.e., η is a Lagrangian associated with) the constraint ∫X×yr(x, y)px(x)py(y)dxdy=1. In this regard,
Lemma 4: For any function g that maps X×Y to R+ we have:
KL(px,y, pxpy)+p
Accordingly: I(X;Y)=supf∈G(X×Y) Epx,y f(x, y)−Epy log(Epx ef(x,y)) where G(X×Y) is the entire space of functions defined on X×Y. For any function space H map X×Y to R there is the following:
Using Jensen's inequality, the following hierarchy of lower bounds is provided:
I(X;Y)≥(X;Y)≥(X;Y)≥(X;Y) (Eq. 38)
In Theorem 2, the η-trick of Lemma 1 to Iα−DvH(X; Y) is applied, where each log term (e.g., (p
Theorem 2 (η-trick for Improved DVxBbund). Let H be a space of bounded functions defined on X×Y we have I(X; Y)≥−inff∈H,η∈G(Y)Epx,pyef(x,y)−η(y)+Ep η(y)−1−Epxy(f(x, y)), where G (Y) is the entire space of functions defined on Y. Let Hη be a function space subset of G (Y), the following expression is obtained:
Accordingly, we recover the lower bound given in Table 1 of
Second, as discussed above, in the embodiment where we restrict H to an RKHS, η(y) plays a role of a Lagrangian multiplier or a dual potential in a ratio estimation problem. In this embodiment, the Lagrangian multiplier is enforcing multiple constraints ∫Xr(x, y)p(x)dx=1 for each y, instead of imposing a single constraint ∫x∫yr(x, y)p(x)p(h)dxdy=1 as in the Ia−DVH(X; Y) bound.
Third, while Ia−DVH(X; Y) may be tighter than the Iη−DVH(X; Y), it may come at the price of estimating a function 11(y) together with the witness function f. This triggers a higher computational cost and larger estimation errors. Stated differently, the MI provided herein provides results that are more precise and involve less computational resources. It is also noted that Ia−DVH(X; Y) can be extended to conditional MI estimation. Further, the η-DV bound discussed herein is applicable for general probability distributions P and Q: KL(P, Q)≥Dη−DVH(X; Y) generally is applicable to P=pxy and Q=pxpy. In this regard, reference is made to
With the foregoing mathematical explanation of the accuracy and reduced computational complexity of the MI estimation discussed herein, it may be helpful to provide some example use cases. In various scenarios, synthetic and/or real data can be used on a number of applications to compare the proposed η-MINE MI estimator to some existing baselines. In one example, different MI estimators are compared on three synthetically generated Gaussian datasets (e.g., 5K training and 1K testing samples). By way of example and not by way of limitation, each evaluated MI estimator was run 10 times for a 2-D, 10-D and 20-D gaussian. Thus, data was sampled from 2, 10, and 20-dimensional Gaussian distributions with randomly generated means and covariance matrices. As the data complexity is increased, the divergence between estimators decreases, although applicants observed that η-MINE on average performed better than the baseline methods, converging to the true MI [red line] faster.
It will be understood that MI estimation in high D (dimensions) is a challenging task, where the estimation error for all methods increases as the data dimensionality grows. Nevertheless, simulations have shown that the proposed η-MINE is able to achieve on average more accurate results compared to the existing baselines. It was also observed that the convex formulation of η-MINE has overall a better performance and fast convergence rate. This estimator has a linear witness function that is defined as f(⋅)=w, Φ(⋅) using pre-trained fixed feature map Φ(⋅) from the regular η-MINE. In the experiments that follow the proposed estimator η-MINE was compared to DV-MINE, as a main baseline approach.
MI-regularized GAN. We investigate GAN training improvements with MI, and especially diminishing mode collapse. By way of example and not by way of limitation, conditional generator G is supplied with random codes c along with noise z. The mutual information between I(G(z, c), c) is maximized using η-MINE estimators. The task is repeated to the 10-D gaussian, where it is established that η-MINE can recover all the modes within fewer steps than DV-MINE (i.e., 50K vs. 140K) and with a stable training. For example, as used herein, stable relates to, as training progresses, the loss is continuously decreasing. During unstable training, the loss can increase or fluctuate up and down with large magnitude.
Self-supervised: Deep InfoMax In one embodiment, the teachings herein can be applied to unsupervised or self-supervised learning. For example, in unsupervised learning, machine learning computing device 102 of
The encoder is trained by maximizing MI I(x y) between features from one of the shallow layers l(x′=Ei(x)) and a deeper layer k(y′=Ek(y)). Applicants examined different layer combinations and found that the encoder comprising of only the first two convolutional layers provides the best performance on the downstream tasks. As shown in
As illustrated in
Self-Supervised: Jigsaws with MI. In one embodiment, a self-supervision Jigsaw pretext task aims at solving an image jigsaw puzzle by predicting the scrambling permutation π. In this regard, reference is made to
With the foregoing overview of the example architecture 100, various building blocks, and mathematical explanations, it may be helpful now to consider a high-level discussion of an example process. To that end,
At block 902, during a training phase, the MI estimator engine 104 of the machine learning computing device 102 receives a first data X having N samples and a second data Y having N samples.
At block 904, the computing device 102 defines a function (i.e., f(x,y)) to be a trainable neural network based on the received first data X and the second data Y. Eta (η) is defined to be a trainable scalar.
In one embodiment, the computing device 102 determines a similarity between the data X and the second data Y.
At block 908, a permuted version of the second data Y is created by the MI estimator engine 104. For example, the permuted version is a random distribution of the N samples of the second data Y with respect to the N samples of the first data X. Stated differently, the N samples of data Y are randomly rearranged with respect to the N samples of data X.
At block 910, a loss mean is computed by the MI estimator engine 104, based on the trainable neural network f(x,y), the permuted version of the second data Y, and a trainable scalar variable if For example, the loss mean is f(x_i, y_i)−exp(-eta) [mean exp [f(x_i, y_i)]]+η−1.
At block 912, the MI estimator engine 104 minimizes a loss with respect to the scalar variable η and the trainable neural network f(x,y). For example, the loss during this training process is the negative of an MI estimate. The goal is to maximize MI or minimize loss. In one embodiment, the minimization is performed using stochastic gradient descent.
At block 914, the MI estimator engine 104 determines whether the trainable neural network f(x,y), having undergone the minimization of the loss, provides an output that is at or below a predetermined threshold. Upon determining that the trainable neural network f(x,y) is not at or below the predetermined threshold or a threshold number of iterations have not yet been reached (i.e., “NO” at decision block 914), the iterative process returns to block 908 to create a permuted version of the second sample data Y, thereby providing another opportunity to finetune the trainable neural network f(x,y) and eta. In one embodiment, the iterative process is repeated until a predetermined threshold (e.g., number of iterations or time) is met. In one embodiment, the iterative process is based on stochastic optimization (SO).
For example, during this training process, the goal is to avoid using a log term in the calculation because otherwise the log term may bias the result. In this regard, it is noted that training data is used for training (i.e., to estimate f( ) and eta). But once the training is completed, the obtained estimator can be used on new, fresh data to compute the actual MI estimate, referred to herein as test data, as discussed in more detail below. During this computation the MI estimator engine 104 uses the original formula that includes a log term (e.g., equation 3).
Returning to block 914, upon determining that the trainable neural network f(x,y) is at or above the predetermined threshold or a threshold number of iterations have been performed (i.e., “YES” at determination block 914), the process continues with block 916, where the MI estimator engine 104 is promoted from training to testing mode and receives test data XT and YT.
At block 918, the MI estimator engine 104 determines an MI between the test data XT and YT. For example, the MI estimate is: I(xt, yt)>=mean f(xt_i, yt_i)−log [mean exp[f(xt_i, yt_i)]].
At block 920, the MI estimator engine 104 determines whether the computed MI estimate is at or above a predetermined threshold. If not (i.e., “NO” at decision block 920), the test data XT and YT are deemed to be independent (i.e., at block 924). However, if the MI estimate is at or above the predetermined threshold (i.e., “YES” at decision block 920), the test data XT and YT are deemed to be dependent (i.e., at block 926). For example, at decision block 920, a determination is made whether the test data XT and the test data YT came from joint or marginal probability densities. Based on the MI between the test data XT and the test data YT, a corresponding scalar output is provided by the MI estimator engine 104 of the machine learning computing device 102. In one embodiment, upon determining that the similarity is at or above a predetermined threshold, a positive number is provided, and upon determining that the similarity is below the predetermined threshold, a negative number is provided.
As discussed above, functions relating to efficient and unbiased mutual information estimation for the pair of random variables/features X and Y can be performed with the use of one or more computing devices connected for data communication via wireless or wired communication, as shown in
The computer platform 1000 may include a central processing unit (CPU) 1004, a hard disk drive (HDD) 1006, random access memory (RAM) and/or read only memory (ROM) 1008, a keyboard 1010, a mouse 1012, a display 1014, and a communication interface 1016, which are connected to a system bus 1002.
In one embodiment, the HDD 1006, has capabilities that include storing a program that can execute various processes, such as the MI estimator engine 1040, in a manner described herein. The MI estimator engine 1040 may have various modules configured to perform different functions. For example, there may be an interaction module 1042 that is operative to receive raw electronic data from various sources, including electronic data related to variables/features X and Y, as discussed herein.
In one embodiment, there is a similarity module 1044 that is operative to determine similarities between different datasets. Based on the similarity, the similarity module 104 can assign a corresponding scalar score.
There may be a machine learning module 1048 operative to, during a training phase, learn from historical data to build one or more machine learning models that can be used to train various functions, including neural networks. There may be a permutation module 1052 operative to provide a permuted version of received data during an iterative training process. For example, the permuted version may be a random distribution of N samples of data Y with respect to the N samples of data X.
In one embodiment, there is a threshold determination module 1056 operative to determine whether a trainable network has an accuracy that is at or above a predetermined threshold.
In one embodiment, a program, such as ApacheTM, can be stored for operating the system as a Web server. In one embodiment, the HDD 1006 can store an executing application that includes one or more library software modules, such as those for the JavaTM Runtime Environment program for realizing a JVM (JavaTM virtual machine).
The descriptions of the various embodiments of the present teachings have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.
While the foregoing has described what are considered to be the best state and/or other examples, it is understood that various modifications may be made therein and that the subject matter disclosed herein may be implemented in various forms and examples, and that the teachings may be applied in numerous applications, only some of which have been described herein. It is intended by the following claims to claim any and all applications, modifications and variations that fall within the true scope of the present teachings.
The components, steps, features, objects, benefits and advantages that have been discussed herein are merely illustrative. None of them, nor the discussions relating to them, are intended to limit the scope of protection. While various advantages have been discussed herein, it will be understood that not all embodiments necessarily include all advantages. Unless otherwise stated, all measurements, values, ratings, positions, magnitudes, sizes, and other specifications that are set forth in this specification, including in the claims that follow, are approximate, not exact. They are intended to have a reasonable range that is consistent with the functions to which they relate and with what is customary in the art to which they pertain.
Numerous other embodiments are also contemplated. These include embodiments that have fewer, additional, and/or different components, steps, features, objects, benefits and advantages. These also include embodiments in which the components and/or steps are arranged and/or ordered differently.
Aspects of the present disclosure are described herein with reference to a flowchart illustration and/or block diagram of a method, apparatus (systems), and computer program products according to embodiments of the present disclosure. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.
These computer readable program instructions may be provided to a processor of an appropriately configured computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The call-flow, flowchart, and block diagrams in the figures herein illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the Figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.
While the foregoing has been described in conjunction with exemplary embodiments, it is understood that the term “exemplary” is merely meant as an example, rather than the best or optimal. Except as stated immediately above, nothing that has been stated or illustrated is intended or should be interpreted to cause a dedication of any component, step, feature, object, benefit, advantage, or equivalent to the public, regardless of whether it is or is not recited in the claims.
It will be understood that the terms and expressions used herein have the ordinary meaning as is accorded to such terms and expressions with respect to their corresponding respective areas of inquiry and study except where specific meanings have otherwise been set forth herein. Relational terms such as first and second and the like may be used solely to distinguish one entity or action from another without necessarily requiring or implying any actual such relationship or order between such entities or actions. The terms “comprises,” “comprising,” or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by “a” or “an” does not, without further constraints, preclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.
The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed
Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments have more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.