Method for processing tensor data for pattern recognition and computer device

Abstract

A method for processing tensor data for pattern recognition and a computer device are provided. The method includes: constructing a decision function by the optimal projection tensor W which has been rank-one decomposed together with the offset scalar b, and inputting to-be-predicted tensor data which has been rank-one decomposed into the decision function for prediction.

Description

FIELD OF THE INVENTION

The present application belongs to the technical field of pattern recognition, especially to a method for processing tensor data for pattern recognition and a computer device.

BACKGROUND OF THE INVENTION

With the advent of big data era, tensor expression of data has been widely used. However, during achieving the invention, the inventor found out that in prior art vector model algorithm is still utilized to process tensor data. On the basis of the concept of vector model algorithm, during a preprocessing phase, feature extraction for original data (vectorization) should be performed, which firstly is easy to destroy spatial information and inner correlation which are specific to tensor data, secondly possesses superabundant modern parameters which would easily lead to issues such as curse of dimensionality, over learning and small amount of samples.

A plurality of tensor mode algorithms have become a trend of the era. However, solving an objective function of STM is a non-convex optimization issue, in which solving by using an alternative projection method is required; the time complexity of the algorithm is high and a local minimum value occurs frequently.

SUMMARY OF THE INVENTION

In light of this, an embodiment of the invention provides a method for processing tensor data for pattern recognition and a computer device so as to solve the problems such as curse of dimensionality, over learning and small amount of samples occurred when the vector mode algorithms provided by the prior art process the tensor data and overcome the shortcomings of the tensor mode algorithms of the prior art. The algorithm of the invention aims at solve the limitations of the algorithms of the prior art, for example, the time complexity of the algorithms is high, and a local minimum value occurs frequently, etc.

On one hand, a method for processing tensor data for pattern recognition is provided; the method includes:

receiving an input training tensor data set;

introducing a within class scatter matrix into an objective function such that between class distance is maximized, at the same time, within class distance is minimized by the objective function;

constructing an optimal frame of the objective function of an optimal projection tensor machine subproblem;

transforming N vector modes of quadratic programming subproblems into a multiple quadratic programming problem under a single tensor mode, and constructing an optimal frame of an objective function of an optimal projection tensor machine problem;

according to lagrangian multiplier method, obtaining a dual problem of the optimal frame of the objective function, introducing a tensor rank one decomposition into calculation of tensor transvection, and obtaining a revised dual problem;

utilizing sequential minimal optimization algorithm to solve the revised dual problem and output an alagrangian optimal combination and an offset scalar b;

calculating a projection tensor W_*;

performing the rank one decomposition to the projection tensor W_*;

performing a back projection to a component obtained after performing the rank one decomposition to the projection tensor W_*;

performing rank one decomposition inverse operation to the component obtained after performing the back projection to obtain an optimal projection tensor W which is corresponded to the training tensor data set;

decision function construction phase: by the optimal projection tensor W which has been rank-one decomposed together with the offset scalar b, constructing a decision function;

application prediction phase: inputting to-be-predicted tensor data which has been rank-one decomposed into the decision function for prediction.

Furthermore, after introducing the within class scatter matrix into an objective function of an STM subproblem, through an eta coefficient the objection function of the quadratic programming problem of an n-th subproblem is changed into:

$\underset{w^{\left(n\right)},b^{\left(n\right)},{\zeta }^{\left(n\right)}}{\min }\frac{1}{2}\left[\left({w^{\left(n\right)}}_F^2+\eta \left({\left(w^{\left(n\right)}\right)}^T{S}_w^{\left(n\right)}{w}^{\left(n\right)}\right)\right)\prod _{1<i<N}^{i\ne n}\left({w^{\left(i\right)}}_F^2+\eta \left({\left(w^{\left(i\right)}\right)}^T{S}_w^{\left(n\right)}{w}^{\left(i\right)}\right)\right)\right]+C\sum _{m=1}^M{\xi }_m^{\left(n\right)}$

wherein, S_w⁽ⁿ⁾ is an n-th order within class scatter matrix estimated after the training tensor data set is expanded along the n-th order; w⁽ⁿ⁾ is an n-th order optimal projection vector of the training tensor data, n=1, 2, . . . N; C is a penalty factor; ξ_m⁽ⁿ⁾ is a slack variable; eta coefficient η is configured to measure the importance of the within class scatter matrix.

Furthermore, the optimal frame of an objective function of an OPSTM problem is a combination of N vector modes of quadratic programming problems, which respectively corresponds to a subproblem; wherein, a quadratic programming problem of an n-th subproblem is:

$\underset{w_{*}^{\left(n\right)},b^{\left(n\right)},{\xi }^{\left(n\right)}}{\min }\frac{1}{2}{w_{*}^{\left(n\right)}}_F^2\prod _{1\le i\le N}^{i\ne n}{w_{*}^{\left(i\right)}}_F^2+C\sum _{m=1}^M{\xi }_m^{\left(n\right)}$ $y_m\left({\left(w_{*}^{\left(n\right)}\right)}^T\left(V_m\prod _{1\le i\le N}^{i\ne n}{\times }_i{w}_{*}^{\left(i\right)}\right)+b^{\left(n\right)}\right)\ge 1-{\xi }_m^{\left(n\right)}$ $S.t$ ${\xi }_m^{\left(n\right)}\ge 0m=1,2,\dots M$

wherein, w_*⁽ⁿ⁾ is the n-th order projection vector of the training tensor data set; w_*⁽ⁿ⁾=Λ^(n)1/2P^(n)Tw⁽ⁿ⁾, wherein Λ⁽ⁿ⁾ and P⁽ⁿ⁾ meet the equation, P^(n)T(E+ηS_w⁽ⁿ⁾)P⁽ⁿ⁾=Λ⁽ⁿ⁾; E is an identity matrix;

$V_m=X_m\prod _{i=1}^N{\times }_i{\left[{\left(P^{\left(i\right)}{\Lambda }^{\left(i\right)1/2}\right)}^{-1}\right]}^T$ is tensor input data obtained after tensor input data X_m in the training tensor data set is projected along each order; X_i is an i-mode multiplication operator; b⁽ⁿ⁾ is the n-th order offset scalar of the training tensor data set.

Furthermore, according to a formula

${W_{*}}_F^2=\prod _{n=1}^N{w_{*}^{\left(n\right)}}_F^2,$ and a formula (w_*⁽ⁿ⁾)^T(V_mΠ_1<i<N^i≠n×_iw_*⁽ⁱ⁾)=<W_*, V_m>, transforming the N vector modes of quadratic programming subproblems into the multiple quadratic programming subproblem under a single tensor mode. A constructed optimal frame of the objective function of the OPSTM problem meets that:

$\underset{W_{*}b,\xi }{\min }\frac{1}{2}{W_{*}}_F^2+C\sum _{m=1}^M{\xi }_m$ $y_m\left(\langle W_{*},V_m\rangle +b\right)\ge 1-{\xi }_{m}S.t{\xi }_m\ge 0m=1,2,\dots M$

wherein, < > is a transvection operator, and

${\xi }_m=\underset{n=1,2,\dots N}{\max }\left\{{\xi }_m^{\left(n\right)}\right\}.$

Furthermore, according to lagrangian multiplier method, an obtained dual problem of the optimal frame of the objective function is:

$\underset{\alpha }{\max }\sum _{m=1}^M{\alpha }_m-\frac{1}{2}\sum _{i,j=1}^M{\alpha }_i{\alpha }_j{y}_i{y}_j\langle V_i,V_j\rangle$ $\sum _{m=1}^M\left({\alpha }_m{y}_m\right)=0S.t0<a_m<Cm=1,2,\dots M;$

introducing the tensor rank one decomposition into the calculation of the tensor transvection. An obtained revised dual problem is:

$\underset{\alpha }{\max }\sum _{m=1}^M{\alpha }_m-\frac{1}{2}\sum _{i,j=1}^M\sum _{p=1}^R\sum _{q=1}^R{\alpha }_i{\alpha }_j{y}_i{y}_j\sum _{n=1}^N\langle v_{\mathrm{ip}}^{\left(n\right)},v_{\mathrm{jq}}^{\left(n\right)}\rangle$ $\sum _{m-1}^M\left({\alpha }_m{y}_m\right)=0S.t0<{\alpha }_m<Cm=1,2,\dots M.$

Furthermore, calculating the projection tensor W_* according to a formula,

$W_{*}=\sum _{m=1}^M{\alpha }_m{y}_m{V}_m.$

On the other hand, a computer device is provided. The computer device includes a storage medium with computer instructions stored therein. The computer instructions are configured to enable the computer device to execute a method for processing tensor data for pattern recognition for supervised learning under tensor mode; the method includes:

receiving an input training tensor data set;

introducing a within class scatter matrix into an objective function such that between class distance is maximized, at the same time, within class distance is minimized by the objective function;

constructing an optimal frame of an objective function of an optimal projection tensor machine subproblem;

transforming N vector modes of quadratic programming subproblems into a multiple quadratic programming problem under a single tensor mode, and constructing an optimal frame of an objective function of an optimal projection tensor machine problem;

obtaining a dual problem of the optimal frame of the objective function, introducing a tensor rank one decomposition into calculation of tensor transvection, and obtaining a revised dual problem according to lagrangian multiplier method;

utilizing sequential minimal optimization SMO algorithm to solve the revised dual problem and outputting an alagrangian optimal combination and an offset scalar b;

calculating a projection tensor W_*;

performing the rank one decomposition to the projection tensor W_*;

performing a back projection to a component obtained after performing the rank one decomposition to the projection tensor W_*;

performing rank one decomposition inverse operation to the component obtained after performing the back projection to obtain an optimal projection tensor W which is corresponded to the training tensor data set;

constructing a decision function construction phase and construct a decision function by the optimal projection tensor W which has been rank-one decomposed together with the offset scalar b; and

inputting to-be-predicted tensor data which has been rank-one decomposed into the decision function for prediction in an application prediction phase.

Furthermore, through an eta coefficient η, after the within class scatter introducing unit introduces the within class scatter matrix into an objective function of an STM subproblem, the objection function of the quadratic programming problem of the n-th subproblem is changed into:

$\underset{w^{\left(n\right)},b^{\left(n\right)},{\xi }^{\left(n\right)}}{\min }\frac{1}{2}\left[\left({w^{\left(n\right)}}_F^2+\eta \left({\left(w^{\left(n\right)}\right)}^T{S}_w^{\left(n\right)}{w}^{\left(n\right)}\right)\right)\prod _{1<i<N}^{i\ne n}\left({w^{\left(i\right)}}_F^2+\eta \left({\left(w^{\left(i\right)}\right)}^T{S}_w^{\left(n\right)}{w}^{\left(i\right)}\right)\right)\right]+C\sum _{m=1}^M{\xi }_m^{\left(n\right)}$

wherein, S_w⁽ⁿ⁾ is an n-order within class scatter matrix estimated after the training tensor data set is expanded along the n-th order; w⁽ⁿ⁾ is the n-th order optimal projection vector of the training tensor data, n=1, 2, . . . N; C is a penalty factor; ξ_m⁽ⁿ⁾ is a slack variable; eta coefficient η is configured to measure the importance of the within class scatter matrix.

Furthermore, in the subproblem optimal frame constructing unit, the optimal frame of the objective function of the OPSTM problem is a combination of N vector modes of quadratic programming problems, which respectively corresponds to a subproblem; wherein, a quadratic programming problem of the n-th subproblem is:

$\underset{w_{*}^{\left(n\right)},b^{\left(n\right)},{\xi }^{\left(n\right)}}{\min }\frac{1}{2}{w_{*}^{\left(n\right)}}_F^2\prod _{1\le i\le N}^{i\ne n}{w_{*}^{\left(i\right)}}_F^2+C\sum _{m=1}^M{\xi }_m^{\left(n\right)}$ $y_m\left({\left(w_{*}^{\left(n\right)}\right)}^T\left(V_m\prod _{1\le i\le N}^{i\ne n}{\times }_i{w}_{*}^{\left(i\right)}\right)+b^{\left(n\right)}\right)\ge 1-{\xi }_m^{\left(n\right)}$ $S.t$ ${\xi }_m^{\left(n\right)}\ge 0m=1,2,\dots M$

wherein, w_*⁽ⁿ⁾ is the n-th order projection vector of the training tensor data set; w_*⁽ⁿ⁾=Λ^(n)1/2P^(n)Tw⁽ⁿ⁾, wherein Λ⁽ⁿ⁾ and P⁽ⁿ⁾ meet the equation, P^(n)T(E+ηS_w⁽ⁿ⁾)P⁽ⁿ⁾=Λ⁽ⁿ⁾; E is an identity matrix;

$V_m=X_m\prod _{i=1}^N{\times }_i{\left[{\left(P^{\left(i\right)}{\Lambda }^{\left(i\right)1/2}\right)}^{-1}\right]}^T$ is tensor input data obtained after tensor input data X_m in the training tensor data set is projected along each order; X_i is an i-mode multiplication operator; b⁽ⁿ⁾ is the n-th order offset scalar of the training tensor data set.

Furthermore, according to a formula

${W_{*}}_F^2=\prod _{n=1}^N{w_{*}^{\left(n\right)}}_F^2,$ and a formula (w_*⁽ⁿ⁾)^T(V_mΠ_1<i<N^i≠n×_iw_*⁽ⁱ⁾)=<W_*, V_m>, the problem optimal frame constructing unit transforms the N vector modes of quadratic programming subproblems into the multiple quadratic programming subproblem under a single tensor mode. A constructed optimal frame of the objective function of the OPSTM problem meets that:

$\underset{W_{*}b,\xi }{\min }\frac{1}{2}{W_{*}}_F^2+C\sum _{m=1}^M{\xi }_m$ $y_m\left(\langle W_{*},V_m\rangle +b\right)\ge 1-{\xi }_m$ $S.t$ ${\xi }_m\ge 0m=1,2,\dots M$

wherein, < > is a transvection operator, and

${\xi }_m=\underset{n=1,2,\dots N}{\max }\left\{{\xi }_m^{\left(n\right)}\right\}.$

Furthermore, according to lagrangian multiplier method, the dual problem solving unit obtains the dual problem of the optimal frame of the objective function, which is:

$\underset{\alpha }{\max }\sum _{m=1}^M{\alpha }_m-\frac{1}{2}\sum _{i,j=1}^M{\alpha }_i{\alpha }_j{y}_i{y}_j\langle V_i,V_j\rangle$ $\sum _{m=1}^M\left({\alpha }_m{y}_m\right)=0S.t0<a_m<Cm=1,2,\dots M;$

the dual problem solving unit introduces the tensor rank one decomposition into the calculation of the tensor transvection. An obtained revised dual problem is:

$\underset{\alpha }{\max }\sum _{m=1}^M{\alpha }_m-\frac{1}{2}\sum _{i,j=1}^M\sum _{p=1}^R\sum _{q=1}^R{\alpha }_i{\alpha }_j{y}_i{y}_j\prod _{n=1}^N\langle v_{\mathrm{ip}}^{\left(n\right)},v_{\mathrm{jq}}^{\left(n\right)}\rangle \sum _{m=1}^M\left({\alpha }_m{y}_m\right)=0$ $S.t$ $0<{\alpha }_m<Cm=1,2,\dots M.$

Furthermore, the projection tensor calculating unit calculates the projection tensor W_* according to a formula,

$W_{*}=\sum _{m=1}^M{\alpha }_m{y}_m{V}_m.$

In the embodiments of the present invention, N vector modes of quadratic programming problems are transformed into a multiple quadratic programming problem under a single tensor mode. The transformed optimal frame of the objective function is the optimal frame of the objective function of the OPSTM problem. This can reduce the number of model parameters significantly, overcome issues such as curse of dimensionality, over learning and small amount of samples occurred when traditional vector mode algorithms process the tensor data, which ensures efficient processing, at the same time, highlights excellent classifying effects. Above all, the algorithms provided by the embodiments of the invention can process tensor data effectively and directly in tensor field, at the same time, possesses features of optimal classifying ability as well as strong practicability and popularization.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an implementation flow chart of a method for processing tensor data for pattern recognition of the invention;

FIG. 2 is a system for processing tensor data for pattern recognition of the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

In order to make the purposes, technical solutions and advantages of the present invention clearer, the invention is described hereinafter in further details with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are merely for explaining the invention, but not intended for limitation.

In one embodiment of the invention, receiving an input training tensor data set; introducing a within class scatter matrix into an objective function such that between class distance is maximized, at the same time, within class distance is minimized by the objective function; constructing an optimal frame of the objective function of an optimal projection tensor machine OPSTM subproblem; constructing an optimal frame of an objective function of an OPSTM problem; according to lagrangian multiplier method, obtaining a dual problem of the optimal frame of the objective function, introducing a tensor rank one decomposition into calculation of tensor transvection, and obtaining a revised dual problem; utilizing sequential minimal optimization SMO algorithm to solve the revised dual problem and output a lagrangian optimal combination and an offset scalar; calculating a projection tensor; performing the rank one decomposition to the projection tensor; performing a back projection to a component obtained after performing the rank one decomposition to the projection tensor; performing rank one decomposition inverse operation to the component obtained after performing the back projection to obtain an optimal projection tensor W which is corresponded to the training tensor data set; decision function construction phase: by the optimal projection tensor W which has been rank-one decomposed together with the offset scalar, constructing a decision function; application prediction phase: inputting to-be-predicted tensor data which has been rank-one decomposed into the decision function for prediction.

The implementation of the present invention will be described in detail with reference to specific embodiments:

The First Embodiment

FIG. 1 shows an implementation process of a method for processing tensor data for pattern recognition, which is an optimization method for supervised learning under tensor mode provided by the first embodiment of the present invention. The details are as follows:

Step 101, receiving an input training tensor data set.

In the embodiment of the invention, let the training tensor data set be {Xm, ym|m=1, 2 . . . M}, wherein X_m represents tensor input data, y_m∈{+1, −1} represents a label.

Take a gray level image for example, sample points are stored in a form of a second-order tensor (matrix), and all the sample points which are in a form of a column vector comprise an input data set. In a similar way, a label set is also a column vector; furthermore, the location of each label is corresponded to the location of the corresponding sample point.

$X=\left[\begin{array}{c}{X}_1\\ X_2\\ ⋮\\ X_M\end{array}\right]Y=\left[\begin{array}{c}{y}_1\\ y_2\\ ⋮\\ y_M\end{array}\right]$

Step 102, introducing a within class scatter matrix into an objective function such that between class distance is maximized, at the same time, within class distance is minimized by the objective function.

In the embodiment of the invention, the optimal frame of the objective function of the support tensor machine (STM) problem is a combination of N vector modes of quadratic programming problems, which respectively correspond to a subproblem, wherein, a quadratic programming problem of the n-th subproblem is:

$\begin{array}{cc}\underset{w^{\left(n\right)},b^{\left(n\right)},{\xi }^{\left(n\right)}}{\min }\frac{1}{2}{w^{\left(n\right)}}_F^2\prod _{1\le i\le N}^{i\ne n}{w^{\left(i\right)}}_F^2+C\sum _{m=1}^M{\xi }_m^{\left(n\right)}& \left(1\text{-}1\right)\\ y_m\left({\left(w^{\left(n\right)}\right)}^T\left(X_m\prod _{1\le i\le N}^{i\ne n}{\times }_i{w}^{\left(i\right)}\right)+b^{\left(n\right)}\right)\ge 1-{\xi }_m^{\left(n\right)}S.t& \left(1\text{-}2\right)\\ {\xi }_m^{\left(n\right)}\ge 0m=1,2,\dots M& \left(1\text{-}3\right)\end{array}$

wherein, w⁽ⁿ⁾: the n-th order optimal projection vector of the training tensor data set, n=1, 2, . . . N;

b⁽ⁿ⁾: the n-th order offset scalar of the training tensor data set, n=1, 2, . . . N;

C: a penalty factor;

ξ_m⁽ⁿ⁾: a slack variable.

After introducing the within class scatter matrix into an objective function of an STM subproblem, through an eta coefficient η, the objection function of the quadratic programming problem of the n-th subproblem is changed into:

$\begin{array}{cc}\underset{w^{\left(n\right)},b^{\left(n\right)},{\xi }^{\left(n\right)}}{\min }\frac{1}{2}\left[\left({w^{\left(n\right)}}_F^2+\eta \left({\left(w^{\left(n\right)}\right)}^T{S}_w^{\left(n\right)}{w}^{\left(n\right)}\right)\right)\prod _{1<i<N}^{i\ne n}\left({w^{\left(i\right)}}_F^2+\eta \left({\left(w^{\left(i\right)}\right)}^T{S}_w^{\left(n\right)}{w}^{\left(i\right)}\right)\right)\right]+C\sum _{m=1}^M{\xi }_m^{\left(n\right)}& \left(1\text{-}4\right)\end{array}$

wherein, S_w⁽ⁿ⁾ is the n-th order within class scatter matrix estimated after the training tensor data set is expanded along the n-th order; w⁽ⁿ⁾ herein possesses Fisher criterion effect, “maximum between class distance, minimum within class distance” at the n-th order of the training tensor data set; eta coefficient η is configured to measure the importance of the within class scatter.

Step 103, constructing an optimal frame of the objective function of an optimal projection tensor machine OPSTM subproblem.

In the embodiment of the invention, the optimal frame of an objective function of an optimal projection tensor machine OPSTM problem is a combination of N vector modes of quadratic programming problems, which respectively correspond to a subproblem; wherein, a quadratic programming problem of the n-th subproblem is:

$\begin{array}{cc}\underset{w_{*}^{\left(n\right)},b^{\left(n\right)},{\xi }^{\left(n\right)}}{\min }\frac{1}{2}{w_{*}^{\left(n\right)}}_F^2\prod _{1\le i\le N}^{i\ne n}{w_{*}^{\left(i\right)}}_F^2+C\sum _{m=1}^M{\xi }_m^{\left(n\right)}& \left(2\text{-}1\right)\\ y_m\left({\left(w_{*}^{\left(n\right)}\right)}^T\left(V_m\prod _{1\le i\le N}^{i\ne n}{\times }_i{w}_{*}^{\left(i\right)}\right)+b^{\left(n\right)}\right)\ge 1-{\xi }_m^{\left(n\right)}S.t& \left(2\text{-}2\right)\\ {\xi }_m^{\left(n\right)}\ge 0m=1,2,\dots M& \left(2\text{-}3\right)\end{array}$

wherein, w_*⁽ⁿ⁾: the n-th order projection vector of the training tensor data set; w_*⁽ⁿ⁾=Λ^(n)1/2P^(n)Tw⁽ⁿ⁾ n=1, 2, . . . N; w⁽ⁿ⁾: the n-th order optimal projection vector of the training tensor data set of formula (1-4); Λ⁽ⁿ⁾ and P⁽ⁿ⁾ meet the equation, P^(n)T(E+ηS_w⁽ⁿ⁾)P⁽ⁿ⁾=Λ⁽ⁿ⁾; E is an identity matrix;

$V_m=X_m\prod _{i=1}^N{\times }_i{\left[{\left(P^{\left(i\right)}{\Lambda }^{\left(i\right)1/2}\right)}^{-1}\right]}^T$ is tensor input data obtained after tensor input data X_m is projected along each order; X_i is an i-mode multiplication operator.

Step 104, transforming N vector modes of quadratic programming subproblems into a multiple quadratic programming problem under a single tensor mode, and constructing an optimal frame of an objective function of an OPSTM problem.

In the embodiment of the invention,

$\begin{array}{cc}{W_{*}}_F^2={w_{*}^{\left(1\right)}\circ w_{*}^{\left(2\right)}\circ \dots w_{*}^{\left(N\right)}}_F^2=\sum _{i_1=1}^{I_1}\sum _{i_2=1}^{I_2}\dots \sum _{i_N=1}^{I_N}{w}_{{*}_{i_1,i_2\dots i_N}}^2=\sum _{i_1=1}^{I_1}\sum _{i_2=1}^{I_2}\dots \sum _{i_N=1}^{I_N}{\left(w_{{*}_{i_1}}^{\left(1\right)}·w_{{*}_{i_2}}^{\left(2\right)}\dots w_{{*}_{i_N}}^{\left(N\right)}\right)}^2=\langle w_{*}^{\left(1\right)},w_{*}^{\left(1\right)}\rangle \langle w_{*}^{\left(2\right)},w_{*}^{\left(2\right)},\rangle \dots \langle w_{*}^{\left(N\right)},w_{*}^{\left(N\right)}\rangle =\prod _{n=1}^N{w_{*}^{\left(n\right)}}_F^2& \mathrm{Eq}.1\\ {\left(w_{*}^{\left(n\right)}\right)}^T\left(V_m\prod _{i<i<N}^{i\ne n}{\times }_i{w}_{*}^{\left(i\right)}\right)=V_m{\times }_1{w}_{*}^{\left(1\right)}{\times }_2{w}_{*}^{\left(2\right)}\times \dots {\times }_{\left(n-1\right)}{w}_{*}^{\left(n-1\right)}{\times }_n{w}_{*}^{\left(n\right)}\times w_{*}^{\left(n+1\right)}\dots {\times }_N{w}_{*}^{\left(N\right)}=\sum _{i_1=1}^{I_1}\sum _{i_2=1}^{I_2}\dots \sum _{i_N=1}^{I_N}{v}_{m,i_1,i_2,\dots i_N}{w}_{{*}_{i_1}}^{\left(1\right)}{w}_{{*}_{i_2}}^{\left(2\right)}\dots w_{{*}_{i_N}}^{\left(N\right)}=\sum _{i_1=1}^{I_1}\sum _{i_2=1}^{I_2}\dots \sum _{i_N=1}^{I_N}{v}_{m,i_1,i_2,\dots i_N}{w}_{{*}_{i_1,i_2,\dots i_N}}=\langle W_{*},V_m\rangle .& \mathrm{Eq}.2\end{array}$

wherein, ∥ ∥_F² represents a norm and “∘” represents an outer product operator. According to the formulas, Eq. 1 and Eq. 2,

${W_{*}}_F^2=\prod _{n=1}^N{w_{*}^{\left(n\right)}}_F^2,$ and (w_*⁽ⁿ⁾)^T(V_mΠ_1<i<N^i≠n×_iw_*⁽ⁱ⁾)=<W_*, V_m>. Therefore, vector mode of quadratic programming problems of N subproblems can be transformed into a multiple quadratic programming problem under a single tensor mode, which means the optimal frame of an objective function of an OPSTM problem is:

$\begin{array}{cc}\underset{W_{*},b,\xi }{\min }\frac{1}{2}{W_{*}}_F^2+C\sum _{m=1}^M{\xi }_m& \left(3\text{-}1\right)\\ S.t{y}_m\left(\langle W_{*},V_m\rangle +b\right)\ge 1-{\xi }_m& \left(3\text{-}2\right)\\ {\xi }_m\ge 0m=1,2,\dots M& \left(3\text{-}3\right)\end{array}$

wherein, W_* is a projection tensor; < > is a transvection operator, and

${\xi }_m=\underset{n=1,2,\dots N}{\max }\left\{{\xi }_m^{\left(n\right)}\right\}.$

Through the Eq. 1 and Eq. 2, N vector modes of quadratic programming problems are transformed into a multiple quadratic programming problem under a single tensor mode. The transformed optimal frame of the objective function is the optimal frame of the objective function of the OPSTM problem. This can reduce the number of model parameters significantly, overcome issues such as curse of dimensionality, over learning and small sample occurred when vector mode algorithms process the tensor data.

Step 105, according to lagrangian multiplier method, obtaining a dual problem of the optimal frame of the objective function, introducing a tensor rank one decomposition into calculation of tensor transvection, and obtaining a revised dual problem.

In the embodiment, according to lagrangian multiplier method, a dual problem of the optimal frame [(3-1), (3-2), (3-3)] of the objection function of the OPSTM problem is obtained, wherein α_m is a lagrangian multiplier.

$\begin{array}{cc}\underset{\alpha }{\max }\sum _{m=1}^M{\alpha }_m-\frac{1}{2}\sum _{i,j=1}^M{\alpha }_i{\alpha }_j{y}_i{y}_j\langle V_i,V_j\rangle & \left(4\text{-}1\right)\\ \sum _{m=1}^M\left({\alpha }_m{y}_m\right)=0S.t& \left(4\text{-}2\right)\\ 0<{\alpha }_m<Cm=1,2,\dots M& \left(4\text{-}3\right)\end{array}$

A tensor CP (CANDECOMP/PARAFAC) is decomposed and introduced to the calculation of tensor transvection.

Rank one decompositions of tensor data V_i, V_j are respectively:

$V_i=\sum _{r=1}^R{v}_{\mathrm{ir}}^{\left(1\right)}\circ v_{\mathrm{ir}}^{\left(2\right)}\circ \dots \circ v_{\mathrm{ir}}^{\left(N\right)}$ $V_j=\sum _{r=1}^R{v}_{\mathrm{jr}}^{\left(1\right)}\circ v_{\mathrm{jr}}^{\left(2\right)}\circ \dots \circ v_{\mathrm{jr}}^{\left(N\right)}\mathrm{and}$ $\langle V_i,V_j\rangle =\sum _{p=1}^R\sum _{q=1}^R\prod _{n=1}^N\langle v_{\mathrm{ip}}^{\left(n\right)},v_{\mathrm{jq}}^{\left(n\right)}\rangle$

Therefore, the dual problem can be changed into:

$\begin{array}{cc}\underset{\alpha }{\max }\sum _{m=1}^M{\alpha }_m-\frac{1}{2}\sum _{i,j=1}^M\sum _{p=1}^R\sum _{q=1}^R{\alpha }_i{\alpha }_j{y}_i{y}_j\prod _{n=1}^N\langle v_{\mathrm{ip}}^{\left(n\right)},v_{\mathrm{jq}}^{\left(n\right)}\rangle & \left(4\text{-}4\right)\\ \sum _{m=1}^M\left({\alpha }_m{y}_m\right)=0S.t& \left(4\text{-}2\right)\\ 0<{\alpha }_m<Cm=1,2,\dots M& \left(4\text{-}3\right)\end{array}$

In the objection function (4-1) of the dual problem, a tensor rank one decomposition auxiliary calculation,

$\langle V_i,V_j\rangle =\sum _{p=1}^R\sum _{q=1}^R\prod _{n=1}^N\langle v_{\mathrm{ip}}^{\left(n\right)},v_{\mathrm{jq}}^{\left(n\right)}\rangle ,$ is introduced into the tensor transvection calculation part, which further reduces calculation complexity and storage cost, at the same time, the tensor rank one decomposition can obtain a more compact and more meaningful representation of a tensor objection, extract structural information and internal correlation of the tensor data more effectively, and avoid a time-consuming alternative projection iterative process of the tensor mode algorithms of the prior art.

Step 106, utilizing sequential minimal optimization SMO algorithm to solve the revised dual problem and output an alagrangian optimal combination α=[α₁, α₂, . . . α_M] and an offset scalar b.

Step 107, calculating a projection tensor W_*.

In the embodiment of the invention, the projection tensor W_* is calculated according to the formula

$W_{*}=\sum _{m=1}^M{\alpha }_m{y}_m{V}_m.$

Step 108, performing the rank one decomposition to the projection tensor W_*.

In the embodiment of the invention, the rank one decomposition is performed to the projection tensor W_*, and the formula W_*=w_*⁽¹⁾∘w_*⁽²⁾∘ . . . w_*^(N) is obtained.

Step 109, performing a back projection to a component obtained after performing the rank one decomposition to the projection tensor W_*.

In the embodiment of the invention, the back projection to a component is performed after performing the rank one decomposition to the projection tensor W_* to obtain the formula w⁽ⁿ⁾=(Λ^(n)1/2P⁽ⁿ⁾)⁻¹w_*⁽ⁿ⁾, wherein w⁽ⁿ⁾ corresponds to the optimal projection vector of (1-4), and is the n-th order optimal projection vector of the training tensor data set, n=1, 2, . . . N.

Step 110, performing rank one decomposition inverse operation to the component obtained after performing the back projection to obtain an optimal projection tensor W which is corresponded to the training tensor data set.

In the embodiment of the invention, the components obtained after performing back projection are blended (rank one decomposition inverse operation) into the optimal projection tensor W, W=w⁽¹⁾∘w⁽²⁾∘ . . . w^(N). therefore, the optimal projection tensor W can embody Fisher criterion at each order.

Step 111, decision function construction phase: by the optimal projection tensor W which has been rank-one decomposed together with the offset scalar b, constructing a decision function.

In the embodiment of the invention, at the decision function construction phase, the rank one decomposition should be performed to the optimal projection tensor W, the decomposed optimal projection tensor W and the offset scalar b are used for constructing the decision function:

$y\left(X\right)=\mathrm{sign}\left[\prod _{n=1}^N\sum _{p=1}^R\sum _{q=1}^R<w_p^{\left(n\right)},x_q^{\left(n\right)}>+b\right].$

Step 112, at the application prediction phase, inputting to-be-predicted tensor data which has been rank-one decomposed into the decision function for prediction.

In the embodiment of the invention, at the application prediction phase, the to-be-predicted tensor data which has been performed rank-one decomposed is input into the decision function for prediction.

Compared with the prior art, the embodiment possesses the following advantages: 1) N vector modes of quadratic programming problems are transformed into a multiple quadratic programming problem under a single tensor mode. The optimal frame of the transformed objective function is the optimal frame of the objective function of the OPSTM problem. This can reduce the number of model parameters significantly, overcome issues such as curse of dimensionality, over learning and small sample occurred when traditional vector mode algorithms process the tensor data, which ensures high-activity processing, at the same time, highlights excellent classifying effects. Above all, the algorithms provided by the embodiments of the invention can process tensor data effectively and directly in tensor field, at the same time, possesses a feature of optimal classifying ability as well as strong practicability and popularization. 2) the within scatter matrix is introduced into the object function, which can receive and process the tensor data directly in the tensor field, and the output optimal projection tensor W can embody the Fisher criterion effect, “maximum between class distance, minimum within class distance” at each order. 3) In the objection function (4-1) of the dual problem, a tensor rank one decomposition auxiliary calculation,

$\langle V_i,V_j\rangle =\sum _{p=1}^R\sum _{q=1}^R\prod _{n=1}^N\langle v_{\mathrm{ip}}^{\left(n\right)},v_{\mathrm{jq}}^{\left(n\right)}\rangle ,$ is introduced into the tensor transvection calculation part, which further reduces calculation complexity and storage cost, at the same time, the tensor rank one decomposition can obtain a more compact and more meaningful representation of the tensor objection, and extract structural information and internal correlation of the tensor data more effectively, and avoid a time-consuming alternative projection iterative process.

It should be understood that the serial number of each step in this embodiment does not signify the execution sequence; the execution sequence of each step should be determined according to its function and internal logic, and should not form any limitation to the implementation process of the embodiment of the invention.

It should be understood by those skilled in the art that the whole or partial steps of the methods in each embodiment can be achieved by relevant hardware instructed by program, and the corresponding program can be stored in a computer-readable storage medium; the storage medium can be for example ROM/RAM, magnetic disk or light disk, etc.

The Second Embodiment

FIG. 2 shows a specific structural block diagram of a system for processing tensor data for pattern recognition provided by the second embodiment of the invention. For illustration purposes, merely the part relevant to the embodiment of the invention is shown. The optimization system 2 for supervised learning under tensor mode includes: a data receiving unit 21, a within class scatter introducing unit 22, a subproblem optimal frame constructing unit 23, a problem optimal frame constructing unit 24, a dual problem obtaining unit 25, a dual problem solving unit 26, a projection tensor calculating unit 27, a projection tensor decomposition unit 28, a back projection unit 29, an optimal projection tensor calculating unit 210, a decision function constructing unit 211 and a predicting unit 212.

The data receiving unit 21 is configured to receive an input training tensor data set;

The within class scatter introducing unit 22 is configured to introduce a within class scatter matrix into an objective function such that between class distance is maximized, at the same time, within class distance is minimized by the objective function;

The subproblem optimal frame constructing unit 23 is configured to construct an optimal frame of an objective function of an optimal projection tensor machine OPSTM subproblem;

The problem optimal frame constructing unit 24 is configured to transform N vector modes of quadratic programming subproblems into a multiple quadratic programming problem under a single tensor mode, and construct an optimal frame of an objective function of an OPSTM problem;

The dual problem obtaining unit 25 is configured to obtain a dual problem of the optimal frame of the objective function, introduce tensor rank one decomposition into calculation of tensor transvection, and obtain a revised dual problem according to lagrangian multiplier method;

The dual problem solving unit 26 is configured to utilize sequential minimal optimization SMO algorithm to solve the revised dual problem and output a lagrangian optimal combination and an offset scalar b;

The projection tensor calculating unit 27 is configured to calculate a projection tensor W_*;

The projection tensor decomposition unit 28 is configured to perform the rank one decomposition to the projection tensor W_*;

The back projection unit 29 is configured to perform a back projection to a component obtained after performing the rank one decomposition to the projection tensor W_*;

The optimal projection tensor calculating unit 210 is configured to perform rank one decomposition inverse operation to the component obtained after performing the back projection to obtain an optimal projection tensor W which is corresponded to the training tensor data set;

The decision function constructing unit 211 is configured to construct a decision function construction phase and construct a decision function by the optimal projection tensor W which has been rank-one decomposed together with the offset scalar b;

The predicting unit 212 is configured to input to-be-predicted tensor data which has been rank-one decomposed into the decision function for prediction in the application prediction phase.

Furthermore, through an eta coefficient η, after the within class scatter introducing unit 22 introduces the within class scatter matrix into an objective function of an STM subproblem, the objection function of the quadratic programming problem of the n-th subproblem is changed into:

$\underset{w^{\left(n\right)},b^{\left(n\right)},{\xi }^{\left(n\right)}}{\min }\frac{1}{2}\left[\left({w^{\left(n\right)}}_F^2+\eta \left({\left(w^{\left(n\right)}\right)}^T{S}_w^{\left(n\right)}{w}^{\left(n\right)}\right)\right)\prod _{1<i<N}^{i\ne n}\left({w^{\left(i\right)}}_F^2+\eta \left({\left(w^{\left(i\right)}\right)}^T{S}_w^{\left(n\right)}{w}^{\left(i\right)}\right)\right)\right]+C\sum _{m=1}^M{\xi }_m^{\left(n\right)}$

wherein, S_w⁽ⁿ⁾ is an n-order within class scatter matrix estimated after the training tensor data set is expanded along the n-th order; w⁽ⁿ⁾ is the n-th order optimal projection vector of the training tensor data, n=1, 2, . . . N; C is a penalty factor; ξ_m⁽ⁿ⁾ is a slack variable; eta coefficient η is configured to measure the importance of the within class scatter matrix.

Furthermore, in the subproblem optimal frame constructing unit 23, the optimal frame of the objective function of the OPSTM problem is a combination of N vector modes of quadratic programming problems, which respectively corresponds to a subproblem; wherein a quadratic programming problem of the n-th subproblem is:

$\underset{w_{*}^{\left(n\right)},b^{\left(n\right)},{\xi }^{\left(n\right)}}{\min }\frac{1}{2}{w_{*}^{\left(n\right)}}_F^2\prod _{1\le i\le N}^{i\ne n}{w_{*}^{\left(i\right)}}_F^2+C\sum _{m=1}^M{\xi }_m^{\left(n\right)}$ $S.t.y_m\left({\left(w_{*}^{\left(n\right)}\right)}^T\left(V_m\prod _{1\le i\le N}^{i\ne n}{\times }_i{w}_{*}^{\left(i\right)}\right)+b^{\left(n\right)}\right)\ge 1-{\xi }_m^{\left(n\right)}{\xi }_m^{\left(n\right)}\ge 0m=1,2,\dots M$ wherein, w_*⁽ⁿ⁾ is the n-th order projection vector of the training tensor data set; w_*⁽ⁿ⁾=Λ^(n)1/2P^(n)Tw⁽ⁿ⁾, wherein Λ⁽ⁿ⁾ and P⁽ⁿ⁾ meet the equation, P^(n)T(E+ηS_w⁽ⁿ⁾)P⁽ⁿ⁾=Λ⁽ⁿ⁾; E is an identity matrix;

$V_m=X_m\prod _{i=1}^N{\times }_i{\left[{\left(P^{\left(i\right)}{\Lambda }^{\left(i\right)1/2}\right)}^{-1}\right]}^T$ is tensor input data obtained after tensor input data X_m in the training tensor data set is projected along each order; X_i is an i-mode multiplication operator; b⁽ⁿ⁾ is the n-th order offset scalar of the training tensor data set.

Furthermore, according to a formula

${W_{*}}_F^2=\prod _{n=1}^N{w_{*}^{\left(n\right)}}_F^2,$ and a formula (w_*⁽ⁿ⁾)^T(V_mΠ_1<i<N^i≠n×_iw_*⁽ⁱ⁾)=<W_*, V_m>, the problem optimal frame constructing unit 24 transforms the N vector modes of quadratic programming subproblems into the multiple quadratic programming subproblem under a single tensor mode. A constructed optimal frame of the objective function of the OPSTM problem meets that:

$\underset{W_{*},b,\xi }{\min }\frac{1}{2}{W_{*}}_F^2+C\sum _{m=1}^M{\xi }_m$ $S.t.y_m\left(\langle W_{*},V_m\rangle +b\right)\ge 1-{\xi }_m{\xi }_m\ge 0m=1,2,\dots M$

wherein, < > is a transvection operator, and

${\xi }_m=\underset{n=1,2,\dots N}{\max }\left\{{\xi }_m^{\left(n\right)}\right\}.$

Furthermore, according to lagrangian multiplier method, the dual problem solving unit 26 obtains the dual problem of the optimal frame of the objective function, which is:

$\underset{\alpha }{\max }\sum _{m=1}^M{\alpha }_m-\frac{1}{2}\sum _{i,j=1}^M{\alpha }_i{\alpha }_j{y}_i{y}_j\langle V_i,V_j\rangle$ $S.t.\sum _{m=1}^M\left({\alpha }_m{y}_m\right)=0$ $0<{\alpha }_m<Cm=1,2,\dots M.$

The dual problem solving unit 26 introduces the tensor rank one decomposition into the calculation of the tensor transvection. An obtained revised dual problem is:

$\underset{\alpha }{\max }\sum _{m=1}^M{\alpha }_m-\frac{1}{2}\sum _{i,j=1}^M\sum _{p=1}^R\sum _{q=1}^R{\alpha }_i{\alpha }_j{y}_i{y}_j\prod _{n=1}^N\langle V_{\mathrm{ip}}^{\left(n\right)},V_{\mathrm{jq}}^{\left(n\right)}\rangle$ $S.t.\sum _{m=1}^M\left({\alpha }_m{y}_m\right)=0$ $0<{\alpha }_m<Cm=1,2,\dots M.$

Furthermore, the projection tensor calculating unit 27 calculates the projection tensor W_* according to a formula,

$W_{*}=\sum _{m=1}^M{\alpha }_m{y}_m{V}_m.$

The optimization system for supervised learning under tensor mode provided by the embodiment of the invention can be applied to the corresponding method of the first embodiment. Please refer to the description of the first embodiment, which would not be further described herein.

Those skilled in the art should understand that the exemplary units and algorithm steps described in accompany with the embodiments disclosed in the specification can be achieved by electronic hardware, or the combination of computer software with electronic hardware. Whether these functions are executed in a hardware manner or a software manner depends on the specific applications and design constraint conditions of the technical solutions. With respect to each specific application, a professional technician can achieve the described functions utilizing different methods, and these achievements should not be deemed as going beyond the scope of the invention.

Those skilled in the art can be clearly understood that for convenience and briefness, the specific working process of the described system, apparatus and unit can refer to the corresponding process of the above method embodiment, which would be further described herein.

It should be understood that the systems, devices and methods disclosed in several embodiments provided by the present application can be achieved in alternative ways. For example, the described device embodiments are merely schematically. For example, the division of the units is merely a division based on logic function, whereas the units can be divided in other ways in actual realization; for example, a plurality of units or components can be grouped or integrated into another system, or some features can be omitted or not executed. Furthermore, the shown or discussed mutual coupling or direct coupling or communication connection can be achieved by indirect coupling or communication connection of some interfaces, devices or units in electric, mechanical or other ways.

The units described as isolated elements can be or not be separated physically; an element shown as a unit can be or not be physical unit, which means that the element can be located in one location or distributed at multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the schemes of the embodiments.

Furthermore, each functional unit in each embodiment of the present invention can be integrated into a processing unit, or each unit can exist in isolation, or two or more than two units can be integrated into one unit.

If the integrated unit is achieved in software functional unit and sold or used as an independent product, the integrated unit can be stored in a computer-readable storage medium. Based on this consideration, the substantial part, or the part that is contributed to the prior art of the technical solution of the present invention, or part or all of the technical solutions can be embodied in a software product. The computer software product is stored in a storage medium, and includes several instructions configured to enable a computer device (can be a personal computer, device, network device, and so on) to execute all or some of the steps of the method of each embodiment of the present invention. The storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or a light disk, and other various mediums which can store program codes.

The above contents are merely specific embodiments of the present invention, however, the protection scope of the present invention should not be limited by this. Any person skilled in the art can easily envisage alternations and displacements within the technical scope disclosed by the invention, which should also be within the protection scope of the present invention. Therefore, the protection scope of the present invention should be subjected to the protection scope of the claims.

Claims

1. A method for processing tensor data for pattern recognition, wherein the method comprises: receiving an input training tensor data set;introducing a within class scatter matrix into an objective function such that between class distance is maximized, at the same time, within class distance is minimized by the objective function;constructing an optimal frame of the objective function of an optimal projection tensor machine subproblem;transforming N vector modes of quadratic programming subproblems into a multiple quadratic programming problem under a single tensor mode, and constructing an optimal frame of an objective function of an optimal projection tensor machine problem;according to lagrangian multiplier method, obtaining a dual problem of the optimal frame of the objective function, introducing a tensor rank one decomposition into calculation of tensor transvection, and obtaining a revised dual problem;utilizing sequential minimal optimization algorithm to solve the revised dual problem and output an alagrangian optimal combination and an offset scalar b;calculating a projection tensor W*;performing the rank one decomposition to the projection tensor W*;performing a back projection to a component obtained after performing the rank one decomposition to the projection tensor W*;performing rank one decomposition inverse operation to the component obtained after performing the back projection to obtain an optimal projection tensor W which is corresponded to the training tensor data set;decision function construction phase: by the optimal projection tensor W which has been rank-one decomposed together with the offset scalar b, constructing a decision function;application prediction phase: inputting to-be-predicted tensor data which has been rank-one decomposed into the decision function for prediction.

2. A computer device, comprising a storage medium with computer instructions stored thereon, wherein the computer instructions are configured to enable the computer device to execute a method for processing tensor data for pattern recognition, wherein the method comprises: receiving an input training tensor data set;introducing a within class scatter matrix into an objective function such that between class distance is maximized, at the same time, within class distance is minimized by the objective function;constructing an optimal frame of an objective function of an optimal projection tensor machine subproblem;transforming N vector modes of quadratic programming subproblems into a multiple quadratic programming problem under a single tensor mode, and constructing an optimal frame of an objective function of an optimal projection tensor machine problem;obtaining a dual problem of the optimal frame of the objective function, introducing a tensor rank one decomposition into calculation of tensor transvection, and obtaining a revised dual problem according to lagrangian multiplier method;utilizing sequential minimal optimization algorithm to solve the revised dual problem and outputting an alagrangian optimal combination and an offset scalar b;calculating a projection tensor W*;performing the rank one decomposition to the projection tensor W*;performing a back projection to a component obtained after performing the rank one decomposition to the projection tensor W*;performing rank one decomposition inverse operation to the component obtained after performing the back projection to obtain an optimal projection tensor W which is corresponded to the training tensor data set;constructing a decision function by the optimal projection tensor W which has been rank-one decomposed together with the offset scalar b; andinputting to-be-predicted tensor data which has been rank-one decomposed into the decision function for prediction in an application prediction phase.