METHOD FOR CLUSTERING DATA BASED CONVEX OPTIMIZATION

Abstract

A method for clustering data based convex optimization is provided. The method includes the steps of: obtaining an optimal feasible solution that satisfies given strong duality using convex optimization for an objective function; and clustering data by extracting eigenvalue from the obtained optimal feasible solution.

Description

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a further understanding of the invention, are incorporated in and constitute a part of this application, illustrate embodiments of the invention and together with the description serve to explain the principle of the invention. In the drawings:

FIG. 1 is an overall flowchart illustrating a method for clustering data based on convex optimization according to an embodiment of the present invention;

FIG. 2 is a flowchart illustrating the optimization step using semidefinite programming for obtaining an optimal feasible matrix in the method for clustering data using convex optimization according to an embodiment of the present invention;

FIG. 3 is a flowchart illustrating the clustering step from the optimal feasible matrix in the method for clustering data using convex optimization according to an embodiment of the present invention; and

FIG. 4 is a diagram illustrating a simulation result for clustering data for graph multi-way partition that satisfies uniform distribution strong duality defined by a user based on FIG. 1 to FIG. 3.

DETAILED DESCRIPTION OF THE INVENTION

Reference will now be made in detail to the preferred embodiments of the present invention, examples of which are illustrated in the accompanying drawings.

Hereinafter, a method and system for semidefinite spectral clustering via convex programming according to an embodiment of the present invention will be described with reference to accompanying drawings.

FIG. 1 is an overall flowchart illustrating a method for clustering data based on convex optimization according to an embodiment of the present invention.

That is, FIG. 1 shows an overall framework for an objective function related to graph multi-way partitioning and semidefinite spectral clustering from the corresponding objective function.

Although a well-known conventional spectral clustering method also uses graph partitioning that is an object of the present invention, the clustering method according to the present embodiment is different therefrom in a relaxation method. The conventional spectral clustering method using a spectral relaxation method groups data with adjacent clusters using the eigenvectors of an affinity matrix that represents similarity or a graph Laplacian generated from data. On the contrary, the semidefinite spectral clustering method according to the present embodiment clusters data using the eigenvectors of an optima feasible Solution that is obtained to determine whether given strong duality for semidefinite relaxation is satisfied or not. That is, since the semidefinite relaxation makes it possible to obtain a globally optimal solution in various combination problems such as graph multi-way partition, the semidefinite relaxation is used in the clustering method according to the present embodiment.

As shown in FIG. 1, the semidefinite spectral clustering method according to the present embodiment includes the object function defining step S1 for defining an object function, the optimization steps S2 and S3 for calculating a globally optimal solution through semidefinite programming for graph multi-way partitioning of the objective function, and the clustering step S4 for clustering data using a general clustering method with the globally optimal solution at step S4.

The optimization steps S2 and S3 are steps for obtaining the globally optimal solution that satisfies strong duality and an object function which are defined by a user. In more detail, an optimal feasible matrix is calculated using semidefinite programming at step S2, and an optimal partition matrix is calculated from the optimal feasible matrix at step S3. The optimization steps S2 and S3 will be described in more detail with reference to FIG. 2 in later.

The clustering step S4 is the last step that clusters data using the optimal feasible matrix obtained from the optimization step. The clustering step S4 will be described in more detail with reference to FIG. 3.

The object function is defined as arg_x min tr(X^T LX).

Herein, X denotes an optimal partition matrix, L is a graph Laplacian, and T denotes the transpose of a matrix.

In order to cluster data, clustering methods including k-means, EM, or k-nn may be used.

The optimal feasible solution is defined based on the similarity or the difference between data. When the affinity matrix or the difference matrix of the data is generated, it is preferable to use a kernel function. Herein, the object of the optimization is to obtain the optimal feasible solution that satisfies the given strong duality. All solutions in a range of satisfying the given strong duality are feasible solutions, and one having the height value or the smallest value among the feasible solutions is the optimal feasible solution. It is preferable to extract feature points from the data for generating the affinity matrix and the difference matrix of the data. It is further preferable to apply the affinity matrix and the difference matrix to identical data or different data.

FIG. 2 is a flowchart illustrating the optimization step using semidefinite programming for obtaining an optimal feasible matrix in the method for clustering data using convex optimization according to an embodiment of the present invention.

The flowchart shown in FIG. 2 is a framework corresponding to the steps S2 and S3 of FIG. 1, which illustrates the step for calculating a globally optimal feasible matrix using semidefinite programming that is one of convex optimization methods.

As shown in FIG. 2, Lagrangian that satisfies the objective function and the strong duality defined by a user is obtained at steps S11 and S12, and a dual function is obtained based on the obtained Lagrangian at step S13. Then, a standard SDP form of basic semidefinite program is obtained using the obtained dual function and the other features such as self-duality and minmax inequality at step S14.

Herein, it is determined whether a relaxed standard semidefinite programming satisfies the strong duality or not at step S15. Herein, the relaxed standard SDP is a function relaxed through semidefinite programming which is one of convex programs. If the strong duality is not satisfied by the relaxed stand SDP, the optimal solution is obtained based on a barycenter-based method using the barycenter matrix of convex hull for partition matrices at step S16. If the strong duality is satisfied by the relaxed stand SDP, the optimal solution is calculated using an interior-point method that is one of Newton's methods as a technique for solving a linear equality constrained optimization problem at step S17. Herein, the interior-point method solves an optimization problem with linear equality and inequality constraints by reducing it to a sequence of linear equality constrained problems.

FIG. 3 is a flowchart illustrating the clustering step from the optimal feasible matrix in the method for clustering data using convex optimization according to an embodiment of the present invention.

The flowchart shown in FIG. 3 is framework corresponding to the clustering step S4 in FIG. 1. As shown in FIG. 3, the clustering result is obtained at step S23 by applying conventional clustering methods such as k-means at step S22 from the optimal feasible solution obtained through the semidefinite programming at step S21.

FIG. 4 is a diagram illustrating a simulation result for clustering data for graph multi-way partition that satisfies uniform distribution strong duality defined by a user based on FIG. 1 to FIG. 3.

A clustering simulation is performed by making the structure of matrix directly related to the generation of eigenvector to have a block diagonal structure using the semidefinite relaxation and forming principle vectors, the 1^st column vector, and the 2^nd column vector, obtained from the optimal feasible matrix, and the clustering result of the clustering simulation (sample data set) is illustrated in FIG. 4. In FIG. 4, 7 and X are used to easily distinguish each clustered data. Like the clustering simulation results shown in FIG. 4, the method for semidefinite spectral clustering based on convex optimization according to the present embodiment can provide the reliable clustering performance.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention. Thus, it is intended that the present invention covers the modifications and variations of this invention provided they come within the scope of the appended claims and their equivalents.

As described above, the method for clustering data using convex optimization according to the present invention can be used in various fields where vast data are classified and analyzed. Such an automation process can save huge resources such as time and man power. Also, the method for clustering data using convex optimization according to the present invention can simultaneously cluster not only homogenous data but also heterogeneous data. Therefore, useful data can be provided to a user. Furthermore, the method for clustering data using convex optimization according to the present invention can provide the reliable clustering performance by overcoming the heuristic limitation of the conventional clustering methods through the convex optimization.

Claims

1. A method for clustering data based on convex optimization comprising the steps of: obtaining an optimal feasible solution that satisfies given strong duality using convex optimization for an objective function; andclustering data by extracting eigenvalue from the obtained optimal feasible solution.

2. The method of claim 1, wherein semidefinite relaxation is used as the convex optimization.

3. The method of claim 2, wherein semidefinite relaxation includes the steps of: a) obtaining a dual function by obtaining a Lagrangian that satisfy the objective function and the strong duality;b) determining whether the storing duality is satisfied by relaxed standard semidefinite programming obtained by relaxing the semidefinite programming; andc) obtaining an optimal partition matrix through an interior-point method if the strong duality is satisfied.

4. The method of claim 3, wherein an optimal partition matrix is calculated using a barycenter-based method with a barycenter matrix of a convex hull for partition matrices if the strong duality is not satisfied.

5. The method of anyone of claims 3 and 4, wherein the objective function is argx min tr(XT LX), where X denotes an optimal partition matrix, L is a graph Laplacian, and T denotes the transpose of a matrix.

6. The method of claim 1, wherein clustering methods including k-means, EM, and k-nn are applied for clustering.

7. The method of claim 1, wherein the optimal feasible solution defines similarity and difference between data.

8. The method of claim 1, wherein a kernel function is used when an affinity matrix or a difference matrix of the data is generated.

9. The method of claim 8, wherein feature points are extracted from the data to generate the affinity matrix and the difference matrix of the data.

10. The method of anyone of claims 7 to 9, wherein the affinity matrix or the difference matrix is applied to homogenous data or heterogeneous data.