Clustering apparatus, clustering method, and clustering program

Abstract

In a clustering apparatus comprising an input unit (1) supplied with a dataset including a plurality of samples, a data processing unit (4) for processing the samples to classify each sample into a class, and an output unit (3) for producing a processing result representative of classification, a parameter memory (51) in a memory unit (5) memorizes a target parameter obtained from past experiment. A parameter estimating section (24) of the data processing unit estimates a clustering parameter by the use of the target parameter memorized in the parameter memory. An unidentifiable sample detecting section (25) of the data processing unit detects a sample as an unidentifiable sample if posterior probabilities calculated for the sample by a probability density function produced by the clustering parameter estimated by the parameter estimating section are smaller than a predetermined value.

Description

[0001] This application claims priority to prior Japanese application JP 2003-58511, the disclosure of which is incorporated herein by reference.

BACKGROUND OF THE INVENTION

[0002] This invention relates to a clustering apparatus, a clustering method, and a clustering program and, in particular, to a clustering apparatus, a clustering method, and a clustering program each of which is based on normal mixture distribution.

[0003] As a typical statistical clustering technique, K-means clustering is known. In addition, a clustering method based on normal mixture distribution is also known.

[0004] Referring to FIG. 1, the K-means clustering will be described. At first, a dataset including a plurality of samples is given (step A1). The number k of classes or clusters is determined (step A2). From all the samples, k samples are extracted at random and used as initial class centers for k classes, respectively (step A3). Next, for each of all the samples, the distance from each of the class centers, k in number, is calculated and each sample is classified into a particular class having the class center closest from the sample (steps A4 and A5). From those samples classified in each class, a new class center is calculated as an average value (step A6). Until the total of the distances between the new class centers and the former class centers becomes equal to or smaller than a predetermined value, the steps of calculating the distance from the class center and classifying the sample into a new class are repeated for all of the samples (step A7). After completion of the classification, the result is produced (step A8).

[0005] In the clustering method based on normal mixture distribution, parameter estimation is generally carried out by the use of the maximum likelihood method. By the use of an estimated clustering parameter, each sample is clustered according to the Bayes rule. In the clustering method based on normal mixture distribution, posterior probabilities are calculated by a probability density function given by Equation (1) to determine the class into which the sample is to be classified. 1$\begin{array}{cc}f\left(x;\theta \right)=\sum _{j=1}^k{\omega }_j\phi \left(x;{\mu }_j,{\sigma }_j^2\right)& \left(1\right)\end{array}$

[0006] Herein, x represents a sample, ωj, a weight parameter, φ(x; μj, σ2j), the probability density function of normal distribution having an average μj and a variance σ2j, θ=(ωj, μj, σ2j), a clustering parameter collectively representing the weight, the average, and the variance. k represents the number of classes.

[0007] In the clustering method based on normal mixture distribution, it is necessary to estimate the clustering parameter θ contained in Equation (1). The maximum likelihood method is well known as a classical method for parameter estimation and is often used in parameter estimation by the clustering method based on normal mixture distribution. The maximum likelihood method is a method of estimating the clustering parameter θ which maximizes a logarithmic likelihood function L(θ) as a function of the clustering parameter θ. The logarithmic likelihood function L(θ) is given by Equation (2). 2$\begin{array}{cc}L\left(\theta \right)=\frac{1}{n}\sum _{i=1}^n\log f\left(x_i,\theta \right)& \left(2\right)\end{array}$

[0008] Herein, n represents the number of samples.

[0009] For each sample, posterior probabilities pj in the respective classes are calculated by the use of the estimated clustering parameter θ according to Equation (3) and the class to which the sample is to be classified is determined with reference to the posterior probabilities pj. 3$\begin{array}{cc}{p}_j=\frac{{\omega }_j\phi \left(x;{\mu }_j,{\sigma }_j^2\right)}{f\left(x;\theta \right)}& \left(3\right)\end{array}$

[0010] Referring to FIGS. 2 and 3, a clustering apparatus for realizing the clustering method based on normal mixture distribution will be described.

[0011] As illustrated in FIG. 2, the clustering apparatus comprises an input unit 1 such as a keyboard, a data processing unit 2 operable under control of a program, and an output unit 3 such as a display or a printer.

[0012] The data processing unit 2 comprises a parameter estimating section 21, an outlier detecting section 22, and a clustering section 23.

[0013] Referring to FIG. 3, description will be made of the clustering method based on normal mixture distribution executed in the clustering apparatus illustrated in FIG. 2.

[0014] At first, the input unit 1 is supplied with a dataset including a plurality of samples (step B1). Various parameters are initialized (step B2). By the use of the dataset supplied in the step B2 and the parameters initialized in the step B2, the parameter estimating section 21 estimates the clustering parameter θ in accordance with Equation (2) (step B3). By the use of the clustering parameter θ estimated in the step B3, the parameter estimating section 21 calculates the probability density function given by Equation (1) (step B4).

[0015] The outlier detecting section 22 judges whether or not each sample is present within a predetermined confidence interval of the probability density function to thereby judge an outlier (step B5). If the sample is not present in the confidence interval, the outlier detecting section 22 detects the sample as the outlier (step B8).

[0016] For each sample which has not been detected as the outlier in the outlier detecting section 22, the clustering section 23 calculates the posterior probabilities pj in Equation (3) (step B6). The clustering section 23 determines a class j which, according to Equation (3), gives the maximum posterior probability pj for the sample and classifies the sample into the class j (step B7). These steps B5 to B8 in FIG. 3 are repeated for all the samples (step B9). After completion of clustering, the result is produced (step B10).

[0017] The existing technique has five disadvantages which will presently be described.

[0018] As a first disadvantage, clustering is difficult in case where the number of classes is unknown for the dataset to be subjected to clustering. This is because, in the K-means clustering and the clustering method based on normal mixture distribution using the maximum likelihood method, stable and reliable clustering is difficult unless the number of classes is known.

[0019] As a second disadvantage, the K-means clustering can not detect, as an unidentifiable or unclusterable sample, a sample whose class is ambiguous and indefinite. This is because the K-means clustering has no more than a function of unexceptionally classifying every sample to any one of the classes.

[0020] As a third disadvantage, improper clustering may be carried out due to the presence of an outlier. This is because both of the K-means clustering and the clustering method based on normal mixture distribution using the maximum likelihood method are not robust against the outlier but is seriously affected by the outlier.

[0021] As a fourth disadvantage, classification into a proper class is difficult in the clustering method based on normal mixture distribution using the maximum likelihood method in case where only one sample belongs to a particular class. This is because, in the clustering method based on normal mixture distribution using the maximum likelihood method, an estimated value of the clustering parameter θ can not be obtained if only one sample belongs to a particular class.

[0022] As a fifth disadvantage, the K-means clustering may carry out improper clustering in case where respective classes are different in data spread or variation. This is because the K-means clustering does not have a function of coping with the difference in data spread or variation.

SUMMARY OF THE INVENTION

[0023] It is a first object of this invention to provide a clustering technique and a clustering system capable of properly carrying out clustering even if the number of classes is unknown.

[0024] It is a second object of this invention to provide a clustering technique and a clustering system capable of detecting, as an unidentifiable sample, a sample whose class is ambiguous and indefinite.

[0025] It is a third object of this invention to provide a clustering technique and a clustering system capable of carrying out clustering robust against an outlier which may be contained in samples.

[0026] It is a fourth object of this invention to provide a clustering technique and a clustering system capable of properly carrying out clustering, even if only one sample belongs to a particular class, without judging the sample as an outlier.

[0027] It is a fifth object of this invention to provide a clustering technique and a clustering system capable of properly carrying out clustering even if respective classes are different in data spread or variation.

[0028] According to a first aspect of this invention, there is provided a clustering apparatus comprising an input unit supplied with a dataset including a plurality of samples, a data processing unit for processing the samples supplied from the input unit to classify each sample into a class, and an output unit for producing a processing result representative of classification carried out in the data processing unit, the clustering apparatus further comprising a parameter memory for memorizing a target parameter obtained from past experiments, the data processing unit comprising a parameter estimating section for estimating a clustering parameter by the use of the target parameter memorized in the parameter memory. Herein, the parameter estimating section estimates the clustering parameter by the use of a modified likelihood function which is robust against an outlier.

[0029] Thus, the parameter memory in this invention memorizes the target parameter obtained from past experiments. By using the target parameter in the parameter estimating section, past parameters are utilized in clustering. With the above-mentioned structure, the number of classes can be determined to be an appropriate value. The parameter estimating section adopts the modified likelihood function as a technique robust against an outlier. Thus, it is possible to achieve the first, the third, and the fourth objects of this invention.

[0030] In the clustering apparatus according to the first aspect, it is preferable that the data processing unit further comprises an unidentifiable sample detecting section for detecting a particular sample as an unidentifiable sample if posterior probabilities calculated for the particular sample by a probability density function produced by the clustering parameter estimated by the parameter estimating section are smaller than a predetermined value.

[0031] Thus, the unidentifiable sample detecting section in this invention detects the particular sample as the unidentifiable sample if the posterior probabilities to be used in clustering are smaller than the predetermined value. By adopting the above-mentioned structure, a particular sample whose class is ambiguous and indefinite can be detected as an unidentifiable sample. Thus, it is possible to achieve the second object of this invention.

[0032] In the clustering apparatus according to the first aspect, it is preferable that the data processing unit further comprises:

[0033] an outlier detecting section for detecting, by the use of a probability density function produced by an estimated parameter estimated by the parameter estimating section, a particular sample as an outlier if the particular sample is deviated from a predetermined confidence interval,

[0034] an unidentifiable sample detecting section for detecting, from those samples which have not been detected as the outlier in the outlier detecting section, a specific sample as an unidentifiable or unclusterable sample if posterior probabilities calculated by the probability density function for the specific sample are smaller than a predetermined probability value, and

[0035] a clustering section for classifying each sample which has not been detected as the outlier or the unidentifiable sample in the outlier detecting section or the unidentifiable sample detecting section into each class by the use of the posterior probabilities.

[0036] In the clustering apparatus according to the first aspect, it is preferable that the data processing unit further comprises:

[0037] an unidentifiable sample detecting section for detecting a particular sample as an unidentifiable sample if posterior probabilities calculated for the particular sample by a probability density function produced by an estimated parameter estimated by the parameter estimating section are smaller than a predetermined probability value,

[0038] an outlier detecting section for detecting, from those samples which have not been detected as the unidentifiable sample in the unidentifiable sample detecting section, a specific sample as an outlier by the use of the probability density function if the specific sample is deviated from a predetermined confidence interval, and

[0039] a clustering section for classifying each sample which has not been detected as the unidentifiable sample or the outlier in the unidentifiable sample detecting section or the outlier detecting section into each class by the use of the posterior probabilities.

[0040] Preferably, the clustering section uses normal mixture distribution having a variance parameter selected per each class.

[0041] Thus, the variance parameter of the normal mixture distribution used in the clustering section of this invention is freely selected per each class. Therefore, the difference in variation among the respective classes can be accommodated. It is thus possible to achieve the fifth object of this invention.

[0042] According to a second aspect of this invention, there is provided a clustering method comprising the steps of

[0043] supplying an input unit with a dataset including a plurality of samples,

[0044] processing, in a data processing unit, the samples supplied from the input unit to classify each sample into a class, and

[0045] producing, by an output unit, a processing result representative of classification carried out in the data processing unit,

[0046] the clustering method further comprising the steps of

[0047] memorizing, in a parameter memory of a memory unit, a target parameter obtained from past experiments, and

[0048] estimating, in a parameter estimating section of the data processing unit, a clustering parameter by the use of the target parameter memorized in the parameter memory.

[0049] Preferably, the clustering method according to the second aspect further comprises the step of detecting, by an unidentifiable sample detecting section of the data processing unit, a particular sample as an unidentifiable sample if posterior probabilities calculated for the particular sample by a probability density function produced by the clustering parameter estimated by the parameter estimating section are smaller than a predetermined value.

[0050] Preferably, the clustering method according to the second aspect further comprises the steps of

[0051] detecting, by an outlier detecting section of the data processing unit, a particular sample as an outlier by the use of a probability density function produced by an estimated clustering parameter estimated by the parameter estimating section if the particular sample is deviated from a predetermined confidence interval,

[0052] detecting, by an unidentifiable sample detecting section of the data processing unit, a specific sample as an unidentifiable sample from those samples which have not been detected as the outlier in the outlier detecting section, if posterior probabilities calculated by the probability density function for the specific sample are smaller than a predetermined probability value, and

[0053] classifying, by a clustering section of the data processing unit, each sample which has not been detected as the outlier or the unidentifiable sample in the outlier detecting section or the unidentifiable sample detecting section into each class by the use of the posterior probabilities.

[0054] Preferably, the clustering method according to the second aspect further comprises the steps of

[0055] detecting, by an unidentifiable sample detecting section of the data processing unit, a particular sample as an unidentifiable sample if posterior probabilities calculated for the particular sample by a probability density function produced by an estimated clustering parameter estimated by the parameter estimating section are smaller than a predetermined probability value,

[0056] detecting, by an outlier detecting section of the data processing unit, a specific sample as an outlier by the use of the probability density function from those samples which have not been detected as the unidentifiable sample in the unidentifiable sample detecting section, if the specific sample is deviated from a predetermined confidence interval, and

[0057] classifying, by a clustering section of the data processing unit, each sample which has not been detected as the unidentifiable sample or the outlier in the unidentifiable sample detecting section or the outlier detecting section into each class by the use of the posterior probabilities.

[0058] According to a third aspect of this invention, there is provided a clustering program for making a computer execute a function of supplying a dataset including a plurality of samples, a function of processing the samples supplied by the supplying function to classify each sample into a class, and a function of producing a processing result representative of classification carried out by the classifying function, the clustering program further including a function of memorizing, in a memory unit, a target parameter obtained from past experiments, the classifying function including a function of estimating a clustering parameter by the use of the target parameter memorized in the memory unit.

[0059] In the clustering program according to the third aspect, it is preferable that the classifying function further includes a function of detecting a particular sample as an unidentifiable sample if posterior probabilities calculated for the particular sample by a probability density function produced by the clustering parameter estimated by the estimating function are smaller than a predetermined value.

[0060] In the clustering program according to the third aspect, the classifying function further includes the functions of

[0061] detecting, by the use of a probability density function produced by an estimated clustering parameter estimated by the parameter estimating function, a particular sample as an outlier if the particular sample is deviated from a predetermined confidence interval,

[0062] detecting, from those samples which have not been detected as the outlier in the outlier detecting function, a specific sample as an unidentifiable or unclusterable sample if posterior probabilities calculated by the probability density function for the specific sample are smaller than a predetermined probability value, and

[0063] classifying each sample which has not been detected as the outlier or the unidentifiable sample in the outlier detecting function or the unidentifiable sample detecting function into each class by the use of the posterior probabilities.

[0064] In the clustering program according to the third aspect, the classifying function further includes the functions of

[0065] detecting a particular sample as an unidentifiable sample if posterior probabilities calculated for the particular sample by a probability density function produced by an estimated clustering parameter estimated by the parameter estimating function are smaller than a predetermined probability value,

[0066] detecting, from those samples which have not been detected as the unidentifiable sample in the unidentifiable sample detecting function, a specific sample as an outlier by the use of the probability density function if the specific sample is deviated from a predetermined confidence interval, and

[0067] classifying each sample which has not been detected as the unidentifiable sample or the outlier in the unidentifiable sample detecting function or the outlier detecting function into each class by the use of the posterior probabilities.

BRIEF DESCRIPTION OF THE DRAWING

[0068] FIG. 1 is a flow chart for describing K-means clustering as an existing technique;

[0069] FIG. 2 is a block diagram of a clustering apparatus for realizing, as another existing technique, a clustering method based on normal mixture distribution using a maximum likelihood method;

[0070] FIG. 3 is a flow chart for describing the clustering method realized by the apparatus illustrated in FIG. 2;

[0071] FIG. 4 is a block diagram of a clustering apparatus for realizing a clustering method according to a first embodiment of this invention;

[0072] FIG. 5 is a flow chart for describing the clustering method according to the first embodiment of this invention;

[0073] FIG. 6 is a block diagram of a clustering apparatus for realizing a clustering method according to a second embodiment of this invention;

[0074] FIG. 7 is a flow chart for describing the clustering method according to the second embodiment of this invention;

[0075] FIG. 8 is a block diagram of a clustering apparatus according to a third embodiment of this invention;

[0076] FIG. 9 shows Gene3335 as a dataset to be subjected to clustering;

[0077] FIG. 10 shows a clustering result for Gene3335 in FIG. 9 according to the K-means clustering;

[0078] FIG. 11 shows a clustering result for Gene3335 in FIG. 9 according to this invention;

[0079] FIG. 12 shows simulation data as a dataset to be subjected to clustering;

[0080] FIG. 13 shows a clustering result for the simulation data in FIG. 12 according to the clustering method based on normal mixture distribution using a maximum likelihood method;

[0081] FIG. 14 shows a clustering result for the simulation data in FIG. 12 according to this invention;

[0082] FIG. 15 is a flow chart for describing a clustering method based on normal mixture distribution using a maximum likelihood method with an unidentifiable sample detecting section incorporated therein;

[0083] FIG. 16 shows Gene10530 as a dataset to be subjected to clustering;

[0084] FIG. 17 shows a clustering result for Gene10530 in FIG. 16 according to the clustering method based on normal mixture distribution using a maximum likelihood method; and

[0085] FIG. 18 shows a clustering result for Gene10530 in FIG. 16 according to this invention.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0086] Now, this invention will be described in detail with reference to the drawing.

[0087] Referring to FIG. 4, a clustering apparatus according to a first embodiment of this invention comprises an input unit 1 such as a keyboard, a data processing unit 4 operable under control of a program, a memory unit 5 for memorizing information, and an output unit 3 such as a display or a printer.

[0088] The memory unit 5 comprises a parameter memory 51.

[0089] The parameter memory 51 preliminarily memorizes a target parameter 4 obtained from past experiments and parameters P and X for tuning a parameter estimated value as a resultant value.

[0090] The data processing unit 4 comprises a parameter estimating section 24, an outlier detecting section 22, an unidentifiable sample detecting section 25, and a clustering section 26.

[0091] The parameter estimating section 24 estimates a clustering parameter θ by the use of a dataset including a plurality of samples supplied from the input unit 1 and the parameters ζ, β, and λ memorized in the parameter memory 51. By the use of a probability density function produced by the clustering parameter θ estimated by the parameter estimating section 24, the outlier detecting section 22 detects a particular sample as an outlier if the particular sample is deviated from a predetermined confidence interval.

[0092] The unidentifiable sample detecting section 25 detects, from those samples which have not been detected as the outlier in the outlier detecting section 22, a specific sample as an unidentifiable sample if posterior probabilities calculated for the specific sample by the probability density function similar to that mentioned above are smaller than a predetermined probability value γ.

[0093] The clustering section 26 classifies each sample, which has not been detected as the outlier or the unidentifiable sample in the outlier detecting section 22 or the unidentifiable sample detecting section 25, into each class by the use of the posterior probabilities similar to that mentioned above.

[0094] Next referring to FIG. 5 in addition to FIG. 4, description will be made in detail about a clustering method realized by the clustering apparatus illustrated in FIG. 4.

[0095] The input unit 1 is supplied with the dataset including a plurality of samples x (step C1 in FIG. 5). By the use of the parameters ζ, β, and λ memorized in the parameter memory 51, various parameters are initialized (step C2). By the use of the dataset supplied in the step C1 and the parameters initialized in the step C2, the parameter estimating section 24 obtains the clustering parameter θ by maximizing a modified likelihood function PL(θ) given by Equation (4) with respect to the clustering parameter θ (step C3). 4$\begin{array}{cc}PL\left(\theta \right)=l_{\beta }\left(\theta \right)-\frac{\lambda }{n}\sum _{j=1}^{k}KL\left({\zeta }_{\mathrm{j0}},{\zeta }_j\right)& \left(4\right)\end{array}$

[0096] Herein, n represents the number of samples, λ, a tuning parameter, k, the number of classes. θ=(ωj, μj, σ2j). ζj=(μj, σ2j), ωjrepresents a weight parameter of the probability density function given by Equation (1), μj, an average, σ2j, a variance value. The modified likelihood function PL(θ) is a function robust against the outlier. lβ(θ) and KL(ζj0, ζj) are given by Equations (5) and (6), respectively. 5$\begin{array}{cc}{l}_{\beta }\left(\theta \right)=\frac{1}{n\beta }\sum _{i=1}^n{f\left(x_i;\theta \right)}^{\beta }-b_{\beta }\left(\theta \right)\mathrm{Herein},\beta >0\mathrm{and}{b}_{\beta }\left(\theta \right)=\frac{1}{1+\beta }\int {f\left(x;\theta \right)}^{1+\beta }dx.& \left(5\right)\\ KL\left({\zeta }_{\mathrm{j0}},{\zeta }_j\right)=\int \phi \left(z;{\zeta }_{\mathrm{j0}}\right)\log \frac{\phi \left(z;{\zeta }_{\mathrm{j0}}\right)}{\phi \left(z;{\zeta }_j\right)}dz& \left(6\right)\end{array}$

[0097] Herein, φ(z; ζj) represents a probability density function of normal distribution having an average μj and a variance σ2j. The parameter estimating section 24 calculates the probability density function given by Equation (1) by the use of the clustering parameter θ obtained as mentioned above (step C4). The outlier detecting section 22 judges whether or not each sample supplied in the step C1 is present within a predetermined confidence interval of the probability density function (step C5). If the sample is not present within the confidence interval, the sample is detected as an outlier (step C9).

[0098] In FIG. 5, it is assumed that the sample is judged to be present within the confidence interval in the step C5. In this event, the sample is supplied to the unidentifiable sample detecting section 25. For the sample, the unidentifiable sample detecting section 25 calculates the posterior probabilities for the respective classes according to Equation (3) (step C6). The unidentifiable sample detecting section 25 judges whether or not the posterior probabilities exceed the value of γ (step C7). If the posterior probabilities are smaller than γ, the sample is detected as an unidentifiable sample (step C10).

[0099] In FIG. 5, it is assumed that any of the posterior probabilities for the sample is not smaller than γ In the step C7. In this event, the sample is supplied to the clustering section 26. With reference to the posterior probabilities calculated for the sample, the clustering section 26 selects, as a class of the sample, a class j giving the maximum value of pj (step C8). The data processing unit 4 repeatedly carries out the above-mentioned steps for all the samples (step C11). After compleion of clustering for all the samples, the result is supplied to the output unit 3 (step C12).

[0100] Next, the effect of this embodiment will be described.

[0101] In this embodiment, the modified likelihood function given by Equation (4) is established and maximized so that the clustering complying with the objects of this invention can be carried out. By adjusting the confidence interval and the threshold value γ for the posterior probabilities, it is possible to adjust the clustering. The variance parameter σ2j of normal mixture distribution used in the clustering section 26 is freely selected per each class.

[0102] Referring to FIG. 6, a clustering apparatus according to a second embodiment of this invention will be described. The clustering apparatus in this embodiment is different from the clustering apparatus illustrated in FIG. 4 in that the outlier detecting section 22 and the unidentifiable detecting section 25 are different or reversed in the order of arrangement.

[0103] Referring to FIG. 7 in addition to FIG. 6, description will be made in detail about a clustering method executed by the clustering apparatus illustrated in FIG. 6.

[0104] The parameter estimating section 24, the clustering section 26, and the parameter memory 51 involved in steps D1 to D4 and D8 to D12 in FIG. 7 in this embodiment are same in function as those in the first embodiment. Therefore, description thereof will be omitted.

[0105] In the first embodiment, the outlier is first detected in the steps C5 to C7 in FIG. 5 and the unidentifiable sample is detected from remaining samples. In the second embodiment, the unidentifiable sample is first detected in steps D5 to D7 in FIG. 7 and the outlier is detected from remaining samples. Thus, even if detection of the outlier and detection of the unidentifiable sample are different in the order of operation, the similar effect is obtained.

[0106] Referring to FIG. 8, a clustering apparatus according to a third embodiment of this invention will be described. The clustering apparatus in this embodiment comprises the input unit 1, a data processing unit 8, the memory unit 5, and the output unit 3, like the first and the second embodiments, and further comprises a recording medium 7 memorizing a clustering program. The recording medium 7 may be a magnetic disk, a semiconductor memory, a CD-ROM, a DVD-ROM, and any other appropriate recording medium.

[0107] The clustering program is loaded from the recording medium 7 into the data processing unit 8 to control the operation of the data processing unit 8 and to create a parameter memory 51 in the memory unit 5. Under control of the clustering program, the data processing unit 8 executes an operation similar to those executed by the data processing units 4 and 6 in the first and the second embodiments.

EXAMPLES

[0108] Next, description will be made of a first example with reference to the drawing. The first example corresponds to the first embodiment.

[0109] In this example, consideration will be made of the case where a large number of individuals data plotted on a two-dimensional plane are used as the dataset and a genotype in each sample is judged with reference to their positional relationship. In recent years, typing of a large amount of human single nucleotide polymorphisms (SNPs) is carried out. The genotype is analyzed, for example, by the invader assay. The invader assay is a genome analysis technique developed by Third Wave Technologies, INC in USA. In the invader assay, allele-specific oligo with fluorescent labeling and a template are hybridized with a DNA to be analyzed. The result of hybridization is obtained as the strength of fluorescence so as to analyze the genotype.

[0110] The result of a particular genotype analyzed by the invader assay is plotted on a two-dimensional plane as shown in FIG. 9. The X axis and the Y axis represent the strengths of fluorescence of agents for detecting two alleles (allelic genes), respectively. If the value of X is greater and if the value of Y is greater, it is judged that the individual has an allele 1 and an allele 2, respectively. A sample close to the X axis, a sample close to the Y axis, and a sample close to an inclination of about 45 degrees are judged to have a genotype 1/1, a genotype 2/2, and a genotype 1/2, respectively. A sample near an origin is intended to check the experiment and is not directed to a human sequence.

[0111] In this example, this invention is applied in order to cluster the above-mentioned data according to the three genotypes 1/1, 2/2, and 1/2.

[0112] At first, comparison will be made between this invention and the K-means clustering described in the background. A dataset to be analyzed is shown in FIG. 9. The dataset Gene3335 in FIG. 9 already cleared the examination by the ethics panel with respect to “Guideline for Ethics Related to Research for Human Genome/Gene Analysis” in the Japanese Foundation for Cancer Research to which Mr. Miki, one of the inventors, belongs. The dataset shown in FIG. 9 includes an unidentifiable sample and an outlier between classes. Those samples near the origin need not be judged. It is well known that appropriate one-dimensional angular data are effective in clustering. Therefore, the clustering which will hereinafter be described was carried out based on the one-dimensional angular data without including those samples near the origin.

[0113] FIG. 10 shows the result when the K-means clustering was applied to the one-dimensional angular data in FIG. 9. FIG. 11 shows the result when the clustering of this invention is applied to the one-dimensional angular data in FIG. 9. In FIGS. 10 and 11, the numerals 1 to 3 represent class numbers, 7, an outlier, 0, an unidentifiable sample.

[0114] In FIG. 10, each of all samples except those near the origin is classified into any one of the classes. However, a number of obvious clustering errors are observed. On the other hand, in FIG. 11, the result of clustering is reasonable. In addition, each sample which can not clearly be classified into either the class 1 or the class 2 is detected as an unidentifiable sample. Thus, a significant result is obtained.

[0115] Next, comparison will be made between this invention and the clustering method based on normal mixture distribution using the maximum likelihood method described in the background. A dataset to be analyzed is simulation data shown in FIG. 12. The dataset in FIG. 12 includes a plurality of samples to be classified into two classes and have one outlier. For the dataset also, clustering was carried out based on one-dimensional angular data without including those samples near the origin.

[0116] FIG. 13 shows the result when the dataset in FIG. 12 was subjected to clustering according to the normal mixture distribution using the maximum likelihood method. FIG. 14 shows the result when the dataset in FIG. 12 was subjected to clustering according to this invention.

[0117] In FIG. 13, the samples are classified into two classes since the number of classes in the maximum likelihood method is preliminarily selected to be equal to two. However, the sample as the outlier is also classified into the class 2. On the other hand, in FIG. 14, even if the number of classes is selected to be equal to three, the samples are correctly classified into two classes and the outlier is detected.

[0118] By the use of another dataset, comparison between this invention and the maximum likelihood method will be made. The clustering method based on normal mixture distribution using the maximum likelihood method herein used for comparison incorporates the unidentifiable sample detecting section which is a part of this invention. The clustering is carried out through steps shown in FIG. 15.

[0119] The steps in FIG. 15 are different from those in FIG. 3 in that steps E7 and E10 are added. Specifically, the steps E1 to E6 in FIG. 15 correspond to the steps B1 to B6 in FIG. 3, respectively. The steps E8 and E9 in FIG. 15 correspond to the steps B7 and B8 in FIG. 3, respectively. The steps E11 and E12 in FIG. 15 correspond to the steps B9 and B10 in FIG. 3, respectively.

[0120] The dataset to be analyzed is shown in FIG. 16. The dataset Gene10530 in FIG. 16 already cleared the examination by the ethics panel with respect to “Guideline for Ethics Related to Research for Human Genome/Gene Analysis” in the Japanese Foundation for Cancer Research to which Mr. Miki, one of the inventors, belongs. The dataset shown in FIG. 16 include a plurality of samples to be classified into two classes except those samples near the origin. An outlier is present also.

[0121] FIG. 17 shows the result when the dataset Gene10530 in FIG. 16 was subjected clustering according to the normal mixture distribution using the maximum likelihood method. FIG. 18 shows the result when the dataset Gene10530 in FIG. 16 was subjected to clustering according to this invention.

[0122] In FIG. 17, the samples which should be classified into two classes are classified into three classes since the number of classes in the maximum likelihood method is selected to be equal to 3. Those samples to belong to the class 2 are unreasonably classified into two different classes and intermediate samples are judged unidentifiable. On the other hand, in FIG. 18, even if the number of classes is selected to be equal to three, the samples are properly classified into two classes. In addition, an outlier is detected.

[0123] From the above-mentioned results, it has been found out that this invention solves the problems in the existing techniques.

[0124] It is noted here that this invention is not restricted to the foregoing embodiment but may be modified in various other manners within the scope of this invention.

[0125] As described above, this invention has the following effects.

[0126] As a first effect, clustering is stably and properly carried out even if the number of classes of data is unknown. This is because the parameter obtained from past experiments is preliminarily memorized and used for estimation of a new parameter.

[0127] As a second effect, those samples whose class is ambiguous and indefinite can be detected as an unidentifiable sample. This is because the function of detecting the unidentifiable sample is created and applied to this invention.

[0128] As a third effect, clustering is properly carried out even if an outlier is present. This is because an algorithm robust against the outlier is created so that no serious influence is given by the outlier.

[0129] As a fourth effect, even if only one sample belongs to a particular class, the sample can be classified into a proper class. This is because, in this invention, a proper estimated value is obtained, even if only one sample belongs to a particular class, so that the sample is not judged as an outlier.

[0130] As a fifth effect, proper clustering is carried out even if variation in respective classes is different. This is because a model structure is adopted in which difference in variation can be considered by estimating the variance parameter of normal mixture distribution per each class.

Claims

1. A clustering apparatus comprising an input unit supplied with a dataset including a plurality of samples, a data processing unit for processing the samples supplied from the input unit to classify each sample into a class, and an output unit for producing a processing result representative of classification carried out in the data processing unit, the clustering apparatus further comprising a parameter memory for memorizing a target parameter obtained from past experiments, the data processing unit comprising a parameter estimating section for estimating a clustering parameter by the use of the target parameter memorized in the parameter memory.

2. A clustering apparatus as claimed in claim 1, wherein the parameter estimating section estimates the clustering parameter by the use of a modified likelihood function which is robust against an outlier.

3. A clustering apparatus as claimed in claim 1, wherein the data processing unit further comprises an unidentifiable sample detecting section for detecting a particular sample as an unidentifiable sample if posterior probabilities calculated for the particular sample by a probability density function produced by the clustering parameter estimated by the parameter estimating section are smaller than a predetermined value.

4. A clustering apparatus as claimed in claim 1, wherein the data processing unit further comprises: an outlier detecting section for detecting, by the use of a probability density function produced by an estimated parameter estimated by the parameter estimating section, a particular sample as an outlier if the particular sample is deviated from a predetermined confidence interval, an unidentifiable sample detecting section for detecting, from those samples which have not been detected as the outlier in the outlier detecting section, a specific sample as an unidentifiable or unclusterable sample if posterior probabilities calculated by the probability density function for the specific sample are smaller than a predetermined probability value, and a clustering section for classifying each sample which has not been detected as the outlier or the unidentifiable sample in the outlier detecting section or the unidentifiable sample detecting section into each class by the use of the posterior probabilities.

5. A clustering apparatus as claimed in claim 4, wherein the clustering section uses normal mixture distribution having a variance parameter selected per each class.

6. A clustering apparatus as claimed in claim 1, wherein the data processing unit further comprises: an unidentifiable sample detecting section for detecting a particular sample as an unidentifiable sample if posterior probabilities calculated for the particular sample by a probability density function produced by an estimated parameter estimated by the parameter estimating section are smaller than a predetermined probability value, an outlier detecting section for detecting, from those samples which have not been detected as the unidentifiable sample in the unidentifiable sample detecting section, a specific sample as an outlier by the use of the probability density function if the specific sample is deviated from a predetermined confidence interval, and a clustering section for classifying each sample which has not been detected as the unidentifiable sample or the outlier in the unidentifiable sample detecting section or the outlier detecting section into each class by the use of the posterior probabilities.

7. A clustering apparatus as claimed in claim 6, wherein the clustering section uses normal mixture distribution having a variance parameter selected per each class.

8. A clustering method comprising the steps of supplying an input unit with a dataset including a plurality of samples, processing, in a data processing unit, the samples supplied from the input unit to classify each sample into a class, and producing, by an output unit, a processing result representative of classification carried out in the data processing unit, the clustering method further comprising the steps of memorizing, in a parameter memory of a memory unit, a target parameter obtained from past experiments, and estimating, in a parameter estimating section of the data processing unit, a clustering parameter by the use of the target parameter memorized in the parameter memory.

9. A clustering method as claimed in claim 8, further comprising the step of detecting, by an unidentifiable sample detecting section of the data processing unit, a particular sample as an unidentifiable sample if posterior probabilities calculated for the particular sample by a probability density function produced by the clustering parameter estimated by the parameter estimating section are smaller than a predetermined value.

10. A clustering method as claimed in claim 8, further comprising the steps of detecting, by an outlier detecting section of the data processing unit, a particular sample as an outlier by the use of a probability density function produced by an estimated clustering parameter estimated by the parameter estimating section if the particular sample is deviated from a predetermined confidence interval, detecting, by an unidentifiable sample detecting section of the data processing unit, a specific sample as an unidentifiable sample from those samples which have not been detected as the outlier in the outlier detecting section, if posterior probabilities calculated by the probability density function for the specific sample are smaller than a predetermined probability value, and classifying, by a clustering section of the data processing unit, each sample which has not been detected as the outlier or the unidentifiable sample in the outlier detecting section or the unidentifiable sample detecting section into each class by the use of the posterior probabilities.

11. A clustering method as claimed in claim 8, further comprising the steps of detecting, by an unidentifiable sample detecting section of the data processing unit, a particular sample as an unidentifiable sample if posterior probabilities calculated for the particular sample by a probability density function produced by an estimated clustering parameter estimated by the parameter estimating section are smaller than a predetermined probability value, detecting, by an outlier detecting section of the data processing unit, a specific sample as an outlier by the use of the probability density function from those samples which have not been detected as the unidentifiable sample in the unidentifiable sample detecting section, if the specific sample is deviated from a predetermined confidence interval, and classifying, by a clustering section of the data processing unit, each sample which has not been detected as the unidentifiable sample or the outlier in the unidentifiable sample detecting section or the outlier detecting section into each class by the use of the posterior probabilities.

12. A clustering program for making a computer execute a function of supplying a dataset including a plurality of samples, a function of processing the samples supplied by the supplying function to classify each sample into a class, and a function of producing a processing result representative of classification carried out by the classifying function, the clustering program further comprising a function of memorizing, in a memory unit, a target parameter obtained from past experiments, the classifying function including a function of estimating a clustering parameter by the use of the target parameter memorized in the memory unit.

13. A clustering program as claimed in claim 12, wherein the classifying function further comprises a function of detecting a particular sample as an unidentifiable sample if posterior probabilities calculated for the particular sample by a probability density function produced by the clustering parameter estimated by the estimating function are smaller than a predetermined value.

14. A clustering program as claimed in claim 12, wherein the classifying function further comprises the functions of detecting, by the use of a probability density function produced by an estimated clustering parameter estimated by the parameter estimating function, a particular sample as an outlier if the particular sample is deviated from a predetermined confidence interval, detecting, from those samples which have not been detected as the outlier in the outlier detecting function, a specific sample as an unidentifiable or unclusterable sample if posterior probabilities calculated by the probability density function for the specific sample are smaller than a predetermined probability value, and classifying each sample which has not been detected as the outlier or the unidentifiable sample in the outlier detecting function or the unidentifiable sample detecting function into each class by the use of the posterior probabilities.

15. A clustering program as claimed in claim 12, wherein the classifying function further includes the functions of detecting a particular sample as an unidentifiable sample if posterior probabilities calculated for the particular sample by a probability density function produced by an estimated clustering parameter estimated by the parameter estimating function are smaller than a predetermined probability value, detecting, from those samples which have not been detected as the unidentifiable sample in the unidentifiable sample detecting function, a specific sample as an outlier by the use of the probability density function if the specific sample is deviated from a predetermined confidence interval, and classifying each sample which has not been detected as the unidentifiable sample or the outlier in the unidentifiable sample detecting function or the outlier detecting function into each class by the use of the posterior probabilities.