How Many Clusters? Which Clustering Method? Answers Via Model-Based Cluster Analysis.


C. Fraley and A. E. Raftery Technical Report No. 329
Department of Statistics University of Washington Box 354322 Seattle, WA 98195-4322 USA
We consider the problem of determining the structure of clustered data, without prior knowledge of the number of clusters or any other information about their composition. Data are represented by a mixture model in which each component corresponds to a different cluster. Models with varying geometric properties are obtained through Gaussian components with different parameterizations and cross-cluster constraints. Noise and outliers can be modeled by adding a Poisson process component. Partitions are determined by the EM (expectation-maximization) algorithm for maximum likelihood, with initial values from agglomerative hierarchical clustering.
Models are compared using an approximation to the Bayes factor based on the Bayesian Information Criterion (BIC); unlike signi cance tests, this allows comparison of more than two models at the same time, and removes the restriction that the models compared be nested. The problems of determining the number of clusters and the clustering method are solved simultaneously by choosing the best model. Moreover, the EM result provides a measure of uncertainty about the associated classi cation of each data point. Examples are given, showing that this approach can give performance that is much better than standard procedures, which often fail to identify groups that are either overlapping or of varying sizes and shapes.
Read the full document

Post to Twitter Post to Yahoo Buzz Post to Delicious Post to Digg Post to Facebook Post to Google Buzz Post to LinkedIn Post to Slashdot Post to StumbleUpon Post to Technorati