Best K: the Critical Clustering Structures in Categorical Data

TitleBest K: the Critical Clustering Structures in Categorical Data
Publication TypeJournal Article
Year of Publication2008
AuthorsLing Liu, Keke Chen
Keywordsbest Ks&#39, how can we efficiently and reliably determine the best Ks?, none has satisfactorily addressed the problem of Best K for categorical clustering. Since categorical data does not have an inherent distance function as the similarity measure, surprisingly, The demand on cluster analysis for categorical data continues to grow over the last decade. A well-known problem in categorical clustering is to determine the best K number of clusters. Although several categorical clustering algorithms have been develope, traditional cluster validation techniques based on geometric shapes and density distributions are not appropriate for categorical data. In this paper, we study the entropy property between the clustering results of categorical data with different K number of clusters, what is the set of candidate &#39
Abstract

The demand on cluster analysis for categorical data continues to grow over the last decade. A well-known problem in categorical clustering is to determine the best K number of clusters. Although several categorical clustering algorithms have been developed, surprisingly, none has satisfactorily addressed the problem of Best K for categorical clustering. Since categorical data does not have an inherent distance function as the similarity measure, traditional cluster validation techniques based on geometric shapes and density distributions are not appropriate for categorical data. In this paper, we study the entropy property between the clustering results of categorical data with different K number of clusters, and propose the BKPlot method to address the three important cluster validation problems: 1) How can we determine whether there is significant clustering structure in a categorical dataset? 2) If there is significant clustering structure, what is the set of candidate 'best Ks'? 3) If the dataset is large, how can we efficiently and reliably determine the best Ks?

Full Text

Keke Chen and Ling Liu: ' Best K: the Critical Clustering Structures in Categorical Data ', Knowedge and Information Systems, 2008 [pdf]
publisher: Knowedge and Information Systems
year: 2008
hasURL: http://knoesis.wright.edu/library/publications/Keke/bestk_kais_final.pdf