SCALE: a Scalable Framework for Efficiently Clustering Large Transactional Data

TitleSCALE: a Scalable Framework for Efficiently Clustering Large Transactional Data
Publication TypeJournal Article
Year of Publication2010
AuthorsHua Yan, Keke Chen, Ling Liu, Zhang Yi
JournalJournal of Data Mining and Knowledge Discovery (DMKD)
KeywordsFramework, Large Data Clusters
Abstract

This paper presents SCALE, a fully automated transactional clustering framework. The SCALE design highlights three unique features. First, we introduce the concept of Weighted Coverage Density as a categorical similarity measure for efficient clustering of transactional datasets. The concept of weighted coverage density is intuitive and it allows the weight of each item in a cluster to be changed dynamically according to the occurrences of items. Second, we develop the weighted coverage density measure based clustering algorithm, a fast, memory-efficient, and scalable clustering algorithm for analyzing transactional data. Third, we introduce two clustering validation metrics and show that these domain specific clustering evaluation metrics are critical to capture the trasactional semantics in clustering analysis. Our SCALE framework combines the weighted coverage density measure for clustering over a sample dataset with self configuring methods. These self-configuring methods can automatically tune the two important parameters of our clustering algorithms: (1) the candidates of the best number K of clusters; and (2) the application of two domain-specific cluster validity measures to find the best result from the set of clustering results.We have conducted extensive experimental evaluation using both synthetic and real datasets and our results show that the weighted coverage density approach powered by the SCALE framework can efficiently generate high quality clustering results in a fully automated manner

Full Text

Hua Yan, Keke Chen and Ling Liu, 'SCALE: a Scalable Framework for Efficiently Clustering Large Transactional Data,'in Journal of Data Mining and Knowledge Discovery (DMKD), 19(4), 2009.
publisher: Springer Netherlands
year: 2009
hasEditor: Geoffrey I. Webb
related resource url: http://www.cs.wright.edu/~keke.chen/pubs.html
hasURL: http://knoesis.wright.edu/library/download/dmkdscale.pdf
hasBookTitle: Journal of Data Mining and Knowledge Discovery (DMKD)

Related Files: