Some Data Mining Methods
Antoine Naud
February 17,2004
This list presents a categorization of various data mining methods into categories corresponding to a type of application. It is simplified and not complete. There is an overlap between the three types of applications (CLAS, DRED, CLUS) in that sense that many methods can be found in more than one category. For instance, most neural networks are designed to solve classification tasks, but they are also used for clustering or dimensionality reduction tasks.
Supervised learning means that information about which sample or data vector belongs to which class is known and used during the teaching of the model.
Unsupervised learning means that class information (whether we know it or not) is not used during the learning process. Most of classifiers need class information so they are supervised methods. Clustering methods are often used when we don't know anything about the classes, and we want the model itself to discover the classes by grouping/partitioning the samples.
1) CLAS - Main families of classification methods:
- ANN - Artificial Neural Networks: (techniques inspired from biology with adaptive inner parameters).
- MLP Multi-Layer Perceptron
- RBF Radial Basis Function
- SVM - Support Vector Machine
- Tree based methods
- KNN - k-nearest neighbors classifiers
- LDA - Linear Discriminant Analysis
- Boosting -> improve the performance of classifiers,
2) DRED - Dimensionality reduction makes high-dimensional data problems more tractable.
a) feature extraction:
- Generative Topographic Mapping
- SOM, Neuroscale, the elastic net, Curvilinear Components Analysis
- Principal curves and principal surfaces.
- PCA - Principal Components Analysis originates in multivariate statistical analysis, it has now many versions: linear, nonlinear (autoassociators), neural, kernel based, ..
- MDS - Multidimensional Scaling (also known as "Sammon mapping")
- PP - projection pursuit
- Local approaches to dimensionality reduction
b) feature selection:
- information theory based feature selection
3) CLUS - Clustering, partitioning:
- SOM Self-Organizing Maps
- LVQ - Learning Vector Quantization
- k-means, C-means, fuzzy C-means
- k-medoids
- hierarchical methods (dendrograms)
- for large datasets: birch, clique, proclus
References:
- Pattern Classification (2 ed.) R. O. Duda, P. E. Hart and D. G. Stork, Wiley Interscience, 2000.
- Ripley B.D., Pattern recognition and neural networks, Cambridge University Press, 1996.
- Pattern Recognition, Theodoridis/ Koutroumbas, Academic Press, 2003.
- Introduction to Statistical Pattern Recognition (2 ed.), Fukunaga, Academic Press, 1990.
- Statistical Pattern Recognition 2/e, Webb, Wiley, 2002.