Machine learning: a review of classification and combining techniques by S. B. Kotsiantis, I. D. Zaharakis, P. E . Pintelas
This paper describes the best-know supervised techniques in relative detail. The key question when dealing with ML classification is not whether a learning algorithm is superior to others, but under which conditions a particular method can significantly outperform others on a given application problem.
If we are only interested in the best possible classification accuracy, it might be difficult or impossible to find a single classifier that performs as well as a good ensemble of classifiers.
However Ensemble has following three main challenges:
1. High Storage.
2. More computational requirement.
3. Difficult to comprehend.
Author has explained following known algorithms:
Logic based algorithms
Decision trees
Inductive logic programming
Perceptron-based techniques:
Neural networks
Statistical learning algorithms:
Bayesian networks
Naive Bayes classifiers
Instance-based learning: k-Nearest Neighbor
Support vector machines
Please find the below table for easier comparison of the experiment outcome details mentioned by author:
Combining Classifiers:
Mechanisms that are used to build ensemble of classifiers include:
1. Single learning method :
· Using different subsets of training data.
· Using different training parameters. (e.g., using different initial weights for each neural network in an ensemble)
2. Using different learning methods.
Using different subsets of training data:
1. Bagging
2. Boosting.
3. Adaboost.
4. Multiboosting. (wagging)
5. DECORATE.
Using different training parameters:
1. Different Initial weights.
2. RSM (Random Subspace Method) – Feature Selection.
Using different learning methods:
1. Voting Method.
2. Stacking.
3. Hybrid Techniques: eg (The Bayesian Network structure is obtained from the data and the Nearest Neighbor algorithm is used in combination with the Bayesian Network in the deduction phase)
*********************************Thank You******************************