Optimization of classifier ensemble diversity - Leif E. Peterson (Academia Molecular Biology and Genomics)
Ensemble diversity was investigated for 12 classifiers and 16 datasets using a generalization error and ambiguity decomposition of the model bias-variance relationship. Diversity was optimized using a genetic algorithm and particle swarm optimization. Classifiers with the greatest contribution to ensemble diversity were learning vector quantization, naïve Bayes classifier, and supervised neural gas, whereas the supervised artificial neural network (SANN) and support vector machines (SVM) classifiers were among the least informative for diversity. An important observation was that classifiers with the greatest optimized employment weights in diverse ensembles also had the greatest association between individual unoptimized diversity, D, and number of classes among datasets, with little association with dataset number of objects or features. The SANN and SVM classifiers had a significant association with the number of objects in datasets and were employed infrequently in diverse ensembles. In conclusion, the diversity of a classifier is more dependent on the number of classes of datasets and less dependent on the number of objects or features. A greater ensemble employment weight for a classifier also occurs when its range of generalization error and ambiguity over all datasets overlap.