An Ensemble Co-Evolutionary based Algorithm for Classification Problems

  • Vũ Văn Trường Le Quy Don Technical University
  • Bùi Thu Lâm Le Quy Don Technical University
  • Nguyễn Thành Trung Liverpool John Moores University
Keywords: Ensemble learning, co-evolutionary algorithm, Feature selection, Classification


In this paper, the authors propose a dual-population co-evolutionary approach using ensemble learning approach (E-SOCA)  to  simultaneously  solve  both  feature  subset selection  and  optimal  classifier  design.  Different  from previous  studies  where  each  population  retains  only  one best individual (Elite) after co-evolution, in this study, an elite  community  will  be  stored  and  calculated  together through  an  ensemble  learning  algorithm  to  produce  the final    classification    result.    Experimental    results    on standard  UCI  problems  with  a  variety  of  input  features ranging from small to large sizes shows that the proposed algorithm  results  in  more  accuracy  and  stability  than traditional algorithms.


G. Chandrashekar and F. Sahin, “A survey on feature selection methods,” Comput. Electr. Eng., vol. 40, no. 1, pp. 16–28, 2014.

Y. Saeys, I. Inza, and P. Larrañaga, “A review of feature selection techniques in bioinformatics,” Bioinformatics, vol. 23, no. 19, pp. 2507–2517, 2007.

M. Dash and H. Liu, “Feature selection for classification, Intelligent Data Analysis, vol. 1, no. 4, pp. 131–156, 1997.

H. Liu and Z. Zhao, “Manipulating data and dimension reduction methods: Feature selection,” in Encyclopedia of Complexity and Systems Science, pp. 5348–5359, Springer, 2009.

H. Liu, H. Motoda, R. Setiono, and Z. Zhao, “Feature selection: An ever evolving frontier in data mining,” in Feature Selection for Data Mining, vol. 10 of JMLR Proceedings, pp. 4–13,, 2010.

W. Siedlecki and J. Sklansky, “A note on genetic algorithms for largescale feature selection,” Pattern Recognition Letters, vol. 10, no. 5, pp. 335–347, 1989.

M. Shoorehdeli, M. Teshnehlab, and H. Moghaddam, “Feature subset selection for face detection using genetic algorithms and particle swarm optimization,” in IEEE International Conference on Networking, Sensing and Control (ICNSC), pp. 686–690, 2006.

D. Chen, K. Chan, and X. Wu, “Gene expression analyses using genetic algorithm based hybrid approaches,” in IEEE Congress on Evolutionary Computation, pp. 963–969, 2008.

F. Tan, X. Z. Fu, Y. Q. Zhang, and A. Bourgeois, “A genetic algorithmbased method for feature subset selection,” Soft Computing, vol. 12, no. 5, pp. 111–120, 2008.

L.-H. Chen and H.-D. Hsiao, “Feature selection to diagnose a business crisis by using a real ga-based support vector machine: An empirical study,” Expert Systems with Applications, vol. 35, no. 3, pp. 1145–1155, 2008.

J.-H. Hong and S.-B. Cho, “Efficient huge-scale feature selection with speciated genetic algorithm,” Pattern Recognition Letters, vol. 27, no. 2, pp. 143 – 150, 2006.

S. C. Yusta, “Different metaheuristic strategies to solve the feature selection problem,” Pattern Recognition Letters, vol. 30, pp. 525–534, 2009

S. F. da Silva, M. X. Ribeiro, J. do E.S. Batista Neto, C. Traina-Jr., and A. J. Traina, “Improving the ranking quality of medical image retrieval using a genetic feature selection method,” Decision Support Systems, vol. 51, no. 4, pp. 810–820, 2011.

R. Leardi, R. Boggia, and M. Terrile, “Genetic algorithms as a strategy for feature selection,” Journal of Chemometrics, vol. 6, no. 5, pp. 267– 281, 1992.

Y. Jeong, K. S. Shin, and M. K. Jeong, “An evolutionary algorithm with the partial sequential forward floating search mutation for largescale feature selection problems,” Journal of the Operational Research Society, pp. 1–10, 2014.

L. Santana, L. Silva, A. Canuto, F. Pintro, and K. Vale,“A comparative analysis of genetic algorithm and ant colony optimization to select attributes for an heterogeneous ensemble of classifiers,” in IEEE Congress on Evolutionary Computation, pp. 1–8, 2010.

J. Tian, M. Li, and F. Chen, “Dual-population based coevolutionary algorithm for designing RBFNN with feature selection,” Expert Syst. Appl., vol. 37, no. 10, pp. 6904–6918, 2010.

M. A. Potter and K. A. Jong, “A cooperative coevolutionary approach to function optimization,” pp. 249–257, 1994.

Lam Thu Bui, Van Truong VU, Dinh Thi Thu Huong,A novel evolutionary multi-objective ensemble learning approach for forecasting currency exchange rates. Data & Knowledge Engineering, 114, pp.40-66, 2018.

Van Truong VU, Lam Thu Bui, Trung Thanh Nguyen,A Coevolutionary approach for classification problems: Preliminary results, The 5th NAFOSTED Conference on Information and Computer Science (NICS), 2018.

T. T. H. Dinh, V. T. Vu, and T. L. Bui, “A multi-objective ensemble learning approach based on the non-dominated sorting differential evolution for forecasting currency exchange rates,” Proc. - 2016 8th Int. Conf. Knowl. Syst. Eng. KSE 2016, pp. 96–102, 2016.

J. H. Holland, Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. The MIT Press, Cambridge, USA, 1992.

ROKACH, Lior. Ensemble-based classifiers. Artificial Intelligence Review, 2010, 33.1-2: 1-39.

Y. Freund, R.E. Schapire, A desicion-theoretic generalizaton of on-line learning and an application to boosting, in:Proceedings of the European Conference on Computational Learning Theory, 1995, pp. 23–37.

Y. Freund, R.E. Schapire, Experiments with a new boosting algorithm, in: Icml, 1996, pp. 148–156.

R. Li, J. Lu, Y. Zhang, and T. Zhao, “Dynamic adaboost learning with feature selection based on parallel genetic algorithm for image annotation,” Knowledge-Based Systems, vol. 23, no. 3, pp. 195–201, 2010.

Regular Articles