1. Home
  2. Vol 7, No 1 (2023)
  3. Sornsuwit

A New Efficiency Improvement of Ensemble Learning for Heart Failure Classification by Least Error Boosting

Ployphan Sornsuwit, Phimkarnda Jundahuadong, Siwarit Pongsakornrungsilp

Abstract


Heart failure is a very common disease, often a silent threat. It's also costly to treat and detect. There is also a steadily higher incidence rate of the disease at present. Although researchers have developed classification algorithms. Cardiovascular disease data were used by various ensemble learning methods, but the classification efficiency was not high enough due to the cumulative error that can occur from any weak learner effect and the accuracy of the vote-predicted class label. The objective of this research is the development of a new algorithm that improves the efficiency of the classification of patients with heart failure. This paper proposes Least Error Boosting (LEBoosting), a new algorithm that improves adaboost.m1's performance for higher classification accuracy. The learning algorithm finds the lowest error among various weak learners to be used to identify the lowest possible errors to update distribution to create the best final hypothesis in classification. Our trial will use the heart failure clinical records dataset, which contains 13 features of cardiac patients. Performance metrics are measured through precision, recall, f-measure, accuracy, and the ROC curve. Results from the experiment found that the proposed method had high performance compared to naïve bayes, k-NN,and decision tree, and outperformed other ensembles including bagging, logitBoost, LPBoost, and adaboost.m1, with an accuracy of 98.89%, and classified the capabilities of patients who died accurately as well compared to decision tree and bagging, which were completely indistinguishable. The findings of this study found that LEBoosting was able to maximize error reductions in the weak learner's training process from any weak learner to maximize the effectiveness of cardiology classifiers and to provide theoretical guidance to develop a model for analysis and prediction of heart disease. The novelty of this research is to improve original ensemble learning by finding the weak learner with the lowest error in order to update the best distribution to the final hypothesis, which will give LEBoosting the highest classification efficiency.

 

Doi: 10.28991/ESJ-2023-07-01-010

Full Text: PDF


Keywords


Adaboost.m1; Ensemble; k-NN; Learning Algorithm; Heart Failure; LEBoosting.

References


Bazoukis, G., Stavrakis, S., Zhou, J., Bollepalli, S. C., Tse, G., Zhang, Q., Singh, J. P., & Armoundas, A. A. (2021). Machine learning versus conventional clinical methods in guiding management of heart failure patients—a systematic review. Heart Failure Reviews, 26(1), 23–34. doi:10.1007/s10741-020-10007-3.

Lippi, G., & Sanchis-Gomar, F. (2020). Global epidemiology and future trends of heart failure. AME Medical Journal, 5(15), 1-6. doi:10.21037/amj.2020.03.03.

Groenewegen, A., Rutten, F. H., Mosterd, A., & Hoes, A. W. (2020). Epidemiology of heart failure. European Journal of Heart Failure, 22(8), 1342–1356. doi:10.1002/ejhf.1858.

Ishaq, A., Sadiq, S., Umer, M., Ullah, S., Mirjalili, S., Rupapara, V., & Nappi, M. (2021). Improving the Prediction of Heart Failure Patients’ Survival Using SMOTE and Effective Data Mining Techniques. IEEE Access, 9, 39707–39716. doi:10.1109/access.2021.3064084.

Mehmood, A., Iqbal, M., Mehmood, Z., Irtaza, A., Nawaz, M., Nazir, T., & Masood, M. (2021). Prediction of Heart Disease Using Deep Convolutional Neural Networks. Arabian Journal for Science and Engineering, 46(4), 3409–3422. doi:10.1007/s13369-020-05105-1.

Sohrabi, B., Vanani, I. R., Gooyavar, A., & Naderi, N. (2019). Predicting the Readmission of Heart Failure Patients through Data Analytics. Journal of Information and Knowledge Management, 18(1), 1950012. doi:10.1142/S0219649219500126.

Myint, K., & Khaung Tin, H. H. (2020). Analyzing the Comparison of C4.5, CART and C5.0 Algorithms on Heart Disease Dataset using Decision Tree Method. Proceedings of the 2nd International Conference on ICT for Digital, Smart, and Sustainable Development, ICIDSSD 2020, 27-28 February 2020, Jamia Hamdard, New Delhi, India. doi:10.4108/eai.27-2-2020.2303221.

Chicco, D., & Jurman, G. (2020). Machine learning can predict survival of patients with heart failure from serum creatinine and ejection fraction alone. BMC Medical Informatics and Decision Making, 20(1), 1–16. doi:10.1186/s12911-020-1023-5.

Angraal, S., Mortazavi, B. J., Gupta, A., Khera, R., Ahmad, T., Desai, N. R., Jacoby, D. L., Masoudi, F. A., Spertus, J. A., & Krumholz, H. M. (2020). Machine Learning Prediction of Mortality and Hospitalization in Heart Failure with Preserved Ejection Fraction. JACC: Heart Failure, 8(1), 12–21. doi:10.1016/j.jchf.2019.06.013.

Lorenzoni, G., Sabato, S. S., Lanera, C., Bottigliengo, D., Minto, C., Ocagli, H., De Paolis, P., Gregori, D., Iliceto, S., & Pisanò, F. (2019). Comparison of machine learning techniques for prediction of hospitalization in heart failure patients. Journal of Clinical Medicine, 8(9), 1298. doi:10.3390/jcm8091298.

Wang, L., Zhou, W., Chang, Q., Chen, J., & Zhou, X. (2019). Deep ensemble detection of congestive heart failure using short-term RR intervals. IEEE Access, 7, 69559–69574. doi:10.1109/ACCESS.2019.2912226.

Ali, F., El-Sappagh, S., Islam, S. M. R., Kwak, D., Ali, A., Imran, M., & Kwak, K. S. (2020). A smart healthcare monitoring system for heart disease prediction based on ensemble deep learning and feature fusion. Information Fusion, 63, 208–222. doi:10.1016/j.inffus.2020.06.008.

UCI Repository (2017). Heart failure clinical records Data Set. UC Irvine Machine Learning Repository, California, United States. Available online: https://archive.ics.uci.edu/ml/machine-learning-databases/00519/ (access 21 June 2021).

Zhou, Z. H. (2012). Ensemble methods: foundations and algorithms. CRC Press, Boca Raton, United States. doi:10.1201/b12207.

Freund, Y., & Schapire, R. E. (1996). Schapire R: Experiments with a new boosting algorithm. 13th International Conference on Machine Learning, 3-6 July, 1996, Bari, Italy.

Jones, Y., Hillen, N., Friday, J., Pellicori, P., Kean, S., Murphy, C., & Cleland, J. (2020). A comparison of machine learning models for predicting rehospitalisation and death after a first hospitalisation with heart failure. European Heart Journal, 41(Supplement_2), 946–0984. doi:10.1093/ehjci/ehaa946.0984.

Holmes, D.E., Jain, L.C. (2008). Introduction to Bayesian Networks. Innovations in Bayesian Networks. Studies in Computational Intelligence, 156, Springer, Berlin, Germany. doi:10.1007/978-3-540-85066-3_1.

Kramer, O. (2011). Dimensionality Reduction by Unsupervised K-Nearest Neighbor Regression. 2011 10th International Conference on Machine Learning and Applications and Workshops. doi:10.1109/icmla.2011.55.

Salzberg, S. L. (1994). Book Review: C4.5: Programs for Machine Learning by J. Ross Quinlan. Morgan Kaufmann Publishers, Inc., 1993. Machine Learning, 16, 235-240. doi:10.1023/A:1022645310020.

Shmueli, G., Patel, N. R., & Bruce, P. C. (2011). Data mining for business intelligence: Concepts, techniques, and applications in Microsoft Office Excel with XLMiner. John Wiley and Sons, Hoboken, United States.

Kaggle. (2019). Cardiovascular Disease dataset. Kaggle Inc. San Francisco, United States. Available online: https://www.kaggle.com/datasets/sulianova/cardiovascular-disease-dataset (accessed on August 2022).

UCI Repository. (2017). Statlog (Heart) Data Set. UC Irvine Machine Learning Repository, California, United States. Available online: https://archive.ics.uci.edu/ml/datasets/statlog+(heart) (accessed on August 2022).

UCI Repository (2022). Heart Disease Data Set (Issue value 0). UC Irvine Machine Learning Repository, California, United States. Available online: https://archive.ics.uci.edu/ml/datasets/Heart+Disease

Louridi, N., Amar, M., & Ouahidi, B. E. (2019). Identification of Cardiovascular Diseases Using Machine Learning. 2019 7th Mediterranean Congress of Telecommunications (CMT). doi:10.1109/cmt.2019.8931411.

Khennou, F., Fahim, C., Chaoui, H., & Chaoui, N. E. H. (2019). A machine learning approach: Using predictive analytics to identify and analyze high risks patients with heart disease. International Journal of Machine Learning and Computing, 9(6), 762–767. doi:10.18178/ijmlc.2019.9.6.870.

Ali, L., Khan, S. U., Anwar, M., & Asif, M. (2019). Early Detection of Heart Failure by Reducing the Time Complexity of the Machine Learning based Predictive Model. 2019 International Conference on Electrical, Communication, and Computer Engineering (ICECCE). doi:10.1109/icecce47252.2019.8940737.

Escamilla, A., El Hassani, A., & Andres, E. (2019). Dimensionality Reduction in Supervised Models-based for Heart Failure Prediction. Proceedings of the 8th International Conference on Pattern Recognition Applications and Methods. doi:10.5220/000731370388039.

Satyanandam, N., & Satyanarayana, C. (2021). An Effective Analytics using Machine Learning Integrated Approaches for Diagnosis, Severity Estimation and Prediction of Heart Disease. IOP Conference Series: Materials Science and Engineering, 1074(1), 012006. doi:10.1088/1757-899x/1074/1/012006.

Javeed, A., Zhou, S., Yongjian, L., Qasim, I., Noor, A., & Nour, R. (2019). An Intelligent Learning System Based on Random Search Algorithm and Optimized Random Forest Model for Improved Heart Disease Detection. IEEE Access, 7, 180235–180243. doi:10.1109/ACCESS.2019.2952107.

Katarya, R., & Meena, S. K. (2021). Machine Learning Techniques for Heart Disease Prediction: A Comparative Study and Analysis. Health and Technology, 11(1), 87–97. doi:10.1007/s12553-020-00505-7.

Ali, L., Khan, S. U., Golilarz, N. A., Yakubu, I., Qasim, I., Noor, A., & Nour, R. (2019). A Feature-Driven Decision Support System for Heart Failure Prediction Based on χ2 Statistical Model and Gaussian Naive Bayes. Computational and Mathematical Methods in Medicine, 2019, 1–8. doi:10.1155/2019/6314328.


Full Text: PDF

DOI: 10.28991/ESJ-2023-07-01-010

Refbacks

  • There are currently no refbacks.


Copyright (c) 2022 Ployphan Sornsuwit, Phimkarnda Jundahuadong, Siwarit Pongsakornrungsilp