1. Home
  2. Vol 6 (2022)
  3. Costa-Mendes

Deep Learning in Predicting High School Grades: A Quantum Space of Representation

Ricardo Costa-Mendes, Frederico Cruz-Jesus, Tiago Oliveira, Mauro Castelli

Abstract


This paper applies deep learning to the prediction of Portuguese high school grades. A deep multilayer perceptron and a multiple linear regression implementation are undertaken. The objective is to demonstrate the adequacy of deep learning as a quantitative explanatory paradigm when compared with the classical econometrics approach. The results encompass point predictions, prediction intervals, variable gradients, and the impact of an increase in the class size on grades. Deep learning’s generalization error is lower in the student grade prediction, and its prediction intervals are more accurate. The deep multilayer perceptron gradient empirical distributions largely align with the regression coefficient estimates, indicating a satisfactory regression fit. Based on gradient discrepancies, a student’s mother being an employer does not seem to be a positive factor. A benign paradigm shift concerning the balance between home and career affairs for both genders should be reinforced. The deep multilayer perceptron broadens the spectrum of possibilities, providing a quantum solution hinged on a universal approximator. In the case of an academic achievement-critical factor such as class size, where the literature is neither unanimous on its importance nor its direction, the multilayer perceptron formed three distinct clusters per the individual gradient signals.

 

Doi: 10.28991/ESJ-2022-SIED-012

Full Text: PDF


Keywords


Data Science Applications in Education; Secondary Education; Social Sciences; Deep Learning; Artificial Neural Networks; Academic Achievement.

References


Haykin, S. (2009). Neural networks and learning machines (3rd Ed.). Pearson, New Yoork City, United States.

Jordan, M. I., & Mitchell, T. M. (2015). Machine learning: Trends, perspectives, and prospects. Science, 349(6245), 255–260. doi:10.1126/science.aaa8415.

Shrestha, A., & Mahmood, A. (2019). Review of deep learning algorithms and architectures. IEEE Access, 7, 53040–53065. doi:10.1109/ACCESS.2019.2912200.

Romero, C., & Ventura, S. (2020). Educational data mining and learning analytics: An updated survey. WIREs Data Mining and Knowledge Discovery, 10(3). Portico. doi:10.1002/widm.1355.

Namoun, A., & Alshanqiti, A. (2020). Predicting Student Performance Using Data Mining and Learning Analytics Techniques: A Systematic Literature Review. Applied Sciences, 11(1), 237. doi:10.3390/app11010237.

Costa-Mendes, R., Cruz-Jesus, F., Oliveira, T., & Castelli, M. (2021). Machine learning bias in predicting high school grades: A knowledge perspective. Emerging Science Journal, 5(5), 576–597. doi:10.28991/esj-2021-01298.

Hanushek, E. A., & Wößmann, L. (2010). Education and Economic Growth. International Encyclopedia of Education, 245–252, Elsevier Science, Amsterdam, Netherlands. doi:10.1016/b978-0-08-044894-7.01227-6.

Lei, D., Chen, X., & Zhao, J. (2018). Opening the black box of deep learning. arXiv preprint arXiv:1805.08355. doi:10.48550/arXiv.1805.08355.

Golovko, V. A. (2017). Deep learning: an overview and main paradigms. Optical Memory and Neural Networks (Information Optics), 26(1), 1–17. doi:10.3103/S1060992X16040081.

Cruz-Jesus, F., Castelli, M., Oliveira, T., Mendes, R., Nunes, C., Sa-Velho, M., & Rosa-Louro, A. (2020). Using artificial intelligence methods to assess academic achievement in public high schools of a European Union country. Heliyon, 6(6). doi:10.1016/j.heliyon.2020.e04081.

Jensen, A. R. (1999). The G factor: The science of mental ability. Psycoloquy, 10(04), 36–2443–36–2443,. doi:10.5860/choice.36-2443.

Georgiou, G. K., Guo, K., Naveenkumar, N., Vieira, A. P. A., & Das, J. P. (2020). PASS theory of intelligence and academic achievement: A meta-analytic review. Intelligence, 79, 101431. doi:10.1016/j.intell.2020.101431.

Frey, M. C., & Detterman, D. K. (2004). Scholastic Assessment or g? Psychological Science, 15(6), 373–378. doi:10.1111/j.0956-7976.2004.00687.x.

Rohde, T. E., & Thompson, L. A. (2007). Predicting academic achievement with cognitive ability. Intelligence, 35(1), 83–92. doi:10.1016/j.intell.2006.05.004.

Francis, B., & Skelton, C. (2005). Reassessing Gender and Achievement. Routledge, London, United Kingdom. doi:10.4324/9780203412923.

Lupart, J. L., Cannon, E., & Telfer, J. A. (2004). Gender differences in adolescent academic achievement, interests, values and life-role expectations. High Ability Studies, 15(1), 25–42. doi:10.1080/1359813042000225320.

Mensah, F. K., & Kiernan, K. E. (2010). Gender differences in educational attainment: Influences of the family environment. British Educational Research Journal, 36(2), 239–260. doi:10.1080/01411920902802198.

King, R. B. (2016). Gender differences in motivation, engagement and achievement are related to students’ perceptions of peer—but not of parent or teacher—attitudes toward school. Learning and Individual Differences, 52, 60–71. doi:10.1016/j.lindif.2016.10.006.

Di Fabio, A., & Busoni, L. (2007). Fluid intelligence, personality traits and scholastic success: Empirical evidence in a sample of Italian high school students. Personality and Individual Differences, 43(8), 2095–2104. doi:10.1016/j.paid.2007.06.025.

Kuhfeld, M., Gershoff, E., & Paschall, K. (2018). The development of racial/ethnic and socioeconomic achievement gaps during the school years. Journal of Applied Developmental Psychology, 57, 62–73. doi:10.1016/j.appdev.2018.07.001.

Perreira, K. M., Harris, K. M., & Lee, D. (2006). Making it in America: High school completion by immigrant and native youth. Demography, 43(3), 511–536. doi:10.1353/dem.2006.0026.

Levels, M., Kraaykamp, G., & Dronkers, J. (2008). Immigrant children’s educational achievement in western countries: Origin, destination, and community effects on mathematical performance. American Sociological Review, 73(5), 835–853. doi:10.1177/000312240807300507.

Lei, J., & Zhao, Y. (2007). Technology uses and student achievement: A longitudinal study. Computers and Education, 49(2), 284–296. doi:10.1016/j.compedu.2005.06.013.

Salomon, A., & Ben-David Kolikant, Y. (2016). High-school students’ perceptions of the effects of non-academic usage of ICT on their academic achievements. Computers in Human Behavior, 64, 143–151. doi:10.1016/j.chb.2016.06.024.

Kubey, R. W., Lavin, M. J., & Barrows, J. R. (2001). Internet use and collegiate academic performance decrements: Early findings. Journal of Communication, 51(2), 366–382. doi:10.1111/j.1460-2466.2001.tb02885.x.

Fan, X., & Chen, M. (2001). Parental Involvement and Students’ Academic Achievement: A Meta-Analysis. Educational Psychology Review, 13(1), 1–22. doi:10.1023/A:1009048817385.

Gilar-Corbi, R., Miñano, P., Veas, A., & Castejón, J. L. (2019). Testing for invariance in a structural model of academic achievement across underachieving and non-underachieving students. Contemporary Educational Psychology, 59, 101780. doi:10.1016/j.cedpsych.2019.101780.

Benner, A. D., Boyle, A. E., & Sadler, S. (2016). Parental Involvement and Adolescents’ Educational Success: The Roles of Prior Achievement and Socioeconomic Status. Journal of Youth and Adolescence, 45(6), 1053–1064. doi:10.1007/s10964-016-0431-4.

Sirin, S. R. (2005). Socioeconomic status and academic achievement: A meta-analytic review of research. Review of Educational Research, 75(3), 417–453. doi:10.3102/00346543075003417.

Steinmayr, R., Dinger, F. C., & Spinath, B. (2010). Parents’ education and children’s achievement: The role of personality. European Journal of Personality, 24(6), 535–550. doi:10.1002/per.755.

Tomul, E., & Savasci, H. S. (2012). Socioeconomic determinants of academic achievement. Educational Assessment, Evaluation and Accountability, 24(3), 175–187. doi:10.1007/s11092-012-9149-3.

Hoxby, C. M. (2000). The effects of class size on student achievement: New evidence from population variation. Quarterly Journal of Economics, 115(4), 1239–1285. doi:10.1162/003355300555060.

Krueger, A. B. (1999). Experimental estimates of education production functions. Quarterly Journal of Economics, 114(2), 497–532. doi:10.1162/003355399556052.

Wößmann, L., & West, M. (2006). Class-size effects in school systems around the world: Evidence from between-grade variation in TIMSS. European Economic Review, 50(3), 695–736. doi:10.1016/j.euroecorev.2004.11.005.

Leithwood, K., & Jantzi, D. (2009). A review of empirical evidence about school size effects: A policy perspective. Review of Educational Research, 79(1), 464–490. doi:10.3102/0034654308326158.

Schneider, M. (2002). Do School Facilities Affect Academic Outcomes?. Information Analysis, National Clearinghouse for Educational Facilities, Washington, United States.

Woolner, P., Hall, E., Higgins, S., McCaughey, C., & Wall, K. (2007). A sound foundation? What we know about the impact of environments on learning and the implications for Building Schools for the Future. Oxford Review of Education, 33(1), 47–70. doi:10.1080/03054980601094693.

Aaronson, D., Barrow, L., & Sander, W. (2007). Teachers and student achievement in the Chicago public high schools. Journal of Labor Economics, 25(1), 95–135. doi:10.1086/508733.

Wayne, A. J., & Youngs, P. (2003). Teacher characteristics and student achievement gains: A review. Review of Educational Research, 73(1), 89–122. doi:10.3102/00346543073001089.

Rockoff, J. E. (2004). The impact of individual teachers on student achievement: Evidence from panel data. American Economic Review, 94(2), 247–252. doi:10.1257/0002828041302244.

Rivkin, S. G., Hanushek, E. A., & Kain, J. F. (2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417–458. doi:10.1111/j.1468-0262.2005.00584.x.

Baashar, Y., Alkawsi, G., Ali, N., Alhussian, H., & Bahbouh, H. T. (2021). Predicting student’s performance using machine learning methods: A systematic literature review. 2021 International Conference on Computer & Information Sciences (ICCOINS). doi:10.1109/iccoins49721.2021.9497185.

Papamitsiou, Z., & Economides, A. A. (2014). Learning analytics and educational data mining in practice: A systematic literature review of empirical evidence. Journal of Educational Technology & Society, 17(4), 49-64.

Alsariera, Y. A., Baashar, Y., Alkawsi, G., Mustafa, A., Alkahtani, A. A., & Ali, N. (2022). Assessment and Evaluation of Different Machine Learning Algorithms for Predicting Student Performance. Computational Intelligence and Neuroscience, 2022, 1–11. doi:10.1155/2022/4151487.

Feng, G., Fan, M., & Chen, Y. (2022). Analysis and Prediction of Students’ Academic Performance Based on Educational Data Mining. IEEE Access, 10, 19558–19571. doi:10.1109/ACCESS.2022.3151652.

Nabil, A., Seyam, M., & Abou-Elfetouh, A. (2021). Prediction of Students’ Academic Performance Based on Courses’ Grades Using Deep Neural Networks. IEEE Access, 9, 140731–140746. doi:10.1109/ACCESS.2021.3119596.

Al-Tameemi, G., Xue, J., Ajit, S., Kanakis, T., Hadi, I., Baker, T., Al-Khafajiy, M., & Al-Jumeily, R. (2021). A Deep Neural Network-Based Prediction Model for Students’ Academic Performance. 2021 14th International Conference on Developments in ESystems Engineering (DeSE). doi:10.1109/dese54285.2021.9719552.

Costa-Mendes, R., Oliveira, T., Castelli, M., & Cruz-Jesus, F. (2021). A machine learning approximation of the 2015 Portuguese high school student grades: A hybrid approach. Education and Information Technologies, 26(2), 1527–1547. doi:10.1007/s10639-020-10316-y.

Musso, M. F., Hernández, C. F. R., & Cascallar, E. C. (2020). Predicting key educational outcomes in academic trajectories: a machine-learning approach. Higher Education, 80(5), 875–894. doi:10.1007/s10734-020-00520-7.

Mengash, H. A. (2020). Using data mining techniques to predict student performance to support decision making in university admission systems. IEEE Access, 8, 55462–55470. doi:10.1109/ACCESS.2020.2981905.

Aydoğdu, Ş. (2020). Predicting student final performance using artificial neural networks in online learning environments. Education and Information Technologies, 25(3), 1913–1927. doi:10.1007/s10639-019-10053-x.

Li, F., Zhang, Y., Chen, M., & Gao, K. (2019). Which Factors Have the Greatest Impact on Student’s Performance. Journal of Physics: Conference Series, 1288(1). doi:10.1088/1742-6596/1288/1/012077.

Altaf, S., Soomro, W., & Rawi, M. I. M. (2019). Student Performance Prediction using Multi-Layers Artificial Neural Networks. Proceedings of the 2019 3rd International Conference on Information System and Data Mining-ICISDM 2019. doi:10.1145/3325917.3325919.

Lau, E. T., Sun, L., & Yang, Q. (2019). Modelling, prediction and classification of student academic performance using artificial neural networks. SN Applied Sciences, 1(9), 1–10,. doi:10.1007/s42452-019-0884-7.

Arunachalam, A. S., & Velmurugan, T. (2018). Analyzing student performance using evolutionary artificial neural network algorithm. International Journal of Engineering and Technology(UAE), 7(2.26), 67–73. doi:10.14419/ijet.v7i2.26.12537.

Mondal, A., & Mukherjee, J. (2018). An Approach to Predict a Student’s Academic Performance using Recurrent Neural Network (RNN). International Journal of Computer Applications, 181(6), 1–5. doi:10.5120/ijca2018917352.

Murphy, K. P. (2012). Machine learning: a probabilistic perspective. MIT Press, Cambridge, Massachusetts, United States.

Hastie, T., Friedman, J., & Tibshirani, R. (2001). The Elements of Statistical Learning. Springer Series in Statistics, New York, United States. doi:10.1007/978-0-387-21606-5.

Seabold, S., & Perktold, J. (2010). Statsmodels: Econometric and Statistical Modeling with Python. Proceedings of the 9th Python in Science Conference. doi:10.25080/majora-92bf1922-011.

Rosenblatt, F. (1958). The perceptron: A probabilistic model for information storage and organization in the brain. Psychological Review, 65(6), 386–408. doi:10.1037/h0042519.

Ramchoun, H., Idrissi, M. A. J., Ghanou, Y., & Ettaouil, M. (2017). Multilayer Perceptron. Proceedings of the 2nd International Conference on Big Data, Cloud and Applications. doi:10.1145/3090354.3090427.

Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1988). Learning Internal Representations by Error Propagation. Readings in Cognitive Science, 399–421, Elsevier, Amsterdam, Netherlands. doi:10.1016/b978-1-4832-1446-7.50035-2.

Basheer, I. A., & Hajmeer, M. (2000). Artificial neural networks: fundamentals, computing, design, and application. Journal of Microbiological Methods, 43(1), 3–31. doi:10.1016/s0167-7012(00)00201-3.

Bishop, C. M., & Nasrabadi, N. M. (2006). Pattern recognition and machine learning. Springer, New York, united States.

Chollet, F. (2015). About Keras. Available online: https://keras.io/about/ (accessed on June 2022)..

Narkhede, M. V., Bartakke, P. P., & Sutaone, M. S. (2021). A review on weight initialization strategies for neural networks. Artificial Intelligence Review, 55(1), 291–322. doi:10.1007/s10462-021-10033-z.

Glorot, X., & Bengio, Y. (2010, March). Understanding the difficulty of training deep feedforward neural networks. Proceedings of the 13th International Conference on Artificial Intelligence and Statistics, 249-256, 13-15 May, 2010, Chia Laguna Resort, Sardinia, Italy.

Lecun, Y., Bengio, Y., & Hinton, G. (2015). Deep learning. Nature, 521(7553), 436–444. doi:10.1038/nature14539.

Garbin, C., Zhu, X., & Marques, O. (2020). Dropout vs. batch normalization: an empirical study of their impact to deep learning. Multimedia Tools and Applications, 79(19–20), 12777–12815. doi:10.1007/s11042-019-08453-9.

Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: A simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(56), 1929–1958.

Kandel, I., & Castelli, M. (2020). The effect of batch size on the generalizability of the convolutional neural networks on a histopathology dataset. ICT Express, 6(4), 312–315. doi:10.1016/j.icte.2020.04.010.

Bengio, Y. (2012). Practical Recommendations for Gradient-Based Training of Deep Architectures. Neural Networks: Tricks of the Trade. Lecture Notes in Computer Science, 7700, Springer, Berlin, Germany. doi:10.1007/978-3-642-35289-8_26.

Ioffe, S., & Szegedy, C. (2015). Batch normalization: Accelerating deep network training by reducing internal covariate shift. 32nd International Conference on Machine Learning, 448–456, 6-11 July, 2015, Lille, France.

Kingma, D. P., & Ba, J. (2014). Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980 doi:10.48550/arXiv.1412.6980.

Bottou, L. (2012). Stochastic Gradient Descent Tricks. Neural Networks: Tricks of the Trade. Lecture Notes in Computer Science, 7700, Springer, Berlin, Germany. doi:10.1007/978-3-642-35289-8_25.

Tibshirani, R. (1997). The lasso method for variable selection in the Cox model. Statistics in medicine, 16(4), 385-395. doi:10.1002/(SICI)1097-0258(19970228)16:4<385::AID-SIM380>3.0.CO;2-3.

Li, L., Jamieson, K., DeSalvo, G., Rostamizadeh, A., & Talwalkar, A. (2018). Hyperband: A novel bandit-based approach to hyperparameter optimization. Journal of Machine Learning Research, 18, 1–52.

Bergstra, J., & Bengio, Y. (2012). Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13, 281–305.

Jamieson, K., & Talwalkar, A. (2016). Non-stochastic best arm identification and hyperparameter optimization. Proceeding of the 19th International Conference on Artificial intelligence and Statistics (AISTATS), 240-248, 9-11 May, Cadiz, Spain.

Brochu, E., Cora, V. M., & De Freitas, N. (2010). A tutorial on Bayesian optimization of expensive cost functions, with application to active user modeling and hierarchical reinforcement learning. arXiv preprint. doi:10.48550/arXiv.1012.2599

Kumar, S., & Srivistava, A. N. (2012). Bootstrap prediction intervals in non-parametric regression with applications to anomaly detection. 18th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 12-16 August, 2012, Beijing, China.

Pearce, T., Brintrup, A., Zaki, M., & Neely, A. (2018). High-quality prediction intervals for deep learning: A distribution-free, ensembled approach. Proceeding of the 35th International Conference International Conference on Machine Learning, 10-15 july, 2018, Stockholm, Sweden.

Rocha-Trindade, M. B. (2020). Chinese Community in Portugal: History , Migration , and Business. Gaudium Scendi Universidade Católica Portuguesa, 19, 15–42. doi: doi:10.34632/gaudiumsciendi.2020.10065.


Full Text: PDF

DOI: 10.28991/ESJ-2022-SIED-012

Refbacks

  • There are currently no refbacks.


Copyright (c) 2022 Ricardo Costa-Mendes