1. Home
  2. Vol 5 (2021)
  3. Khel

Real-Time Monitoring of COVID-19 SOP in Public Gathering Using Deep Learning Technique

Muhammad Haris Kaka Khel, Kushsairy Kadir, Waleed Albattah, Sheroz Khan, MNMM Noor, Haidawati Nasir, Shabana Habib, Muhammad Islam, Akbar Khan

Abstract


Crowd management has attracted serious attention under the prevailing pandemic conditions of COVID-19, emphasizing that sick persons do not become a source of virus transmission. World Health Organization (WHO) guidelines include maintaining a safe distance and wearing a mask in gatherings as part of standard operating procedures (SOP), considered thus far the most effective preventive measures to protect against COVID-19. Several methods and strategies have been used to construct various face detection and social distance detection models. In this paper, a deep learning model is presented to detect people without masks and those not keeping a safe distance to contain the virus. It also counts individuals who violate the SOP. The proposed model employs the Single Shot Multi-box Detector as a feature extractor, followed by Spatial Pyramid Pooling (SPP) to integrate the extracted features to improve the model's detecting capabilities. The MobilenetV2 architecture as a framework for the classifier makes the model highly light, fast, and computationally efficient, allowing it to be employed in embedded devices to do real-time mask and social distance detection, which is the sole objective of this research. This paper's technique yields an accuracy score of 99% and reduces the loss to 0.04%.

 

Doi: 10.28991/esj-2021-SPER-14

Full Text: PDF


Keywords


COVID-19; Social Distancing; Crowd Management; Hajj Umrah; Mask Detection; Convolutional Neural Network.

References


Martella, C., J. Li, C. Conrado, and A. Vermeeren. “On Current Crowd Management Practices and the Need for Increased Situation Awareness, Prediction, and Intervention.” Safety Science 91 (2017): 381–93. doi:10.1016/j.ssci.2016.09.006.

Filingeri, Victoria, Ken Eason, Patrick Waterson, and Roger Haslam. “Factors Influencing Experience in Crowds – The Participant Perspective.” Applied Ergonomics 59 (2017): 431–41. doi:10.1016/j.apergo.2016.09.009.

Zhao, Hantao, Tyler Thrash, Mubbasir Kapadia, Katja Wolff, Christoph Holscher, Dirk Helbing, and Victor R. Schinazi. “Assessing Crowd Management Strategies for the 2010 Love Parade Disaster Using Computer Simulations and Virtual Reality.” Journal of the Royal Society Interface 17, no. 167 (2020): 20200116. doi:10.1098/rsif.2020.0116.

Toyokawa, Wataru, Andrew Whalen, and Kevin N. Laland. “Social Learning Strategies Regulate the Wisdom and Madness of Interactive Crowds.” Nature Human Behaviour 3, no. 2 (2019): 183–93. doi:10.1038/s41562-018-0518-x.

Cai, Qingxian, Minghui Yang, Dongjing Liu, Jun Chen, Dan Shu, Junxia Xia, Xuejiao Liao, et al. “Experimental Treatment with Favipiravir for COVID-19: An Open-Label Control Study.” Engineering 6, no. 10 (2020): 1192–98. doi:10.1016/j.eng.2020.03.007.

Betsch, Cornelia, Lars Korn, Philipp Sprengholz, Lisa Felgendreff, Sarah Eitze, Philipp Schmid, and Robert Böhm. “Social and Behavioral Consequences of Mask Policies during the COVID-19 Pandemic.” Proceedings of the National Academy of Sciences of the United States of America 117, no. 36 (2020): 21851–53. doi:10.1073/pnas.2011674117.

Eikenberry, Steffen E., Marina Mancuso, Enahoro Iboi, Tin Phan, Keenan Eikenberry, Yang Kuang, Eric Kostelich, and Abba B. Gumel. “To Mask or Not to Mask: Modeling the Potential for Face Mask Use by the General Public to Curtail the COVID-19 Pandemic.” Infectious Disease Modelling 5 (2020): 293–308. doi:10.1016/j.idm.2020.04.001.

Meyer, Jacob, Cillian McDowell, Jeni Lansing, Cassandra Brower, Lee Smith, Mark Tully, and Matthew Herring. “Changes in Physical Activity and Sedentary Behavior in Response to Covid-19 and Their Associations with Mental Health in 3052 Us Adults.” International Journal of Environmental Research and Public Health 17, no. 18 (2020): 1–13. doi:10.3390/ijerph17186469.

Hsieh, Chih Chia, Chih Hao Lin, William Yu Chung Wang, David J. Pauleen, and Jengchung Victor Chen. “The Outcome and Implications of Public Precautionary Measures in Taiwan–Declining Respiratory Disease Cases in the COVID-19 Pandemic.” International Journal of Environmental Research and Public Health 17, no. 13 (2020): 1–10. doi:10.3390/ijerph17134877.

Ul Haq, Shamsheer, Pomi Shahbaz, and Ismet Boz. “Knowledge, Behavior and Precautionary Measures Related to COVID-19 Pandemic among the General Public of Punjab Province, Pakistan.” Journal of Infection in Developing Countries 14, no. 8 (2020): 823–35. doi:10.3855/jidc.12851.

Khan, Akbar, Jawad Ali Shah, Kushsairy Kadir, Waleed Albattah, and Faizullah Khan. “Crowd Monitoring and Localization Using Deep Convolutional Neural Network: A Review.” Applied Sciences (Switzerland) 10, no. 14 (2020): 4781. doi:10.3390/app10144781.

Loey, Mohamed, Gunasekaran Manogaran, Mohamed Hamed N. Taha, and Nour Eldeen M. Khalifa. “Fighting against COVID-19: A Novel Deep Learning Model Based on YOLO-v2 with ResNet-50 for Medical Face Mask Detection.” Sustainable Cities and Society 65 (2021): 102600. doi:10.1016/j.scs.2020.102600.

Meenpal, Toshanlal, Ashutosh Balakrishnan, and Amit Verma. “Facial Mask Detection Using Semantic Segmentation.” 2019 4th International Conference on Computing, Communications and Security, ICCCS 2019, 2019. doi:10.1109/CCCS.2019.8888092.

Jignesh Chowdary, G., Narinder S. Punn, Sanjay Kumar Sonbhadra, and Sonali Agarwal. “Face Mask Detection Using Transfer Learning of InceptionV3.” In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 12581 LNCS (2020):81–90. Cham: Springer. doi:10.1007/978-3-030-66665-1_6.

Singh, Sunil, Umang Ahuja, Munish Kumar, Krishan Kumar, and Monika Sachdeva. “Face Mask Detection Using YOLOv3 and Faster R-CNN Models: COVID-19 Environment.” Multimedia Tools and Applications 80, no. 13 (2021): 19753–68. doi:10.1007/s11042-021-10711-8.

Chavda, Amit, Jason Dsouza, Sumeet Badgujar, and Ankit Damani. “Multi-Stage CNN Architecture for Face Mask Detection.” In 2021 6th International Conference for Convergence in Technology, I2CT 2021, 1–8. IEEE, 2021. doi:10.1109/I2CT51068.2021.9418207.

Albattah, Waleed, Muhammad Haris Kaka Khel, Shabana Habib, Muhammad Islam, Sheroz Khan, and Kushsairy Abdul Kadir. “Hajj Crowd Management Using CNN-Based Approach.” Computers, Materials and Continua, 2020. doi:10.32604/cmc.2020.014227.

Gu, Jiuxiang, Zhenhua Wang, Jason Kuen, Lianyang Ma, Amir Shahroudy, Bing Shuai, Ting Liu, et al. “Recent Advances in Convolutional Neural Networks.” Pattern Recognition 77 (May 2018): 354–377. doi:10.1016/j.patcog.2017.10.013.

Yamashita, Rikiya, Mizuho Nishio, Richard Kinh Gian Do, and Kaori Togashi. “Convolutional Neural Networks: An Overview and Application in Radiology.” Insights into Imaging 9, no. 4 (2018): 611–29. doi:10.1007/s13244-018-0639-9.

Hamouda, Maissa, and Med Salim Bouhlel. “Modified Convolutional Neural Networks Architecture for Hyperspectral Image Classification (Extra-Convolutional Neural Networks).” IET Image Processing, 2021. doi:10.1049/ipr2.12169.

Krizhevsky, Alex, Ilya Sutskever, and Geoffrey E. Hinton. “ImageNet Classification with Deep Convolutional Neural Networks.” Communications of the ACM 60, no. 6 (2017): 84–90. doi:10.1145/3065386.

Zhao, Zhong Qiu, Peng Zheng, Shou Tao Xu, and Xindong Wu. “Object Detection with Deep Learning: A Review.” IEEE Transactions on Neural Networks and Learning Systems 30, no. 11 (2019): 3212–32. doi:10.1109/TNNLS.2018.2876865.

Uijlings, J. R.R., K. E.A. Van De Sande, T. Gevers, and A. W.M. Smeulders. “Selective Search for Object Recognition.” International Journal of Computer Vision 104, no. 2 (2013): 154–71. doi:10.1007/s11263-013-0620-5.

Liu, Wei, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng Yang Fu, and Alexander C. Berg. “SSD: Single Shot Multibox Detector.” In Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 9905 LNCS (2016):21–37. Cham: Springer. doi:10.1007/978-3-319-46448-0_2.

Lin, Tsung Yi, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollar. “Focal Loss for Dense Object Detection.” In IEEE Transactions on Pattern Analysis and Machine Intelligence, 42:318–27, 2020. doi:10.1109/TPAMI.2018.2858826.

Tan, Mingxing, Ruoming Pang, and Quoc V. Le. “EfficientDet: Scalable and Efficient Object Detection.” In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 10778–87, 2020. doi:10.1109/CVPR42600.2020.01079.

Zhang, Shifeng, Longyin Wen, Zhen Lei, and Stan Z. Li. “RefineDet++: Single-Shot Refinement Neural Network for Object Detection.” IEEE Transactions on Circuits and Systems for Video Technology 31, no. 2 (2021): 674–87. doi:10.1109/TCSVT.2020.2986402.

Zhu, Chenchen, Yutong Zheng, Khoa Luu, and Marios Savvides. “CMS-RCNN: Contextual Multi-Scale Region-Based CNN for Unconstrained Face Detection.” In Advances in Computer Vision and Pattern Recognition, PartF1:57–79. Cham: Springer, 2017. doi:10.1007/978-3-319-61657-5_3.

Ejaz, Md Sabbir, Md Rabiul Islam, Md Sifatullah, and Ananya Sarker. “Implementation of Principal Component Analysis on Masked and Non-Masked Face Recognition.” In 1st International Conference on Advances in Science, Engineering and Robotics Technology 2019, ICASERT 2019, 1–5. IEEE, 2019. doi:10.1109/ICASERT.2019.8934543.

Loey, Mohamed, Gunasekaran Manogaran, Mohamed Hamed N. Taha, and Nour Eldeen M. Khalifa. “A Hybrid Deep Transfer Learning Model with Machine Learning Methods for Face Mask Detection in the Era of the COVID-19 Pandemic.” Measurement: Journal of the International Measurement Confederation 167 (2021): 108288. doi:10.1016/j.measurement.2020.108288.

Wang, Zhongyuan, Guangcheng Wang, Baojin Huang, Zhangyang Xiong, Qi Hong, Hao Wu, Peng Yi, et al. “Masked Face Recognition Dataset and Application,” 2020. http://arxiv.org/abs/2003.09093.

Cabani, Adnane, Karim Hammoudi, Halim Benhabiles, and Mahmoud Melkemi. “MaskedFace-Net – A Dataset of Correctly/Incorrectly Masked Face Images in the Context of COVID-19.” Smart Health 19 (2021): 100144. doi:10.1016/j.smhl.2020.100144.

Li, Chong, Rong Wang, Jinze Li, and Linyu Fei. “Face Detection Based on YOLOv3.” In Advances in Intelligent Systems and Computing, 1031 AISC:277–84. Singapore: Springer, 2020. doi:10.1007/978-981-13-9406-5_34.

Zhao, Haipeng, Yang Zhou, Long Zhang, Yangzhao Peng, Xiaofei Hu, Haojie Peng, and Xinyue Cai. “Mixed YOLOv3-LITE: A Lightweight Real-Time Object Detection Method.” Sensors (Switzerland) 20, no. 7 (2020). doi:10.3390/s20071861.

Yang, Yang, and Hongmin Deng. “Gc-Yolov3: You Only Look Once with Global Context Block.” Electronics (Switzerland) 9, no. 8 (2020): 1–14. doi:10.3390/electronics9081235.

Punn, Narinder Singh, Sanjay Kumar Sonbhadra, Sonali Agarwal, and Gaurav Rai. “Monitoring COVID-19 Social Distancing with Person Detection and Tracking via Fine-Tuned YOLO v3 and Deepsort Techniques,” 2020. http://arxiv.org/abs/2005.01385.

Ma, Tao, Fen Wang, Jianjun Cheng, Yang Yu, and Xiaoyun Chen. “A Hybrid Spectral Clustering and Deep Neural Network Ensemble Algorithm for Intrusion Detection in Sensor Networks.” Sensors (Switzerland) 16, no. 10 (2016): 1701. doi:10.3390/s16101701.

Ainslie, Kylie E. C., Caroline E. Walters, Han Fu, Sangeeta Bhatia, Haowei Wang, Xiaoyue Xi, Marc Baguelin, et al. “Evidence of Initial Success for China Exiting COVID-19 Social Distancing Policy after Achieving Containment.” Wellcome Open Research 5 (October 1, 2020): 81. doi:10.12688/wellcomeopenres.15843.2.

Nguyen, Cong T., Yuris Mulya Saputra, Nguyen Van Huynh, Ngoc-Tan Nguyen, Tran Viet Khoa, Bui Minh Tuan, Diep N. Nguyen, et al. “A Comprehensive Survey of Enabling and Emerging Technologies for Social Distancing—Part I: Fundamentals and Enabling Technologies.” IEEE Access 8 (2020): 153479–153507. doi:10.1109/access.2020.3018140..

Prem, Kiesha, Yang Liu, Timothy W. Russell, Adam J. Kucharski, Rosalind M. Eggo, Nicholas Davies, Stefan Flasche, et al. “The Effect of Control Strategies to Reduce Social Mixing on Outcomes of the COVID-19 Epidemic in Wuhan, China: A Modelling Study.” The Lancet Public Health 5, no. 5 (2020): e261–70. doi:10.1016/S2468-2667(20)30073-6.

Pouw, Caspar A.S., Federico Toschi, Frank van Schadewijk, and Alessandro Corbetta. “Monitoring Physical Distancing for Crowd Management: Real-Time Trajectory and Group Analysis.” PLoS ONE 15, no. 10 October (2020): 240963. doi:10.1371/journal.pone.0240963.

Ahmad, Misbah, Imran Ahmed, Fakhri Alam Khan, Fawad Qayum, and Hanan Aljuaid. “Convolutional Neural Network–Based Person Tracking Using Overhead Views.” International Journal of Distributed Sensor Networks 16, no. 6 (2020): 1550147720934738. doi:10.1177/1550147720934738.

Jin, Tingxu, Jun Li, Jun Yang, Jiawei Li, Feng Hong, Hai Long, Qihong Deng, et al. “SARS-CoV-2 Presented in the Air of an Intensive Care Unit (ICU).” Sustainable Cities and Society 65 (2021): 102446. doi:10.1016/j.scs.2020.102446.

Bhattacharya, Sweta, Praveen Kumar Reddy Maddikunta, Quoc Viet Pham, Thippa Reddy Gadekallu, Siva Rama Krishnan S, Chiranji Lal Chowdhary, Mamoun Alazab, and Md Jalil Piran. “Deep Learning and Medical Image Processing for Coronavirus (COVID-19) Pandemic: A Survey.” Sustainable Cities and Society 65 (2021): 102589. doi:10.1016/j.scs.2020.102589.

Anisimov, Dmitriy, and Tatiana Khanova. “Towards Lightweight Convolutional Neural Networks for Object Detection.” In 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance, AVSS 2017, 1–8. IEEE, 2017. doi:10.1109/AVSS.2017.8078500.

Howard, Andrew G., Menglong Zhu, Bo Chen, Dmitry Kalenichenko, Weijun Wang, Tobias Weyand, Marco Andreetto, and Hartwig Adam. “MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications,” 2017. http://arxiv.org/abs/1704.04861.

Sanjaya, Samuel Ady, and Suryo Adi Rakhmawan. “Face Mask Detection Using MobileNetV2 in the Era of COVID-19 Pandemic.” In 2020 International Conference on Data Analytics for Business and Industry: Way Towards a Sustainable Economy, ICDABI 2020, 1–5. IEEE, 2020. doi:10.1109/ICDABI51230.2020.9325631.

Sandler, Mark, Andrew Howard, Menglong Zhu, Andrey Zhmoginov, and Liang Chieh Chen. “MobileNetV2: Inverted Residuals and Linear Bottlenecks.” In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 4510–20, 2018. doi:10.1109/CVPR.2018.00474.

Howard, Andrew, Mark Sandler, Bo Chen, Weijun Wang, Liang Chieh Chen, Mingxing Tan, Grace Chu, et al. “Searching for MobileNetV3.” In Proceedings of the IEEE International Conference on Computer Vision, 2019-October:1314–24, 2019. doi:10.1109/ICCV.2019.00140.

Buslaev, Alexander, Vladimir I. Iglovikov, Eugene Khvedchenya, Alex Parinov, Mikhail Druzhinin, and Alexandr A. Kalinin. “Albumentations: Fast and Flexible Image Augmentations.” Information (Switzerland) 11, no. 2 (2020): 125. doi:10.3390/info11020125.

Saleh, Abeer M., and Talal H. “Analysis and Best Parameters Selection for Person Recognition Based on Gait Model Using CNN Algorithm and Image Augmentation.” Journal of Big Data 8, no. 1 (2021): 1–20. doi:10.1186/s40537-020-00387-6.

Fu, Xiaomeng, and Huiming Qu. “Research on Semantic Segmentation of High-Resolution Remote Sensing Image Based on Full Convolutional Neural Network.” 2018 12th International Symposium on Antennas, Propagation and EM Theory (ISAPE) (December 2018). doi:10.1109/isape.2018.8634106.

Simonyan, Karen, and Andrew Zisserman. “Very Deep Convolutional Networks for Large-Scale Image Recognition.” 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, 2015.

Szegedy, Christian, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. “Rethinking the Inception Architecture for Computer Vision.” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (June 2016). doi:10.1109/cvpr.2016.308.

He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. “Deep Residual Learning for Image Recognition.” In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2016-December: 770–78, 2016. doi:10.1109/CVPR.2016.90.

Inamdar, Madhura, and Ninad Mehendale. “Real-Time Face Mask Identification Using Facemasknet Deep Learning Network.” SSRN Electronic Journal 3663305 (2020). doi:10.2139/ssrn.3663305.

Xu, Jianfeng, Yuanjian Zhang, and Duoqian Miao. “Three-Way Confusion Matrix for Classification: A Measure Driven View.” Information Sciences 507 (2020): 772–94. doi:10.1016/j.ins.2019.06.064.

Zhang, Xiaosong, Fang Wan, Chang Liu, Xiangyang Ji, and Qixiang Ye. “Learning to Match Anchors for Visual Object Detection.” IEEE Transactions on Pattern Analysis and Machine Intelligence (2021): 1–1. doi:10.1109/tpami.2021.3050494.

Rezatofighi, Hamid, Nathan Tsoi, Junyoung Gwak, Amir Sadeghian, Ian Reid, and Silvio Savarese. “Generalized Intersection over Union: A Metric and a Loss for Bounding Box Regression.” In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition, (2019-June):658–66, 2019. doi:10.1109/CVPR.2019.00075.

Hou, Feifei, Wentai Lei, Shuai Li, Jingchun Xi, Mengdi Xu, and Jiabin Luo. “Improved Mask R-CNN with Distance Guided Intersection over Union for GPR Signature Detection and Segmentation.” Automation in Construction 121 (2021): 103414. doi:10.1016/j.autcon.2020.103414.

Ahmed, Imran, Misbah Ahmad, and Gwanggil Jeon. “Social Distance Monitoring Framework Using Deep Learning Architecture to Control Infection Transmission of COVID-19 Pandemic.” Sustainable Cities and Society 69 (2021): 102777. doi:10.1016/j.scs.2021.102777.

Loey, Mohamed, Gunasekaran Manogaran, Mohamed Hamed N. Taha, and Nour Eldeen M. Khalifa. “A Hybrid Deep Transfer Learning Model with Machine Learning Methods for Face Mask Detection in the Era of the COVID-19 Pandemic.” Measurement 167 (January 2021): 108288. doi:10.1016/j.measurement.2020.108288.


Full Text: PDF

DOI: 10.28991/esj-2021-SPER-14

Refbacks



Copyright (c) 2021 Sheroz Khan