A Conceptual Framework for Assessing the Success of Online Proctoring

Ghadeer AL Najar 1* & Mohammad Nazir Ahmad 2
1 Infrastructure University Kuala Lumpur, Kajang, Malaysia
2 Infrastructure University Kuala Lumpur, Kajang, Malaysia
DOI- http://doi.org/10.37502/IJSMR.2024.7901

Full Text – PDF

Abstract

Evaluating online proctoring systems has become crucial due to the swift transition to online learning brought on by the COVID-19 pandemic in order to uphold academic integrity and achieve effective learning outcomes. Within educational institutions, this paper presents a conceptual framework for assessing online proctoring systems success, the proposed framework, grounded in the DeLone and McLean Information Systems (IS) Success Model, incorporates key dimensions such as user satisfaction, net benefits, service quality, system quality, and information quality, while also considering digital literacy and personal background as critical factors influencing system success. An online survey was administered as part of a quantitative research study, engaging a sample of 262 students. from an Omani educational institution. The findings reveal that net benefits, user satisfaction, and the intention to use online proctoring systems are significantly influenced by digital literacy. Morever, the intention to use the system was positively influenced by both system quality and service quality. The insights provide practical recommendations for optimizing online assessment strategies, enhancing academic integrity, and improving learning outcomes in the post-pandemic era.

Keywords: IS Success, DeLone and McLean Model, Online Proctoring, Digital Literacy, Personal Background, Academic Integrity.

References

  • Abdulkareem, A. K., & Ramli, R. M. (2021). Does digital literacy predict e-government performance? An extension of Delone and Mclean information system success model. Electronic Government, 17(4), 466. https://doi.org/10.1504/EG.2021.118103
  • Adanır, G. A. (2022). Student acceptance of online proctored exams. In J. L. Gómez Ramos & I. M. Gómez-Barreto (Eds.), Advances in educational technologies and instructional design (pp. 212–229). IGI Global. https://doi.org/10.4018/978-1-7998-9128-4.ch011
  • Al-Kofahi, M. K., Hassan, H., & Mohamad, R. (2020). Information systems success model: A review of literature. International Journal of Innovation, 12(8).
  • Alqahtani, M. A., Alamri, M. M., Sayaf, A. M., & Al-Rahmi, W. M. (2022). Investigating students’ perceptions of online learning use as a digital tool for educational sustainability during the COVID-19 pandemic. Frontiers in Psychology, 13. https://doi.org/10.3389/fpsyg.2022.886272
  • Arnò, S., Galassi, A., Tommasi, M., Saggino, A., & Vittorini, P. (2021). State-of-the-art of commercial proctoring systems and their use in academic online exams. International Journal of Distance Education Technologies, 19(2), 55–76. https://doi.org/10.4018/IJDET.20210401.oa3
  • Bulauitan, J. C. B., et al. (2022). OPEES: Online proctored entrance examination system with degree program recommender for colleges and universities. In 2022 IEEE 14th International Conference on Humanoid Nanotechnology Information Technology Communication and Control Environment and Management (HNICEM) (pp. 1–6). IEEE. https://doi.org/10.1109/HNICEM57413.2022.10109595
  • Burgess, S., & Sievertsen, H. H. (2020). Schools, skills, and learning: The impact of COVID-19 on education. Org. Retrieved from https://voxeu.org/article/impact-covid-19-education
  • Cassibba, R., et al. (2020). Teaching mathematics at distance: A challenge for universities. Education Sciences, 11(1), 1.
  • Cidral, W. A., Aparicio, M., & Oliveira, T. (2020). Students’ long-term orientation role in e-learning success: A Brazilian study. Heliyon, 6(12), e05735. https://doi.org/10.1016/j.heliyon.2020.e05735
  • Clark, T. M., Callam, C. S., Paul, N. M., Stoltzfus, M. W., & Turner, D. (2020). Testing in the time of COVID-19: A sudden transition to unproctored online exams. Journal of Chemical Education, 97(9), 3413-3417.
  • De La Roca, M., Morales, M., & Garcia-Cabot, A. (2022). The impact of online proctoring on students’ perception and level of satisfaction in a MOOC. In 2022 IEEE Learning with MOOCs (LWMOOCS) (pp. 23–27). IEEE. https://doi.org/10.1109/LWMOOCS53067.2022.9927956
  • DeLone, W., & McLean, E. (1992). Information systems success: The quest for the dependent variable. Information Systems Research, 3, 60–95. https://doi.org/10.1287/isre.3.1.60
  • DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9-30.
  • Dendir, S., & Maxwell, R. S. (2020). Cheating in online courses: Evidence from online proctoring. Computers in Human Behavior Reports, 2, 100033. https://doi.org/10.1016/j.chbr.2020.100033
  • Drew, H. (2020, April 1). Mass school closures in the wake of the coronavirus are driving a new wave of student surveillance. The Washington Post. Retrieved from https://www.washingtonpost.com/technology/2020/04/01/onlineproctoring-college-exams-coronavirus
  • Elsalem, L., Al-Azzam, N., Jum’ah, A. A., & Obeidat, N. (2021). Remote e-exams during COVID-19 pandemic: A cross-sectional study of students’ preferences and academic dishonesty in faculties of medical sciences. Annals of Medicine and Surgery, 62, 326–333. https://doi.org/10.1016/j.amsu.2021.01.054
  • González-González, C. S., Infante-Moro, A., & Infante-Moro, J. C. (2020). Implementation of e-proctoring in online teaching: A study about motivational factors. Sustainability, 12(8), 3488. https://doi.org/10.3390/su12083488
  • Huang, F., Sánchez-Prieto, J. C., Teo, T., García-Peñalvo, F. J., Sánchez, E. M. T., & Zhao, C. (2020). The influence of university students’ learning beliefs on their intentions to use mobile technologies in learning: A study in China and Spain. Educational Technology Research and Development, 68, 3547-3565. https://doi.org/10.1007/s11423-020-09795-5
  • Hussein, M. J., Yusuf, J., Deb, A. S., Fong, L., & Naidu, S. (2020). An evaluation of online proctoring tools. Open Praxis, 12(4), 509. https://doi.org/10.5944/openpraxis.12.4.1113
  • Infante-Moro, A., Infante-Moro, J. C., Gallardo-Pérez, J., & Martínez-López, F. J. (2022). Key factors in the implementation of e-proctoring in the Spanish university system. Sustainability, 14(13), 8112. https://doi.org/10.3390/su14138112
  • Junus, K., Santoso, H. B., Putra, P. O. H., Gandhi, A., & Siswantining, T. (2021). Lecturer readiness for online classes during the pandemic: A survey research. Education Sciences, 11(3), 139. https://doi.org/10.3390/educsci11030139
  • Lee, J. W. (2010). Online support service quality, online learning acceptance, and student satisfaction. The Internet and Higher Education, 13(4), 277–283. https://doi.org/10.1016/j.iheduc.2010.08.002
  • Maboe, K. A., & Tomas, N. (2023). Online assessments and COVID-19: A qualitative study of undergraduate nursing students in Southern Africa. International Journal of Africa Nursing Sciences, 19, 100590. https://doi.org/10.1016/j.ijans.2023.100590
  • Martin, A. J. (2020). How to optimize online learning in the age of coronavirus (COVID-19): A 5-point guide for educators. UNSW Newsroom.
  • Nafrees, A. C. M., Liyanage, S. R., & Dias, N. G. J. (2022). Online assessment technologies and future trends: A systematic review.
  • Osman, M. E. (2020). Global impact of COVID-19 on education systems: The emergency remote teaching at Sultan Qaboos University. Journal of Education for Teaching International Research and Pedagogy, 46, 463–471.
  • Paudel, P. (2021). Online education: Benefits, challenges, and strategies during and after COVID-19 in higher education. International Journal on Studies in Education, 3, 70–85.
  • Petter, S., DeLone, W., & McLean, E. R. (2013). Information systems success: The quest for the independent variables. Journal of Management Information Systems, 29(4), 7–62. https://doi.org/10.2753/mis0742-1222290401
  • Raman, R., Vachharajani, S. B., & Nedungadi, P. (2021). Adoption of online proctored examinations by university students during COVID-19: Innovation diffusion study. Education and Information Technologies, 26(6), 7339–7358. https://doi.org/10.1007/s10639-021-10581-5
  • Riady, Y., Sofwan, M., Mailizar, M., Alqahtani, T. M., Yaqin, L. N., & Habibi, A. (2023). How can we assess the success of information technologies in digital libraries? Empirical evidence from Indonesia. International Journal of Information Management Data Insights, 3(2), 100192. https://doi.org/10.1016/j.jjimei.2023.100192
  • Rokhman, F., Mukhibad, H., Hapsoro, B. B., & Nurkhin, A. (2022). E-learning evaluation during the COVID-19 pandemic era based on the updated of Delone and McLean information systems success model
  • Roky, H., & Meriouh, Y. A. (2015). Evaluation by users of an industrial information system (XPPS) based on the DeLone and McLean model for IS success. Procedia Economics and Finance, 26, 903–913. https://doi.org/10.1016/s2212-5671(15)00903-x
  • Sabeh, H. N., Husin, M. H., Kee, D. M. H., Baharudin, A. S., & Abdullah, R. (2021). A systematic review of the DeLone and McLean model of information systems success in an e-learning context (2010–2020). IEEE Access, 9, 81210–81235. https://doi.org/10.1109/ACCESS.2021.3084815
  • Tam, C., & Oliveira, T. (2017). Understanding mobile banking individual performance: The DeLone & McLean model and the moderating effects of individual culture. Internet Research, 27. https://doi.org/10.1108/IntR-05-2016-0117
  • Tarkar, P. (2020). Impact of COVID-19 pandemic on education system. International Journal of Advanced Science and Technology, 29, 3812–3814.
  • Vayre, E., & Vonthron, A.-M. (2019). Relational and psychological factors affecting exam participation and student achievement in online college courses. The Internet and Higher Education, 43, 100671. https://doi.org/10.1016/j.iheduc.2018.07.001
  • Yazıcı, S., Yildiz-Durak, H., Aksu Dunya, B., & Şentürk, B. (2022). Online versus face‐to‐face cheating: The prevalence of cheating behaviours during the pandemic compared to the pre‐pandemic among Turkish University students. Journal of Computer Assisted Learning, 39, n/a-n/a. https://doi.org/10.1111/jcal.12743
  • Yuniarto, D., Firmansyah, E., Saeppani, A., & Junaedi, D. I. (2021). Information system project success based on user experience at the village administration office. In 2021 9th International Conference on Cyber and IT Service Management (CITSM) (pp. 1–7). IEEE. https://doi.org/10.1109/CITSM52892.2021.9588978