Academic student satisfaction and perceived performance in the e

The outbreak of the COVID-19 pandemic has dramatically shaped higher education and seen the distinct rise of e-learning as a compulsory element of the modern educational landscape. Accordingly, this study highlights the factors which have influenced how students perceive their academic performance during this emergency changeover to e-learning. The empirical analysis is performed on a sample of 10,092 higher education students from 10 countries across 4 continents during the pandemic’s first wave through an online survey. A structural equation model revealed the quality of e-learning was mainly derived from service quality, the teacher’s active role in the process of online education, and the overall system quality, while the students’ digital competencies and online interactions with their colleagues and teachers were considered to be slightly less important factors. The impact of e-learning quality on the students’ performance was strongly mediated by their satisfaction with e-learning. In general, the model gave quite consistent results across countries, gender, study fields, and levels of study. The findings provide a basis for policy recommendations to support decision-makers incorporate e-learning issues in the current and any new similar circumstances.

Citation: Keržič D, Alex JK, Pamela Balbontín Alvarado R, Bezerra DdS, Cheraghi M, Dobrowolska B, et al. (2021) Academic student satisfaction and perceived performance in the e-learning environment during the COVID-19 pandemic: Evidence across ten countries. PLoS ONE 16(10): e0258807. https://doi.org/10.1371/journal.pone.0258807

Editor: Dejan Dragan, Univerza v Mariboru, SLOVENIA

Received: July 21, 2021; Accepted: October 5, 2021; Published: October 20, 2021

Copyright: © 2021 Keržič et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The data presented in this study are available in Supporting Information (see S1 Dataset).

Funding: This research and the APC were funded by the Slovenian Research Agency grant number P5-0093. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

COVID-19, as a global public health crisis, has been brutal on the economy, education and food security of people all around the world, regardless of national boundaries. Affected sectors include tertiary education, featuring one of the worst disruptions during the lockdown periods given that most countries have tried to keep their essential economic activities running. Still, such activities did not extend to higher education institutions (HEIs), which were closed completely after the suspension of face-to-face activities in an effort to avoid the virus spreading among their students and staff and, in turn, the general population.

Nevertheless, HEIs have continued to offer education by using various digital media, e-learning platforms and video conferencing systems. The result is that e-learning has become a compulsory educational process. Many HEIs were even encountering this mode of delivery for the first time, making the transition particularly demanding for them since no time was available to organize and adapt to the new landscape for education. Both teachers and students today find themselves in a new environment, where some seem better at adapting than others. This means the quality of teaching and learning call for special consideration. In this article, the term “e-learning” refers to all forms of delivery for teaching and learning purposes that rely on different information communication technologies (ICTs) during the COVID-19 lockdown.

To understand COVID-19’s impact on the academic sphere, especially on students’ learning effectiveness, we explored the factors influencing how students have perceived their academic performance since HEIs cancelled their onsite classes. Students’ satisfaction in e-learning environments has been studied ever since the new mode of delivery via ICT first appeared (e.g. [1]), with researchers having tried to reveal factors that shape success with the implementation of e-learning systems (e.g. [2–4]), yet hitherto little attention has been paid to this topic in the current pandemic context. This study thus aims to fill this gap by investigating students’ e-learning experience in this emergency shift. Therefore, the questions we address in the paper are:

According to previous research and considering the new circumstances (e.g. [5–7]), we propose a model for explaining students’ perceived academic performance. In order to identify relevant variables positively affecting students’ performance, we use data from the multi-country research study “Impacts of the COVID-19 Pandemic on Life of Higher Education Students”, coordinated by the Faculty of Public Administration, University of Ljubljana, Slovenia [8]. Structural equation modelling (SEM) is applied to explore the causal relationships among latent concepts, measured by several observed items. Since the SEM approach has a long history of successful applications in research, especially in the social sciences [9, 10] and also in the educational context [11], it offers a suitable statistical framework that allows us to define a conceptual model containing interrelated variables connected to e-learning’s effect on students’ performance [9, 10].

This study significantly contributes to understanding of students’ satisfaction and performance in the online environment. The research findings may be of interest to higher education planners, teachers, support services and students all around the world.

According to the International Association of Universities (IAU), over 1.5 billion students and young people around the globe have been affected by the suspension of school and university classes due to the pandemic [12]. Thus, to maintain continuity in learning while working on containing the pandemic, countries have had to rely hugely on the e-learning modality, which may be defined as learning experiences with the assistance of online technologies. However, most HEIs were unprepared to effectively deal with the abrupt switch from on-site classes to on-line platforms, either due to infrastructure unavailability or the lack of suitable pedagogic projects [13, 14]. To understand the mechanism and depth of the effects of COVID-19, many research studies have been carried out across the world.

Before COVID-19, as new technologies were developed, different e-learning modalities like blended learning and massive open online courses were gradually spreading around the world during the last few decades [15, 16]. Hence, e-learning was deeply rooted in adequate planning and instructional design based on the available theories and models. It should be noted at the outset that what has been installed at many HEIs during the pandemic cannot even be considered e-learning, but emergency remote teaching, which is not necessarily as efficient and effective as a well-established and strategically organized system [17]. Still, all over the world online platforms, for example MS Teams, Moodle, Google Classroom, and Blackboard are in use. Although e-learning offers some educational continuity when it comes to academic learning, technical education has suffered doubly since the social distancing requirements have disrupted the implementation of both practical and work-based learning activities, which are critical for educational success [18].

According to Puljak et al. [19], while students have mostly been satisfied with how they have adapted to e-learning, they have missed the lectures and personal communication with their teachers. They declared that e-learning could not replace regular learning experiences; only 18.9% of students were interested in e-learning exclusively in the long run. Inadequate readiness among teachers and students to abruptly switch from face-to-face teaching to a digital platform has been reported [20].

The closure of universities and schools due to the COVID-19 pandemic has led to several adverse consequences for students, such as interrupted learning, giving students fewer opportunities to grow and develop [21]. This shift has resulted in various psychological changes among both students and teachers [22] and greatly affected their performance. Tutoring system in higher education is an established model of support, advice, and guidance for students in higher education with a purpose to improve motivation and success and prevent drop-out. Pérez-Jorge et al. [23] studied the effectiveness of the university tutoring system during the Covid-19 pandemic. The relation between tutor and student is based on collaboration and communication, which required to adopting quickly to the new situations using different communication technology. The research focused on four different forms of tutoring: in person, by e-mail, using virtual tutoring (Hangout/Google Meet) and WhatsApp. They pointed out that synchronous models and frequent daily communication are essential for effective and successful tutoring system where application WhatsApp, with synchronous communication by messages and video calls, is the form with which students were most satisfied and gain the most from it.

The goal of shifting teaching and learning over to online platforms is to minimize in-person interactions to reduce the risk of acquiring COVID-19 through physical contact. The form of interaction has also moved from offline mode to online mode. Students interact with each other in online platforms for their close group and also for larger groups [24, 25]. Many clinical skills are learned through direct interactions with patients and caregivers, one area that has been badly affected by the switch to e-learning platforms [26–28].

Student satisfaction has been shown to be a reliable proxy for measuring the success of implementing ICT-based initiatives in e-learning environments. Scholars have documented a strong relationship between how students perceive their academic performance and how satisfied students are with their e-learning environments [1, 29–31].

The literature reveals important antecedents related to students’ satisfaction with e-learning training, such as online interactions [32, 33], computer efficiency [34, 35], online skills [36], teacher support [34, 37, 38], course design [29, 39], teacher feedback [40], quality of information and activity [1] and technical support [34, 36, 41]. During the COVID-19 pandemic, environmental aspects like temperature, lighting and noise have been identified as significant determinants of students’ e-learning performance [42].

Sun et al. [1] consider the effect of overall quality–as a holistic construct–on satisfaction with the e-learning system. Their research identifies several quality factors that facilitate e-learning through factors associated with: learners (mental health, self-efficacy and attitude of the learner), teachers/instructors (attitude and response timelines assigned by the teacher), technology (quality of technology and the Internet), curriculum (quality and flexibility of the curriculum), design (usefulness and complexity of the design) and environment (interactiveness and assessment diversity). This pandemic has challenged HEIs around the world since e-learning requires physical equipment such as computers, servers, learning and communication platforms, but also software applications, operating systems and experts in the use of these technologies. However, teachers must also possess sufficient digital competencies if they are to use ICT effectively in the learning process.

One of the most relevant factors related to success in implementing e-learning relates to how online education is conducted [19]. This includes receiving timely feedback, teachers’ efforts to be organized, delivering online lectures (and recording them), adapting instructions to this learning model, and helping students follow the courses and look for feedback on their experiences. In some cases, students have not been appropriately guided to follow their courses, overloaded with too many assignments, while there has been a general concern about the lack or loss of practical instruction, which has thus not entirely been covered in their e-learning experiences.

According to Chopra et al. [37], timely feedback and responses to students’ actions are key to effective online delivery. Another study also found a positive association between e-service and information quality with students’ satisfaction [43]. Based on interviews with teachers and students from Jordan, Almaiah et al. [44] found that it is crucial to analyse students and teachers’ use and adoption of systems, while their critical challenges included: (1) change management, students’ and teachers’ resistance, since many prefer traditional learning; (2) ICT literacy; (3) students’ self-efficacy and motivation; and (4) technical issues around systems’ accessibility, availability, usability, reliability, personalization, and service quality, mainly because perceived ease of use might benefit students’ performance and their efficacy while using e-learning systems. Perceived ease of use influences both system adoption and perceived usefulness and was clearly an important aspect since many participants complained that the e-learning system implemented was neither easy to use nor flexible, and this affected their experience regarding technical issues.

An Indian study reports a decline in teacher–student interaction when teaching moved across to online platforms [22]. Hence, greater autonomy is required from students, along with self-regulation and skills to learn online for effective learning [45].

Yet, students’ expertise in computer use and different learning platforms deeply influences their participation in e-learning [34]. Similarly, Wu et al. [35] emphasize the lack of adequate computer skills as an important impediment to effective online delivery. It is important to note that not only the lack of soft skills but also not having adequate hardware can obstruct e-learning. The Hungarian Rectors’ Conference [46], on the basis of 42 Hungarian HEIs’ responses, reported that the experiences with e-learning were generally positive. Still, the main issues involved the lack of technical preparation and equipment; in particular, many students did not have adequate equipment or Internet access. The levels of the students’ satisfaction with the e-learning was also reported to be better among students in developed countries than their counterparts in developing ones [26]. Similarly, resource-scarce settings struggle with the unavailability of digital platforms for education, limited Internet access, poor Internet speed, high cost of Internet and inadequate expertise to work via digital platforms [14]. The infrastructure resources in developing countries are incomparable to developed ones because there is a lack of technological infrastructure for e-learning like computers, connectivity and electricity on top of deficient skills and the active participation of both students and teachers due to insufficient ICT literacy [47].

To strengthen e-learning, the following strategies have been suggested as useful:

Several comprehensive models have also been developed for studying e-learning performance. The technology acceptance model (TAM) provides an easy way to assess the effects of two beliefs–perceived usefulness and perceived ease of use–on users’ intention to utilize a certain technology, hence providing a good prediction of students’ participation and involvement in e-learning, which in turn influences their performance [52].

Rizun and Strzelecki [53] employed an extension of the TAM, which suggests that acceptance of e-learning is related to enjoyment and self-efficacy. According to DeLone and McLean [54], system usage–the degree to which an individual uses the capabilities of a given information system in terms of frequency, nature and duration of use–has a direct connection with users’ satisfaction and their online performance. By applying DeLone and McLean’s Model (D&M model) of Information Systems Success, Aldholay et al. [55] were able to prove that system, service and information quality related to e-learning have significant positive effects on system usage, that thereby predicts a user’s satisfaction and has a positive impact on their performance.

Recently, Al-Fraihat et al. [41] used a multidimensional and comprehensive model and found seven types of quality factors that influence the success of e-learning systems, namely: technical system quality, information quality, service quality, education system quality, support system quality, learner quality, and instructor quality as antecedents of perceived satisfaction, perceived usefulness, use and benefits of e-learning. Moreover, Baber [56] relates students’ perception of their learning outcomes and their satisfaction to factors like students’ motivation, course structure, the instructor’s knowledge and facilitation.

Cidral et al. [34] proposed 11 different constructs of effective e-learning, among which we can mention individual skills, system requirements, and interaction-focused elements. System use and user satisfaction were shown to exert the greatest positive impact on individuals’ performance through e-learning. In a similar study, Hassanzadeh et al. [57] identified the following factors as responsible for success with e-learning: use of the system, loyalty to the system, benefits of using the system, intention to use, technical system quality, service quality, user satisfaction, goal achievement, and content and information quality.

Rashid and Yadav [58] draw attention to several critical issues that may affect the effectiveness of e-learning: students’ possibility to have access to and to afford e-learning technologies; the need for educators to be properly trained in the use of the technologies; teachers’ autonomy and trust; and the quality of the communication among higher education stakeholders. Moreover, Deloitte [59] highlights the importance of institutional support in the successful delivery of e-learning.

This study proposes a conceptual model for analysing students’ perceived academic performance during the period of the COVID-19 pandemic, which forced the transition from on-site to on-line teaching and learning. In this research, we combine the theoretical results of previous studies on e-learning with the emergency changeover to various online modes of delivery in response to the pandemic lockdown. The proposed conceptual model builds on the model of students’ satisfaction with e-learning suggested by Sun et al. [1] as well as the D&M model [60], which was used to describe different information systems’ success, including the e-learning system [41]. Cidral et al. [34] studied similar key apects of quality e-learning systems.

In the conceptual model we propose second-order multidimensional construct E-learning Quality of five components. Based on the literature [1, 34, 37] the construct connects three aspects of quality: learner, teacher and system.

Two factors associated with students’ satisfaction corresponding to the learner dimension are included in our proposed model: Home Infrastructure and Computer Skills. The rapid transition to online study meant students were relocated to a home environment where many did not enjoy suitable conditions to study, both a quiet place and digital equipment with access to (high-performance) Internet, which is indispensable for effective online study. Therefore, the latent variable Home Infrastructure covers the ICT conditions at home, i.e. having one’s own computer or access to one, the required software, a webcam, and a stable (and fast) Internet connection [37]. The greater the students’ previous knowledge and experience in using digital media, the easier the transition to e-learning has been. Computer Skills describe students’ expertise in using computers and different learning platforms, which is particularly important for active participation in the online delivery mode [34, 35].

The teacher dimension refers to the organization of teaching in a new e-learning environment. Studies show the organization and delivery of study material is important for student satisfaction and performance. Three constructs related to teachers are defined in the model. Mode of Delivery corresponds to the different forms used in online lectures, tutorials or practical classes providing learning materials and assignments, such as videoconference, audio recording, forum or e-mail [57]. Teachers play a valuable active role in the online environment by guiding students through the learning contents and providing them with timely responses and information. Equally important are prepared assignments that encourage and motivate students to independently learn at home. Online Instruction focuses on teachers’ active role and attitude to online teaching. The construct is explained by Information Quality and two other aspects assessed in our questionnaire, namely preparing regular assignments and being open to students’ suggestions [34, 41, 61]. Information Quality measures teachers’ responsiveness to the students, such as timely feedback or answering questions in an e-learning environment [34, 37]. We also propose a second-order construct System Quality, composed of learner and teacher dimensions: Home Infrastructure and Mode of Delivery.

Previous studies reveal that IT service support has a positive influence on users’ perceptions of their satisfaction with the system. As the transition to online study happened quickly and without prior training, the support of both the IT and the administrative service is vital for ensuring that students are satisfied with their new learning environment [34, 37, 41, 57, 61, 62]. In our model, Service Quality refers to the aspect of administrative, technical and learning assistance. To compensate for the lack of social contact while studying from home, various forms of online interactions are possible. Teacher–student or student–student interactions were shown to be important factors of satisfaction with the e-learning system [34, 41, 61]. The construct Online Interactions describes how often a student communicates with colleagues from the course, the teachers or the administrative staff.

To summarize, E-learning Quality is multidimensional construct of five components Students’ Computer Skills, System Quality, which reflects the Mode of Delivery and Home Infrastructure,. Online Instruction assessed through Information Quality, Online Service Quality and Online Interactions with colleagues, teachers and staff. We hypothesize:

During the COVID-19 pandemic, teaching and learning were completely implemented in the online environment and thus we include the quality dimension, which measures several important aspects of the e-learning system: system quality, information quality, service quality, learner digital quality and interaction quality. Models measuring the success of the information system (also the e-learning system) are usually based on the D&M model, where user satisfaction and quality dimension play an important role [34, 41, 57, 61]. The construct Perceived Student Satisfaction is manifested by students’ satisfaction with the organization of e-learning (i.e. lectures, tutorials, seminars, mentorships) and with the support of the teachers and the student counselling service [34, 57]. Perceived Student Performance aims to capture students’ benefits of using an e-learning system. It measures students’ opinion of their performance and whether it has worsened with the transition to the online learning mode [34, 41, 57]. The proposed model’s structural part includes three constructs: E-learning Quality, Perceived Student Satisfaction and Perceived Student Performance. We may reasonably assume the quality of the e-learning system has a positive effect on satisfaction with the online education environment system, leading to the system’s greater use and thus to improve the student performance. It is unlikely that one can perform well without use of the system.

This leads to three hypotheses being proposed:

Therefore, we propose the conceptual model presented in Fig 1. and construct description in Table 1.

https://doi.org/10.1371/journal.pone.0258807.g001

https://doi.org/10.1371/journal.pone.0258807.t001

The data for this study come from a very comprehensive and large-scale global student survey entitled “Impacts of the COVID-19 Pandemic on Life of Higher Education Students”, aimed at examining how students perceive the impacts of the pandemic’s first wave in early 2020 on various aspects of their lives on a global level [8]. This project was originally promoted by the Faculty of Public Administration, University of Ljubljana (Slovenia), which, thanks to the support of international partners, was able to be disseminated worldwide. The online questionnaire was adapted and extended from the European Students’ Union [63] survey. It was formed by 39 questions, mainly including closed-ended questions (see S1 Questionnaire). It focused on socio-demographic, geographic and other aspects pertaining to the life of university students, such as academic online work and life, social life, emotional life, personal circumstances, changes in habits, the roles and measures of institutions, as well as personal reflections on COVID-19 [64]. Initially, the online questionnaire was designed in English and later translated into six different languages (Italian, North Macedonian, Portuguese, Romanian, Spanish, Turkish). The translation of the questionnaire was carried out by native speakers, being proficient in English. The web-based survey was launched via the open-source web application 1KA (One Click Survey; www.1ka.si) on 5 May 2020 and remained open until 15 June 2020, that is, in a period when most nations were experiencing the onerous restrictions imposed by the lockdown. Participation in the study reached global proportions by exceeding the milestone of 30,000 responses submitted by students from more than 130 countries on all six continents. The entire dataset was first analysed by Aristovnik et al. [8].

The survey was intended for all higher education students at least 18 years of age, representing the target population of this study. The sampling technique used is non-probabilistic, specifically convenience sampling through university communication systems around the world and social media. The students were informed about the details of the study and gave their informed consent before participating. Due to this study’s specific focus on academic online work and life, it only includes student data with respect to selected parts of the questionnaire. However, since the respondents were not obliged to complete the questionnaire in full, the number of respondents varied across questions. Accordingly, a complete-case-analysis approach was applied to mitigate missing data issues [65]. With the assumption of “missing completely at random”, meaning the complete cases are a random sample of the originally identified set of cases, a complete-case approach is the most common method for handling missing data in many research fields, including educational and epidemiologic research [66, 67]. In order to assure a more robust analysis and perform reliable comparisons on the national level, this study focuses on the 10 countries (Chile, Ecuador, India, Italy, Mexico, Poland, Portugal, Romania, Slovenia, Turkey) that provided at least 500 answers with regard to different aspects of students’ academic life.

The final dataset consisted of 10,092 participants or students enrolled in HEIs, of whom 92% were attending a full-time study course. They were at least 18 years old, with a median age of 23 years (IQR [21.0, 24.0]), and about two-thirds of them (67%) being female. Most respondents (82%) were pursuing a bachelor’s degree, 16% a master’s degree, and 2% a doctoral course. Twelve percent were majoring in a study course in the Arts and Humanities, 37% in the Social Sciences, 32% in the Applied Sciences and 19% in the Natural and Life Sciences. Detailed information on the sample, i.e. the number of respondents and participants’ sociodemographic characteristics by country, are given in Table 1.

This study primarily focuses on how COVID-19 has affected different aspects of students’ academic life. Specifically, students reported their experiences with the organization of teaching and administrative services, along with their satisfaction, expectations and perceived impacts on their university career. This involves a total of 34 survey items, representing a basis for measuring the 9 latent constructs used in our proposed conceptual model. Individual satisfaction and concern levels were measured on a 5-point Likert scale, from 1 (lowest value) to 5 (highest value) [68]. A more detailed description, including the set of measuring items and their characteristics, is found in Table 2.

https://doi.org/10.1371/journal.pone.0258807.t002

All participants were informed about the details of the study and gave their informed consent before participating. By clicking on a button ‘next page’ participants agreed to participate in the survey. Study participation was anonymous and voluntary, and students could withdraw from the study without any consequences. For data-protection reasons, the online survey was open to people aged 18 or over and enrolled in a higher education institution. The procedures of this study comply with the provisions of the Declaration of Helsinki regarding research on human participants. Ethical Committees of several of the higher education institutions involved approved this study, such as the University of Verona (protocol number: 152951), ISPA–Instituto Universitário (Ethical Clearance Number: I/035/05/2020), University of Arkansas (IRB protocol number: 2005267431), Walter Sisulu University (Ethical Clearance Number: REC/ST01/2020) and Fiji National University (CHREC ID: 252.20).

We implemented the SEM with use of the lavaan package (v.0.6.4, [69]) in the R statistical environment (v.4.0.2, [70]). A two-step approach was followed. In the first step, we checked the fit of the measurement model to all the latent variables; in the second step, we checked the fit of the structural model. The Comparative Fit Index (CFI), Tucker-Lewis Index (TLI), Root Mean Square Error of Approximation (RMSEA) and Square Root Mean Residual (SRMR) were used as goodness of fit indices. The fit was deemed appropriate for CFI and TLI above .90, and for RMSEA and SRMR values below .06 and .08, respectively (e.g. [71, 72]).

We assessed the reliability of the first-order and second-order factors with McDonald’s omega (ω) and ωL2, respectively, and convergent validity with Average Variance Extracted (AVE) using the semTools package (v.0.5.3, [73]). Omega and AVE values above .70 and .50 were indicative of good reliability and convergent validity, respectively [72, 74, 75].

Invariance analysis was performed [72] by comparing the difference in the fit of a series of sequentially constrained models from configural (Conf), intercepts (Intercpy), loadings (Load), means (Means), to regression coefficients (Regr). Invariance was assumed for nonsignificant Δχ2 or, preferentially, ΔCFI