Hostname: page-component-77c89778f8-gvh9x Total loading time: 0 Render date: 2024-07-17T23:52:54.933Z Has data issue: false hasContentIssue false

Does checking-in help? Understanding L2 learners’ autonomous check-in behavior in an English-language MOOC through learning analytics

Published online by Cambridge University Press:  09 May 2024

Yining Zhang
Affiliation:
Tsinghua University, China (yiningzhang@tsinghua.edu.cn)
Fang Yang
Affiliation:
Tsinghua University, China (yangfang@tsinghua.edu.cn)
Haolin Yang
Affiliation:
Tsinghua University, China (hl-yang20@mails.tsinghua.edu.cn)
Shuyuan Han
Affiliation:
Tsinghua University, China (han-sy20@mails.tsinghua.edu.cn)
Rights & Permissions [Opens in a new window]

Abstract

Concerns over the quality of teaching in massive open online courses devoted to language learning (LMOOCs) have prompted extensive research on learning behavior in such courses. The purpose of this study is to gain a better understanding of autonomous learning check-ins – that is, individuals sharing their own learning records and/or other information about their learning-related experience – a novel behavior that has not been studied in previous LMOOC research. Using learning analytics, we found that just 6.2% (n = 699) of a sample of 11,293 learners autonomously engaged in check-in behavior, and that the content of these learners’ check-ins varied considerably according to their contexts and the language skills they were seeking to acquire. We further found (1) a positive association between check-in behavior and LMOOC completion; (2) that students who chose to check in earned relatively low grades on unit quizzes, especially in their early stage of learning, but outperformed the non-check-in group significantly in final exam scores; and (3) that those who checked in engaged with a significantly wider array of in-LMOOC learning components than those who did not, and thus accessed a wider system of language-learning experiences. Taken together, these results confirm that check-in behavior can aid the process of learning in an LMOOC and further highlight this behavior’s wider potential to aid self-directed autonomous online learning.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2024. Published by Cambridge University Press on behalf of EUROCALL, the European Association for Computer-Assisted Language Learning

1. Introduction

Language massive open online courses (LMOOCs) are web-based online courses that offer unlimited access to language learning for a limited period primarily by higher education institutions catering to individuals seeking to learn a foreign language (Bárcena & Martín-Monje, Reference Bárcena, Martín-Monje, Martín-Monje and Bárcena2014; Gimeno-Sanz, Reference Gimeno-Sanz, Beaven and Rosell-Aguilar2021). LMOOCs have been growing steadily in popularity since their inception in 2012 (Sallam, Martín-Monje & Li, Reference Sallam, Martín-Monje and Li2022; Xue & Dunham, Reference Xue and Dunham2023; Zhang & Sun, Reference Zhang and Sun2023). However, concerns about their low completion rates (Duru, Sunar, White, Diri & Dogan, Reference Duru, Sunar, White, Diri and Dogan2019; Friðriksdóttir, Reference Friðriksdóttir2021; Wang, An & Wright, Reference Wang, An and Wright2018) and low levels of learning autonomy (Agonács, Matos, Bartalesi-Graf & O’Steen, Reference Agonács, Matos, Bartalesi-Graf and O’Steen2020) have called the quality of LMOOC teaching into question and spurred many researchers to study LMOOC learning behavior (e.g. Doğan, Sunar, Duru & White, Reference Doğan, Sunar, Duru and White2018; Martín-Monje, Castrillo & Mañana-Rodríguez, Reference Martín-Monje, Castrillo and Mañana-Rodríguez2018; Wang, Chen, Tai & Zhang, Reference Wang, Chen, Tai and Zhang2021; Zeng, Zhang, Gao, Xu & Zhang, Reference Zeng, Zhang, Gao, Xu and Zhang2022; Zhang & Sun, Reference Zhang and Sun2023). Mac Lochlainn, Nic Giolla Mhichíl and Beirne (Reference Mac Lochlainn, Nic Giolla Mhichíl and Beirne2021) recently called for LMOOC research to identify “which specific behaviours, both L2 oriented and not, are prevalent among LMOOC learners, to allow us to scrutinise and reflect on what types of specific, L2-directed patterns of participation might be established” (pp. 113–114).

However, research to date has focused on what might be called a “canon” of LMOOC learning behaviors that includes watching videos, reading class slides, taking quizzes, and participating in discussions in forums (e.g. Duru et al., Reference Duru, Sunar, White, Diri and Dogan2019; Martín-Monje et al., Reference Martín-Monje, Castrillo and Mañana-Rodríguez2018; Read & Bárcena, Reference Read and Bárcena2021), but broadly ignores learning check-ins. Learning check-ins denotes a relatively new type of learning behavior in which an LMOOC learner posts/shares part or all of their own learning record or information about their learning-related experience (Chen, Sun, Wu & Song, Reference Chen, Sun, Wu and Song2019). Although learning check-in behavior itself has become popular in many technology-assisted language-learning applications, and is a demonstrably effective aid to English learning (Nie et al., Reference Nie, Zheng, Zeng, Zhou, Lei and Wang2020), little is known about the effects of such behavior in LMOOC contexts.

Methodologically, when examining data produced by LMOOC learners, researchers frequently employ questionnaire-based surveys to assess learning behavior (e.g. Xu, Zhu & Chan, Reference Xu, Zhu and Chan2023). Compared to this indirect method, the use of learning analytics has greater potential to capture a more comprehensive picture of such behavior. Such an approach is perhaps especially well suited to analyzing MOOC data, insofar as MOOCs’ learning platforms naturally generate large datasets comprising authentic traces of online learning by their users (Beaven, Codreanu & Creuzé, Reference Beaven, Codreanu, Creuzé, Martín-Monje and Bárcena2014).

Learning check-ins have not to our knowledge been studied in previous LMOOC research. Our decision to adopt learning analytics to study this novel behavior was linked not only to our aim of painting a broad picture of it in a MOOC but also to our desire to test the relationships between such behavior, on the one hand, and, on the other, learning outcomes including both final exam grade and unit grades.

In addition, due to the exceptional diversity of LMOOC learners, some have argued that course-completion rates are not appropriate as the sole metric of LMOOC learning success, as long as learners achieve what they want to achieve (e.g. Jitpaisarnwattana, Darasawang & Reinders, Reference Jitpaisarnwattana, Darasawang and Reinders2022; Mac Lochlainn et al., Reference Mac Lochlainn, Nic Giolla Mhichíl and Beirne2021). Accordingly, the present study will include those who have engaged with the sampled LMOOC, regardless of their completion status. We expect that our results will help answer questions about the effectiveness of specific behaviors by LMOOC learners (Mac Lochlainn et al., Reference Mac Lochlainn, Nic Giolla Mhichíl and Beirne2021), thereby helping such learners make the most of their LMOOC experience (Appel & Pujolà, Reference Appel and Pujolà2021) and facilitating LMOOC educators’ provision of more innovative, engaging, and rewarding LMOOC learning to their students (Duru et al., Reference Duru, Sunar, White, Diri and Dogan2019; Friðriksdóttir, Reference Friðriksdóttir2021; Godwin-Jones, Reference Godwin-Jones2014; Hsu, Reference Hsu2023; Mac Lochlainn et al., Reference Mac Lochlainn, Nic Giolla Mhichíl and Beirne2021; Zhang & Sun, Reference Zhang and Sun2023).

2. Literature review

2.1 Learning behavior and autonomous learning in LMOOCs

Learning behaviors have always been a major topic of the LMOOC-development literature (Mac Lochlainn et al., Reference Mac Lochlainn, Nic Giolla Mhichíl and Beirne2021), and strong links have been found between particular behaviors and particular learning outcomes (e.g. Martín-Monje et al., Reference Martín-Monje, Castrillo and Mañana-Rodríguez2018). For example, Jitpaisarnwattana, Reinders and Darasawang (Reference Jitpaisarnwattana, Reinders and Darasawang2021) found that working in groups and making study plans were associated with completion rate in an LMOOC, while Zeng et al. (Reference Zeng, Zhang, Gao, Xu and Zhang2022) reported that LMOOC learners’ attention to the learning unit distinguished high performers from low ones. At the heart of learning behaviors in MOOC contexts is learner autonomy – that is, learners taking an active role in controlling and making choices and decisions regarding their own learning process, such as setting learning goals, deciding how, when, and where to engage with learning materials or support from teachers or peers as needed, etc. (Benson, Reference Benson2011; Huang & Benson, Reference Huang and Benson2013; Little, Reference Little1991). The massiveness and openness of MOOCs afford learners greater autonomy in directing their own learning path and deciding how to engage with a MOOC primarily at their own will, as indicated by diverse learning behaviors documented in the literature (e.g. Ding & Shen, Reference Ding and Shen2022; Hood, Littlejohn & Milligan, Reference Hood, Littlejohn and Milligan2015).

According to the relevant literature, when learning autonomously, a person remains attentive, motivated, and engaged by course-related materials or activities (Chacón-Beltrán, Reference Chacón-Beltrán2018; Ding & Shen, Reference Ding and Shen2022; Mac Lochlainn et al., Reference Mac Lochlainn, Nic Giolla Mhichíl and Beirne2021; Xu et al., Reference Xu, Zhu and Chan2023). For example, Read and Bárcena (Reference Read and Bárcena2021) showed how highly autonomous learners in a Spanish LMOOC studied diligently, submitted activities with very high rates of correctness, viewed almost all course videos, and remained active in discussion forums. In addition, learners showed their own wills in controlling and making decisions regarding their own learning process. For example, the students sampled by Agonács and Matos (Reference Agonács and Matos2019) actively sought opportunities to use the Italian language outside of their existing LMOOC context, and showed autonomy in engaging with different learning behaviors such as watching Italian movies or reading Italian newspapers and magazines. Taken together, these findings imply that design features and pedagogical practices aimed at encouraging LMOOC learning autonomy (i.e. learners’ control of their own learning) would tend to improve LMOOC learners’ outcomes.

2.2 The potential of check-in behavior to enhance language learning

Check-in behavior first came to prominence when certain mobile-phone applications allowed their users to share location-based information, mainly for social-networking purposes (Kim, Reference Kim2016). Gradually, people came to share not only location-based information but also sports records, learning records, pictures, and other types of personal news, motivated by desires including self-expression, information sharing, providing and seeking social support, and enjoyment (Luarn, Yang & Chiu, Reference Luarn, Yang and Chiu2015). Eventually, check-ins were formally adopted by some English-language mobile-learning applications to enable their users to share their English-learning progress. Such apps generate screenshots of each user’s learning record (e.g. learning content, learning duration, or learning achievement) at regular intervals, at which point the learner in question can decide to click the share button to make the learning record appear on either the learning app or their other social networking sites (Nie et al., Reference Nie, Zheng, Zeng, Zhou, Lei and Wang2020). Additionally, some users choose to check in by sharing their records to their social networks without any such prompting.

To the best of our knowledge, however, only one prior study – by Nie et al. (Reference Nie, Zheng, Zeng, Zhou, Lei and Wang2020) – has explored the role of check-in behavior in language learning. It concluded that learners with positive attitudes toward check-in behavior and high levels of perceived behavioral control were more likely than others to check in with their English-learning records, suggesting a positive relation between autonomous learning and check-in behavior.

Though no prior study appears to have explored the role of check-in behavior in LMOOCs explicitly, the learning activity described by Jitpaisarnwattana et al. (Reference Jitpaisarnwattana, Reinders and Darasawang2021) was relatively similar. The language learners in their study took a MOOC with the aim of developing their English presentation skills and were given recommendations by the learning system once they had uploaded their first video assignment. Then, they created individual learning plans (ILPs), taking account of specific system-recommended activities. We believe that the process of creating and discussing an ILP is fundamentally similar to check-in behavior, in that it involves learners’ (1) setting of personal learning goals based on their known language strengths and weaknesses; (2) autonomous cognitive engagement with learning resources that were not assigned; and (3) sharing evidence of having accomplished such extra work. However, the use of ILPs in Jitpaisarnwattana et al.’s study also differs from the check-in behavior in a typical LMOOC in terms of the formats used for sharing (i.e. uploading one’s final product vs. uploading one’s learning record), restrictions on language skills (i.e. presentation skills only vs. no skills restriction), and the format of the MOOC in question (i.e. cMOOC vs. MOOC in any other format, especially xMOOC). Among the format categories mentioned in the preceding sentence, cMOOCs emphasize strong community connection and extensive content contributions from the learners themselves, whereas xMOOCs focus on the transmission of large amounts of informational content and feature few chances for direct interaction between individuals (Fidalgo-Blanco, Sein-Echaluce & García-Peñalvo, Reference Fidalgo-Blanco, Sein-Echaluce and García-Peñalvo2016). This warrants further study of the effects of check-in behavior, defined in the present study’s LMOOC context as students’ spontaneous uploading and sharing of their own learning records.

2.3 The use of learning analytics in LMOOC studies

Aided by advancements in information technology, today’s learning management systems provide extensive new opportunities for researchers to discern patterns of online learning (Gašević et al., Reference Gašević, Dawson and Siemens2015). By enabling accurate interpretation of such data, learning analytics help to identify predictors of learning success and is increasingly used for decision-making about both instructional design and resource allocation (Gelan et al., Reference Gelan, Fastré, Verjans, Martin, Janssenswillen, Creemers, Lieben, Depaire and Thomas2018; Leitner, Khalil & Ebner, Reference Leitner, Khalil, Ebner and Peña-Ayala2017). In LMOOC research specifically, there is a similar interest in taking advantage of learning platforms’ rich stores of data to analyze learning patterns, behaviors, and outcomes. However, the number of studies that have used learning analytics to study effective learning within LMOOCs remains quite limited.

Within that small body of work, three key approaches can be identified. The first is the adoption of descriptive learning-analytics metrics to explore various forms of learning behaviors (e.g. Martín-Monje et al., Reference Martín-Monje, Castrillo and Mañana-Rodríguez2018; Read & Bárcena, Reference Read and Bárcena2021; Zeng et al., Reference Zeng, Zhang, Gao, Xu and Zhang2022). This answers the question of “what is happening in distance language learning,” via “the provision of more accurate quantitative analysis of online language learning behavior” (Gelan et al., Reference Gelan, Fastré, Verjans, Martin, Janssenswillen, Creemers, Lieben, Depaire and Thomas2018: 313). The second key approach is to use learning-platform data to examine the effects of certain behaviors on learning outcomes. For example, Duru et al. (Reference Duru, Sunar, White, Diri and Dogan2019) and Jitpaisarnwattana et al. (Reference Jitpaisarnwattana, Reinders and Darasawang2021) used log data to build statistical models of the relationship between course completion and various types of learning behaviors, including completion of particular learning activities, contribution to discussions, and student–student interactions, among others. The third approach consists of identifying variance in the learning-behavior patterns of different groups of students (e.g. Duru et al., Reference Duru, Sunar, White, Diri and Dogan2019; Zeng et al., Reference Zeng, Zhang, Gao, Xu and Zhang2022). For example, Martín-Monje et al. (Reference Martín-Monje, Castrillo and Mañana-Rodríguez2018) found statistically significant differences in the incidence of all log behaviors between successful and non-successful students.

To gain a better understanding of learners’ check-in behavior, the present study used learning analytics to look into all three of the research topics mentioned above. Our research questions were as follows:

  • RQ1: How and how often did learners autonomously check in about their learning progress while taking an intermediate-level English LMOOC?

  • RQ2: Did those learners who engaged in language-learning check-in behavior have a significantly higher course-completion rate than those who did not? And if so, to what extent can such behavior be said to encourage course completion?

  • RQ3: Among those learners who successfully completed the course, were there significant differences between the check-in and no-check-in groups in (a) learning outcomes (i.e. final exam grade and eight quiz grades) and/or (b) language-learning behaviors (i.e. total number of clicks on slides and announcements; total and per-unit times spent watching assigned videos)?

  • RQ4: Among all learners who engaged with the course in any discernible way, were there significant differences between the check-in and no-check-in groups in (a) learning outcomes (i.e. final exam grade and eight quiz grades) and/or (b) language-learning behaviors (i.e. the total number of clicks on slides and announcements; total and per-unit times spent watching assigned videos)?

3. Methodology

3.1 Research context

This study’s focal LMOOC was University English for Academic Purposes, as provided on XuetangX over a six-month period in 2019. Aimed at helping university students to master academic skills and join in academic conversations, it encouraged them to read, write and share their ideas while studying. The course was designed following an xMOOC format, focused on transmission of knowledge through embedded video lectures, computer-marked assignments, supplementary materials, and a discussion forum, among other features. As of May 25, 2023, various iterations of this LMOOC had provided learning services to 283,446 people worldwide. At the time of our data collection, it comprised eight learning units, each organized around a different research topic: cultural studies, life science, art, ecology, social issues, engineering and technology, psychology, and economics and management. Within each unit, the learning content was divided into 10 parts, including an introduction, listening, group discussion, topic and text, vocabulary, academic writing, a “skills power station,” student research projects, summary, and discussion.

As well as engaging in the above series of learning activities, there were discussion boards where learners could record and share their views with peers. They were also expected to complete relevant exercises and quizzes. After learning all eight units, students had the option of taking a two-hour online final exam that covered the language knowledge taught in the course. (This was required of those who wished to earn a course certificate.) The final exam score and the scores from the eight units were collected and used as indicators of learning outcomes in our further analyses.

3.2 Check-in behavior

At the start of the course, the teacher posted announcements encouraging the students to post about their learning records and about their learning progress in the discussion forum, regardless of whether the learning in question had been undertaken in or out of the LMOOC. Specifically, the teacher said, “This is totally voluntary work, and it depends on your time and learning interest. You are welcome to show your learning progress or learning record, either in this MOOC course or outside this MOOC course, in the discussion forum. When doing this, you might first want to think carefully about your own strengths and weaknesses in English and make your own check-in plan accordingly. The purpose of doing these check-ins is that I want to encourage you to learn English, and I also want to create a good learning environment for this course. This is not graded work, but I still hope to see your participation.” The teacher also set some rules for learning check-ins: students could either upload screenshots they felt were representative of their learning progress or write textual reports on how much and/or what they had learned.

3.3 Data mining

The initial learning-analytics data that we obtained from the learning platform took the form of a six-sheet Excel workbook that documented the respective logs of all 11,293 students. Each of its spreadsheets represented a separate data theme. We used Python software for data-mining processing (van Rossum & Drake, Reference van Rossum and Drake2009), and created a new dataset containing all 809,049 of the enrolled learners’ learning behaviors, learning outcomes, and check-in statuses.

Based on this first dataset, we created a second one relating specifically to those students who checked in. This was accomplished in the first instance via manual screening of all 2,224 posts in the discussion forum and extracting those that could be identified as check-ins (coded as 1; non-check-ins, as 0). The results were then triangulated by the researchers.

3.4 Measurement

3.4.1 Learning outcomes

A total of nine variables relating to the final exam and the eight unit-specific quizzes were used as learning-outcome indicators. In each unit, there were nine quiz questions aimed at assessing students’ understanding of the content covered in that unit. The maximum total quiz score for any one unit was therefore nine. The maximum total score for the final exam was 100. All exam questions were multiple choice.

3.4.2 Learning completion

Following Zeng et al. (Reference Zeng, Zhang, Gao, Xu and Zhang2022), we labeled learners as having completed the LMOOC if their final exam score was equal to or greater than 60 out of 100. Our data showed that among the LMOOC’s 11,293 learners, 1,046 completed it.

3.4.3 Learning behavior

A total of 11 variables were used as indicators of learning behaviors. They were grouped into three types: (1) total number of clicks on course slides, (2) total number of views of course announcements, and (3) overall and per-unit times spent watching course videos. The total length of the video provided by the course was 8 hours 48 minutes and 33 seconds (i.e. 31,713 seconds).

3.5 Data analysis

We used content analysis and descriptive statistics to answer our first research question. After extracting the posts that pertained to check-in messages, one of the authors categorized them based on whether their check-in content focused on in-LMOOC or out-of-LMOOC learning. In addition, within the out-of-LMOOC learning category, we categorized posts based on the language skills they mentioned: listening, reading, etc. The results were then triangulated by the other authors.

To answer our second research question, a chi-square test of independence was used to analyze inter-group differences in LMOOC completion and logistic regression applied to establish the extent to which check-in behavior encouraged learners to complete the LMOOC.

To answer our third research question, we first checked the distribution of the data. Specifically, we used Shapiro–Wilk tests of normality (Shapiro & Wilk, Reference Shapiro and Wilk1965) to check the distribution of the two independent groups’ final exam and unit scores, and the results indicated that all the tested variables significantly deviated from normal distribution (all p < .001). Therefore, a non-parametric technique, the Mann–Whitney U test (Mann & Whitney, Reference Mann and Whitney1947), was used in our subsequent analyses relating to this question.

To answer our final research question, we first distinguished between a group of learners who had engaged with the LMOOC in any discernible way and a group that had not obviously done so. Specifically, this was done by checking all students’ knowable learning behaviors (i.e. video-watching during each unit, slide accessing, announcement viewing, quiz-taking, and exam-taking) and classified those individuals who had not demonstrated any of them as not having engaged in the LMOOC at all. Among the 11,293 enrolled learners, 2,314 engaged. Within those 2,314, we further examined differences in learning behaviors across the check-in and non-check-in groups. We again conducted Shapiro–Wilk tests of normality and, as the results indicated that all tested variables significantly deviated from normal distribution (all p < .001), we then conducted Mann–Whitney U testing.

4. Results

4.1 An overview of learners’ check-in behavior

Of the 11,293 learners in our sample, 699 exhibited language-learning check-in behavior on the platform, representing a per capita check-in rate of 6.2%. Those learners who checked in did so a total of 1,633 times (M = 2.34). In terms of frequency, 663 of the 699 learners checked in 10 times or fewer; 21 of them did so 11–20 times; eight, 21–30 times; three, 31–40 times; one, 41–50 times; two, 51–60 times; and one, more than 60 times.

Figure 1 illustrates the content of these students’ check-in behaviors. Among the 1,633 check-in posts, the most frequently seen content type was self-reported LMOOC learning progress (n = 699, 42.8%). When posting check-in messages in this category, the students either posted a screenshot showing their current progress, generated by the learning platform, or simply reported their progress textually, such as by saying, “[Name] has completed Unit 1.”

Figure 1. Learners’ check-in behavior by content type.

The second most common type of check-in content was vocabulary learning (n = 357, 21.9%). In this category, learners reported new words that they had learnt either in or out of the LMOOC, either by providing a screenshot from their vocabulary-learning software or by typing the new words in question into the forum.

This was followed by listening (n = 296, 18.1%). In this category, we found posts highlighting the completion of listening materials or the length of time that learners had spent engaged in listening practices.

The fourth category was reading (n = 276, 16.9%), which included posts about which materials the students had read and summaries of such materials. A typical example was, “Today’s reading: Tuesday with Morrie, p158-p180”.

The fifth and final category, “other” (n = 5, 0.3%), included all material that did not fit into any of the previous four categories – for example, posts noting that a student had learned English but without further elaboration.

4.2 Differences in and prediction of completion rate

Table 1 shows frequency counts for students’ completion and check-in statuses. The completion rate was 9.3% for all 11,293 learners, 93.8% for the check-in group, and 3.7% for the non-check-in group (χ2 = 6,333, df = 1, p < .001).

Table 1. Frequency counts by completion status and check-in status

As Table 2 shows, check-in behavior was a significant predictor of course completion (p < .001). Specifically, the odds of completing the course increased by a factor of more than 43 if the learners checked in on their progress. When other independent variables were held at their means, the predicted probability of completing the course was .71 for learners from the check-in group and .05 for others. In addition, the length of viewing slides, announcements and certain units (i.e. Units 5 and 8) also increased the odds of completing the course significantly, at p < .001. On the other hand, as the length of time spent viewing Unit 6 slides increased, the odds of completing the course decreased significantly, at p < .05. The model fit of this analysis was calculated as χ2 = 5,306, df = 11 (p < .001), indicating that the model with predictors fit the data significantly better than the one with just an intercept.

Table 2. Logistic regression, learning behaviors on course completion

4.3 Differences in learning outcomes and behaviors among those who successfully completed the course

Table 3 presents the means of the learning outcomes of the check-in and non-check-in subgroups of those students who successfully completed the course. Of the two, the check-in subgroup’s mean scores were lower on all eight quizzes, but its mean final exam grade was higher (M = 97.69 vs. M = 94.98).

Table 3. Learning outcomes of the check-in and non-check-in subgroups of students who completed the LMOOC

Mann–Whitney U tests were then applied to establish whether learning outcomes differed significantly between those who checked in and those who did not (Table 3). When we checked their quiz scores on a unit-by-unit basis, we found that the check-in subgroup of LMOOC completers scored significantly lower than its non-check-in counterpart on three units – that is, Unit 2 (p < .05), Unit 3 (p < .01), and Unit 7 (p < .05). In the final exam score, the check-in group outperformed the non-check-in group significantly (p < .001).

Turning now to learning behaviors, Table 4 first presents the mean incidence of the 11 captured behaviors across the same two subgroups. Not surprisingly, the means of the incidence of all learning behaviors by those LMOOC completers who checked in on their learning progress were higher than among those who did not check in.

Table 4. Learning behaviors of the check-in and non-check-in subgroups of students who completed the LMOOC

In terms of learning-behavior differences, the check-in subgroup of LMOOC completers watched videos for significantly longer (p < .001) and viewed significantly more class slides (p < .001) than the non-check-in subgroup did. However, we found no significant difference in those two subgroups’ total views of course announcements. Across all eight units, the check-in subgroup of completers watched significantly more videos than its non-check-in counterpart (Table 4).

4.4 Difference in learning outcomes and behaviors among those who engaged in the course

Of the 20.5% of students (n = 2,314) who were demonstrably engaged in the LMOOC, irrespective of their completion status, 29.4% (n = 680) exhibited check-in behavior and the remaining 70.6% (n = 1,634) did not. As compared to its non-check-in counterpart, the check-in subgroup of engaged students earned lower mean quiz scores on the first three units but higher ones starting from Unit 4 (with the exception of Unit 7). The mean final exam score for the check-in subgroup of engaged students (M = 94.59) was also markedly higher than that of the non-check-in subgroup (M = 24.60) (Table 5).

Table 5. Learning outcomes of the check-in and non-check-in subgroups of students who engaged with the LMOOC

When we examined inter-subgroup quiz-score differences by unit, we found that the check-in subgroup of engaged students scored significantly lower on Unit 1 (p < .001). However, the same subgroup began to earn significantly higher scores than its counterpart starting from Unit 4 – that is, Unit 4 (p < .05), Unit 5 (p < .001), Unit 6 (p < .05), and Unit 8 (p < .001). In terms of final exam score, the check-in group outperformed the non-check-in group significantly (p < .001) (Table 5).

We then checked the means of the same two subgroups’ learning behaviors. Those for all learning behaviors exhibited by the check-in subgroup were higher than the parallel means for the non-check-in subgroup. Mann–Whitney U tests confirmed that the check-in subgroup accessed more class slides (p < .001) and class announcements (p < .001), and watched more video footage, both in total (p < .001) and for all eight units (p < .001) (Table 6).

Table 6. Learning behaviors of the check-in and non-check-in subgroups of students who engaged with the LMOOC

5. Discussion

This study aimed to develop a better understanding of learning check-in behavior. Its results suggest that the use of check-ins greatly benefited LMOOC learning, probably by enabling learners to practice autonomous self-directed learning behavior (Agonács & Matos, Reference Agonács and Matos2019; Agonács, Matos, Bartalesi-Graf & O’Steen, Reference Agonács, Matos, Bartalesi-Graf and O’Steen2019; Agonács et al., Reference Agonács, Matos, Bartalesi-Graf and O’Steen2020; Appel & Pujolà, Reference Appel and Pujolà2021; Ding & Shen, Reference Ding and Shen2022; Fuchs, Reference Fuchs2020; Read & Bárcena, Reference Read and Bárcena2021; Wang et al., Reference Wang, An and Wright2018).

Our first key finding, regarding overall patterns of check-in behavior, was that just 6.2% of the 11,293 sampled LMOOC learners chose to check in even once. The content of their check-in behavior was not prescribed by the instructor, and this may have driven the remarkable diversity of such content, with around 43% of it relating to learning progress on course topics, and the rest to students’ out-of-class learning progress in reading, vocabulary and listening. It seems likely that learners’ check-in behavior both reflects and facilitates autonomous online language learning, as it allows learners to control their own learning and decide how to engage with a MOOC at their own wills (Agonács & Matos, Reference Agonács and Matos2019; Agonács et al., Reference Agonács, Matos, Bartalesi-Graf and O’Steen2019; Chacón-Beltrán, Reference Chacón-Beltrán2018; Ding & Shen, Reference Ding and Shen2022; Fuchs, Reference Fuchs2020; Mac Lochlainn et al., Reference Mac Lochlainn, Nic Giolla Mhichíl and Beirne2021; Xu et al., Reference Xu, Zhu and Chan2023). Based on previous studies of the nature of autonomous learning in LMOOCs, such learning is “non-linear [… and] informal, being unconstricted by the formal educational context” (Agonács et al., Reference Agonács, Matos, Bartalesi-Graf and O’Steen2020: 1164). The check-in behavior we observed indeed fit this description, as evidenced by both its spontaneity and the great variety of its content, with some learners choosing to report the progress of their current LMOOC learning in terms of the number of units, videos, quizzes, etc., that they had completed up to that point, and others casting their eyes beyond the course’s content to report on their learning progress in off-class reading, vocabulary and listening, often using authentic novels, movies, and newspaper articles. It is also important to bear in mind that learners were free to adjust their check-in behavior at any time during the course. Perhaps more importantly, however, the check-in behavior observed in this study confirms that LMOOC students have the potential to grow beyond reactive autonomy, in which they regulate the direction of their learning after such a direction is set by others, and achieve proactive autonomy – that is, regulate the direction of learning for themselves (Ding & Shen, Reference Ding and Shen2022). None of the check-in behaviors were set, required, regulated or even monitored by the instructor; rather, while checking in, the sampled students took full responsibility for regulating not only the direction but also the whole process of their learning.

The relationship between check-in behavior and course completion was found to be very close indeed: the predicted probability of completing the course was 0.71 for learners for the check-in group, as compared to just .05 for the non-check-in group. Previous research has (1) shown that a better understanding of learner engagement can help educators minimize dropout rates (Duru et al., Reference Duru, Sunar, White, Diri and Dogan2019); (2) identified lack of motivation as one of the chief obstacles to learners’ completion of LMOOCs (Friðriksdóttir, Reference Friðriksdóttir2021); and (3) confirmed that engagement – with course materials, tasks, and assessments – significantly predicts LMOOC completion (Duru et al., Reference Duru, Sunar, White, Diri and Dogan2019; Martín-Monje et al., Reference Martín-Monje, Castrillo and Mañana-Rodríguez2018). Resonating with such findings, our study not only confirmed the importance of engaging with in-LMOOC learning activities, such as viewing slides, announcements, and videos, but also highlighted the unique role of learning check-ins in LMOOC completion. Although no previous research seems to have tested the effects of check-in behavior per se on LMOOC completion, our findings are consistent with those of Jitpaisarnwattana et al. (Reference Jitpaisarnwattana, Reinders and Darasawang2021), who reported that learners who had ILPs were more likely to complete than those who did not. The present study also extends Jitpaisarnwattana et al.’s findings in that (1) it did not limit language skills to presentation skill; rather, learners were given free rein to decide what language skill(s) they wished to improve; and (2) the LMOOC in Jitpaisarnwattana et al.’s study was cMOOC, which emphasized community-building and knowledge-sharing. Although our sampled LMOOC itself looked more like an xMOOC than a cMOOC – that is, followed a traditional mode of teaching that focused on the transmission of knowledge (Fidalgo-Blanco et al., Reference Fidalgo-Blanco, Sein-Echaluce and García-Peñalvo2016) – the addition of the check-in activity was arguably a community aspect. Thus, pedagogically, the inclusion of check-in elements can be treated as a new design feature that encourages engagement and participation, and therefore could be seen as bridging the gap between cMOOCs and xMOOCs, at least to some extent (Godwin-Jones, Reference Godwin-Jones2014).

Turning now to the relationship between check-in behavior and learning outcomes other than LMOOC completion, it was interesting to see that the check-in group earned lower quiz scores than its non-check-in counterpart, especially on quizzes administered in earlier stages of the course. Such results raise the important possibility that those who chose to check in on their progress had relatively low English proficiency at the beginning of the course. As Read, Sedano and Bárcena (Reference Read, Sedano and Bárcena2018) pointed out, LMOOCs are promising for at-risk learners, but it is nevertheless important to set a combined mind and action plan to facilitate engagement and learning by this group. Through careful course design, diligent work and active engagement with various course activities, vulnerable language learners can improve their language competency (Read & Bárcena, Reference Read and Bárcena2021). A similar positive result was observed in our study, with the check-in group earning significantly higher final exam scores than other students who took the exam. In our specific study context, we suspect that this was because their relatively low initial proficiency in English prompted some learners to make greater efforts to seek and utilize various learning resources, devote more time to learning, and check in on their learning progress more spontaneously and more actively than their initially more-proficient peers. And crucially, these efforts – including ones outside the assigned material – paid off in terms of their final exam scores.

Admittedly, the significantly lower final exam grades of the non-check-in group could have been because doing well in the exam was not their primary reason for taking the course. After all, some LMOOC users have been found to treat such courses differently from regular ones: in effect, as resource banks (Hsu, Reference Hsu2023; Mac Lochlainn et al., Reference Mac Lochlainn, Nic Giolla Mhichíl and Beirne2021). We do not deny this possibility, especially in the case of those students who engaged with the focal LMOOC at least minimally. However, among those who successfully completed it, there were still significant differences in final exam grades between those who checked in and those who did not. This was perhaps because those students who completed the course regarded their final exam with more or less equal seriousness, irrespective of whether they checked in; and, if that was the case, then the observed link between check-in behavior and academic improvement is still meaningful.

Regarding the relationship between the check-in behavior and other course-related learning behaviors, those who checked in generally engaged significantly more with in-LMOOC learning components than those who did not. This probably reflects the open nature of the LMOOC environment, in which language knowledge is learned not in isolation but in a connected and non-linear way (Agonács et al., Reference Agonács, Matos, Bartalesi-Graf and O’Steen2020). In a broader sense, the act of checking in belongs to what Ushioda (Reference Ushioda2009) conceptualized as a wider system of language-learning activities and experiences, in which language acquisition occurs not only while taking the LMOOC but also in an ecological, open-ended manner that renders the boundary between the course and the rest of one’s life effectively meaningless (see also Chacón-Beltrán, Reference Chacón-Beltrán2018; Mac Lochlainn et al., Reference Mac Lochlainn, Nic Giolla Mhichíl and Beirne2021). When LMOOC learners maximize their opportunities to use a variety of learning resources, especially external ones (Agonács et al., Reference Agonács, Matos, Bartalesi-Graf and O’Steen2019), they become more dedicated, motivated, and successful language learners (Chacón-Beltrán, Reference Chacón-Beltrán2018; Ding & Shen, Reference Ding and Shen2022; Mac Lochlainn et al., Reference Mac Lochlainn, Nic Giolla Mhichíl and Beirne2021; Xu et al., Reference Xu, Zhu and Chan2023). However, one should not disregard the slight but potentially important differences we observed in specific learning behaviors. For example, among those who completed the sampled LMOOC, we found significant inter-group differences in total time spent checking course slides and videos, but not in the checking of class announcements. This seems to indicate that these particular behavioral differences are content-related rather than logistics-related.

Based on this discussion so far, it seems reasonable to regard check-in behavior as closely related to LMOOC course completion. Noting that our third and fourth research questions differ in terms of the population data used to answer them (i.e. those who successfully completed the course vs. those who merely engaged with it), it is worth asking whether check-in behavior is actually necessary among people for whom course completion is not a primary goal. In other words, in light of prior findings about various learner profiles, especially those who only treat LMOOCs as resource banks or learning networks (Hsu, Reference Hsu2023; Mac Lochlainn et al., Reference Mac Lochlainn, Nic Giolla Mhichíl and Beirne2021), are check-ins really helpful for all learners?

Our answer to this question is yes, for the reason that, unlike in some MOOCs in other disciplines, the learning materials in LMOOCs are not merely “a way of lecturing” (Ding & Shen, Reference Ding and Shen2022: 3). Rather, they are sources of authentic linguistic and cultural input with which learners are expected to engage actively (Sokolik, Reference Sokolik, Martín-Monje and Bárcena2014). And ungraded learning check-in behavior, despite seeming superficially to be an optional add-on or even wholly irrelevant to courses’ positive impact on their students, should in fact be treated as an integral part of LMOOC learning from the moment a learner starts to exhibit this behavior.

Some limitations of this study should be noted. First, because of the design of the learning platform, we were unable to collect learners’ demographic information, such as their genders, ages, and educational backgrounds. This problem was also raised by Martín-Monje et al. (Reference Martín-Monje, Castrillo and Mañana-Rodríguez2018), who noted that there was a disconnection between different data-storage venues in LMOOC studies. This further points to a need for seamless linkages between MOOC platforms and learning-analytics infrastructure. Second, the current study did not include students’ self-reported data on their learning progress or their own perceptions of their check-in behaviors, due to the difficulty of collecting such data. Third, this study included data drawn directly from the learning platform to indicate the time students spent watching videos. Although this could have accurately reflected the actual time spent watching, it would be impossible to prove or disprove this, except insofar as the platform-recorded time spent is an upper bound for such time; and the students’ levels of attention to the videos they played remain unknown. Lastly, our study was undertaken within a single intermediate-level English-language LMOOC. Thus, it is particularly important to try to replicate our findings about check-in behavior, and in particular, to test whether the effects we observed also exist in LMOOCs (1) with other target languages, especially less-spoken ones; (2) at other levels; (3) with different enrollment numbers; (4) delivered from different countries; and/or (5) used only as an add-on component of regular classroom language learning.

Given the importance of providing LMOOC learners with innovative assignments and instruction (Hsu, Reference Hsu2023; Zhang & Sun, Reference Zhang and Sun2023), future LMOOC educators should consider encouraging check-in behavior explicitly. Such encouragement should also take account of learners’ unique strengths and weaknesses in language-specific knowledge, as well as their respective learning contexts (Read & Bárcena, Reference Read and Bárcena2021; Zhang & Zhang, Reference Zhang and Zhang2017). Teachers should give special encouragement to participate in this activity to those learners with limited language proficiency. In addition, because promoting autonomy is crucial to the design of LMOOC platforms (Ding & Shen, Reference Ding and Shen2022; Zhang & Sun, Reference Zhang and Sun2023), future designers of such platforms should consider embedding check-in functions in their system architecture.

Conclusion

The current study used learning analytics to investigate a relatively new learning behavior in LMOOCs – learning check-ins – and studied the differences between the in-platform behavior of those who chose to engage in it and those who did not. Its findings show that check-ins varied with context and with which language skills learners were seeking to acquire. We also found that a positive role appeared to be played by check-in behavior. First, it was a significant predictor of LMOOC-completion status. Second, those who checked in earned relatively low scores on assessments near the start of their LMOOC learning journeys, but significantly higher final exam scores at the end of the six-month course. And third, students who chose to check in also engaged in 11 other measured learning behaviors significantly more often than those who did not. These findings add considerably to our knowledge of the complex learning elements in LMOOCs (Agonács et al., Reference Agonács, Matos, Bartalesi-Graf and O’Steen2020) and enable us to infer the types of activities that are effective in LMOOC learning (Mac Lochlainn et al., Reference Mac Lochlainn, Nic Giolla Mhichíl and Beirne2021; Zhang & Sun, Reference Zhang and Sun2023). Therefore, we confidently anticipate that support for and encouragement of student check-ins will be embedded in LMOOCs by designers and educators in the near future.

Acknowledgements

This study is funded by the Ministry of Education in China Humanities and Social Science Project, Grant/Award Number: 20YJC880126 and the Graduate Education Teaching Reform Project of Tsinghua University, 202303J032.

Ethical statement and competing interests

This study did not involve human subjects, human material, human tissues, or human data. The authors declare no competing interests and no use of generative AI.

About the authors

Yining Zhang is an associate professor in the Department of Foreign Languages and Literatures at Tsinghua University. Her research interests include online language teaching and learning, individual differences in language learning, and learning analytics in language teaching and learning.

Fang Yang is an associate professor in the Language Centre at Tsinghua University. Her research interests include applied linguistics and online language teaching and learning.

Haolin Yang is an undergraduate student in the Department of Foreign Languages and Literatures at Tsinghua University. His research interests are network analysis and graph neural networks.

Shuyuan Han is an undergraduate student in the Department of Foreign Languages and Literatures at Tsinghua University. Her research interest lies in sustainable development.

References

Agonács, N. & Matos, J. F. (2019) Understanding language MOOC learners: The issue of capability development. International Journal of Emerging Technologies in Learning (iJET), 14(11): 123137. https://doi.org/10.3991/ijet.v14i11.10205 CrossRefGoogle Scholar
Agonács, N., Matos, J. F., Bartalesi-Graf, D. & O’Steen, D. N. (2019) On the path to self-determined learning: A mixed methods study of learners’ attributes and strategies to learn in language MOOCs. International Journal of Learning Technology, 14(4): 304330. https://doi.org/10.1504/IJLT.2019.106553 CrossRefGoogle Scholar
Agonács, N., Matos, J. F., Bartalesi-Graf, D. & O’Steen, D. N. (2020) Are you ready? Self-determined learning readiness of language MOOC learners. Education and Information Technologies, 25(2): 11611179. https://doi.org/10.1007/s10639-019-10017-1 CrossRefGoogle Scholar
Appel, C. & Pujolà, J.-T. (2021) Designing speaking interaction in LMOOCs: An eTandem approach. ReCALL, 33(2): 161176. https://doi.org/10.1017/S0958344021000045 CrossRefGoogle Scholar
Bárcena, E. & Martín-Monje, E. (2014) Introduction. Language MOOCs: An emerging field. In Martín-Monje, E. & Bárcena, E. (eds.), Language MOOCs: Providing learning, transcending boundaries. Warsaw: De Gruyter Open, 115. https://doi.org/10.2478/9783110420067.1 Google Scholar
Beaven, T., Codreanu, T. & Creuzé, A. (2014) Motivation in a language MOOC: Issues for course designers. In Martín-Monje, E. & Bárcena, E. (eds.), Language MOOCs: Providing learning, transcending boundaries. Warsaw: De Gruyter Open, 4866. https://doi.org/10.2478/9783110420067.4 CrossRefGoogle Scholar
Benson, P. (2011) Teaching and researching autonomy (2nd ed.). Harlow: Pearson Education.Google Scholar
Chacón-Beltrán, R. (2018) Vocabulary learning strategies outside the classroom context: What adults learn in a technology-based learner-centred environment. The Language Learning Journal, 46(5): 583593. https://doi.org/10.1080/09571736.2018.1503135 CrossRefGoogle Scholar
Chen, X., Sun, M., Wu, D. & Song, X. Y. (2019) Information-sharing behavior on WeChat moments: The role of anonymity, familiarity, and intrinsic motivation. Frontiers in Psychology, 10: Article 2540. https://doi.org/10.3389/fpsyg.2019.02540 CrossRefGoogle Scholar
Ding, Y. & Shen, H. (2022) Delving into learner autonomy in an EFL MOOC in China: A case study. Computer Assisted Language Learning, 35(3): 247269. https://doi.org/10.1080/09588221.2019.1681464 CrossRefGoogle Scholar
Doğan, G., Sunar, A. S., Duru, İ. & White, S. (2018) Who is the English as a second language speaker in this MOOC? International Journal of Information and Education Technology, 8(3): 213217. https://doi.org/10.18178/ijiet.2018.8.3.1035 CrossRefGoogle Scholar
Duru, I., Sunar, A. S., White, S., Diri, B. & Dogan, G. (2019) A case study on English as a second language speakers for sustainable MOOC study. Sustainability, 11(10): Article 2808. https://doi.org/10.3390/su11102808 CrossRefGoogle Scholar
Fidalgo-Blanco, Á., Sein-Echaluce, M. L. & García-Peñalvo, F. J. (2016) From massive access to cooperation: Lessons learned and proven results of a hybrid xMOOC/cMOOC pedagogical approach to MOOCs. International Journal of Educational Technology in Higher Education, 13(1): Article 24. https://doi.org/10.1186/s41239-016-0024-z CrossRefGoogle Scholar
Friðriksdóttir, K. (2021) The effect of content-related and external factors on student retention in LMOOCs. ReCALL, 33(2): 128142. https://doi.org/10.1017/S0958344021000069 CrossRefGoogle Scholar
Fuchs, C. (2020) Cultural and contextual affordances in language MOOCs: Student perspectives. International Journal of Online Pedagogy and Course Design, 10(2): 4860. https://doi.org/10.4018/IJOPCD.2020040104 CrossRefGoogle Scholar
Gašević, D., Dawson, S. & Siemens, G. (2015) Let’s not forget: Learning analytics are about learning. TechTrends, 59(1), 6471. https://doi.org/10.1007/s11528-014-0822-xCrossRefGoogle Scholar
Gelan, A., Fastré, G., Verjans, M., Martin, N., Janssenswillen, G., Creemers, M., Lieben, J., Depaire, B. & Thomas, M. (2018) Affordances and limitations of learning analytics for computer-assisted language learning: A case study of the VITAL project. Computer Assisted Language Learning, 31(3): 294319. https://doi.org/10.1080/09588221.2017.1418382 CrossRefGoogle Scholar
Gimeno-Sanz, A. (2021) LMOOCs: Free, self-access language learning on a global scale. In Beaven, T. & Rosell-Aguilar, F. (eds.), Innovative language pedagogy report. Voillans: Research-publishing.net, 4955. https://doi.org/10.14705/rpnet.2021.50.1235 CrossRefGoogle Scholar
Godwin-Jones, R. (2014) Global reach and local practice: The promise of MOOCs. Language Learning & Technology, 18(3): 515. https://doi.org/10125/44377 Google Scholar
Hood, N., Littlejohn, A. & Milligan, C. (2015) Context counts: How learners’ contexts influence learning in a MOOC. Computers & Education, 91: 8391. https://doi.org/10.1016/j.compedu.2015.10.019 CrossRefGoogle Scholar
Hsu, L. (2023) What makes good LMOOCs for EFL learners? Learners’ personal characteristics and Information System Success Model. Computer Assisted Language Learning, 36(1–2): 125. https://doi.org/10.1080/09588221.2021.1899243 CrossRefGoogle Scholar
Huang, J. & Benson, P. (2013) Autonomy, agency and identity in foreign and second language education. Chinese Journal of Applied Linguistics, 36(1): 728. https://doi.org/10.1515/cjal-2013-0002 CrossRefGoogle Scholar
Jitpaisarnwattana, N., Darasawang, P. & Reinders, H. (2022) Defining success in a language MOOC from learner perspectives. International Journal of Computer-Assisted Language Learning and Teaching, 12(1): 116. https://doi.org/10.4018/IJCALLT.291108 CrossRefGoogle Scholar
Jitpaisarnwattana, N., Reinders, H. & Darasawang, P. (2021) Understanding the roles of personalization and social learning in a language MOOC through learning analytics. Online Learning, 25(4): 324343. https://doi.org/10.24059/olj.v25i4.2509 CrossRefGoogle Scholar
Kim, H.-S. (2016) What drives you to check in on Facebook?: Motivations, privacy concerns, and mobile phone involvement for location-based information sharing. Computers in Human Behavior, 54: 397406. https://doi.org/10.1016/j.chb.2015.08.016 CrossRefGoogle Scholar
Leitner, P., Khalil, M. & Ebner, M. (2017) Learning analytics in higher education—A literature review. In Peña-Ayala, A. (ed.), Learning analytics: Fundaments, applications, and trends: A view of the current state of the art to enhance e-learning. Cham: Springer International Publishing, 123. https://doi.org/10.1007/978-3-319-52977-6_1 Google Scholar
Little, D. (1991) Learner autonomy 1: Definitions, issues and problems. Teacher & Learner Perspectives, 62(4): 395397.Google Scholar
Luarn, P., Yang, J.-C. & Chiu, Y.-P. (2015) Why people check in to social network sites. International Journal of Electronic Commerce, 19(4): 2146. https://doi.org/10.1080/10864415.2015.1029353 CrossRefGoogle Scholar
Mac Lochlainn, C., Nic Giolla Mhichíl, M. & Beirne, E. (2021) Clicking, but connecting? L2 learning engagement on an ab initio Irish language LMOOC. ReCALL, 33(2): 111127. https://doi.org/10.1017/S0958344021000100 CrossRefGoogle Scholar
Mann, H. B. & Whitney, D. R. (1947) On a test of whether one of two random variables is stochastically larger than the other. The Annals of Mathematical Statistics, 18(1): 5060. https://doi.org/10.1214/aoms/1177730491 CrossRefGoogle Scholar
Martín-Monje, E., Castrillo, M. D. & Mañana-Rodríguez, J. (2018) Understanding online interaction in language MOOCs through learning analytics. Computer Assisted Language Learning, 31(3): 251272. https://doi.org/10.1080/09588221.2017.1378237 CrossRefGoogle Scholar
Nie, J., Zheng, C., Zeng, P., Zhou, B., Lei, L. & Wang, P. (2020) Using the theory of planned behavior and the role of social image to understand mobile English learning check-in behavior. Computers & Education, 156: Article 103942. https://doi.org/10.1016/j.compedu.2020.103942 CrossRefGoogle Scholar
Read, T. & Bárcena, E. (2021) The role of activeness for potentiating learning in LMOOCs for vulnerable groups. Journal of Interactive Media in Education, 2021(1): 113. https://doi.org/10.5334/jime.628 CrossRefGoogle Scholar
Read, T., Sedano, B. & Bárcena, E. (2018) Tailoring language MOOC design for migrants and refugees. In Read, T., Montaner, S. & Sedano, B. (eds.), Technological innovation for specialized linguistic domains: Languages for digital lives and cultures: Proceedings of TISLID’18. Mauritius: Éditions universitaires européennes, 383–396.Google Scholar
Sallam, M. H., Martín-Monje, E. & Li, Y. (2022) Research trends in language MOOC studies: A systematic review of the published literature (2012-2018). Computer Assisted Language Learning, 35(4): 764791. https://doi.org/10.1080/09588221.2020.1744668 CrossRefGoogle Scholar
Shapiro, S. S. & Wilk, M. B. (1965) An analysis of variance test for normality (complete samples). Biometrika, 52(3–4): 591611. https://doi.org/10.1093/biomet/52.3-4.591 CrossRefGoogle Scholar
Sokolik, M. (2014) What constitutes an effective language MOOC? In Martín-Monje, E. & Bárcena, E. (eds.), Language MOOCs: Providing learning, transcending boundaries. Warsaw: De Gruyter Open, 1632. https://doi.org/10.2478/9783110420067.2 CrossRefGoogle Scholar
Ushioda, E. (2009) A person-in-context relational view of emergent motivation, self and identity. In Dörnyei, Z. & Ushioda, E. (eds.), Motivation, language identity and the L2 self. Bristol: Multilingual Matters, 215–228. https://doi.org/10.21832/9781847691293-012 CrossRefGoogle Scholar
van Rossum, G. & Drake, F. L. (2009) Python 3 reference manual. Scotts Valley: CreateSpace.Google Scholar
Wang, J., An, N. & Wright, C. (2018) Enhancing beginner learners’ oral proficiency in a flipped Chinese foreign language classroom. Computer Assisted Language Learning, 31(5–6): 490521. https://doi.org/10.1080/09588221.2017.1417872 CrossRefGoogle Scholar
Wang, N., Chen, J., Tai, M. & Zhang, J. (2021) Blended learning for Chinese university EFL learners: Learning environment and learner perceptions. Computer Assisted Language Learning, 34(3): 297323. https://doi.org/10.1080/09588221.2019.1607881 CrossRefGoogle Scholar
Xu, X., Zhu, X. & Chan, F. M. (2023) System design of Pintrich’s SRL in a supervised-PLE platform: A pilot test in higher education. Interactive Learning Environments, 31(2): 683700. https://doi.org/10.1080/10494820.2020.1802296 CrossRefGoogle Scholar
Xue, X. & Dunham, R. E. (2023) Using a SPOC-based flipped classroom instructional mode to teach English pronunciation. Computer Assisted Language Learning, 36(7): 13091337. https://doi.org/10.1080/09588221.2021.1980404 CrossRefGoogle Scholar
Zeng, S., Zhang, J., Gao, M., Xu, K. M. & Zhang, J. (2022) Using learning analytics to understand collective attention in language MOOCs. Computer Assisted Language Learning, 35(7): 15941619. https://doi.org/10.1080/09588221.2020.1825094 CrossRefGoogle Scholar
Zhang, E. & Zhang, W. (2017) Construction and application of MOOC-based college English micro lesson system. International Journal of Emerging Technologies in Learning (iJET), 12(2): 155165. https://doi.org/10.3991/ijet.v12i02.6374 Google Scholar
Zhang, Y. & Sun, R. (2023) LMOOC research 2014 to 2021: What have we done and where are we going next? ReCALL, 35(3): 356371. https://doi.org/10.1017/S0958344022000246 CrossRefGoogle Scholar
Figure 0

Figure 1. Learners’ check-in behavior by content type.

Figure 1

Table 1. Frequency counts by completion status and check-in status

Figure 2

Table 2. Logistic regression, learning behaviors on course completion

Figure 3

Table 3. Learning outcomes of the check-in and non-check-in subgroups of students who completed the LMOOC

Figure 4

Table 4. Learning behaviors of the check-in and non-check-in subgroups of students who completed the LMOOC

Figure 5

Table 5. Learning outcomes of the check-in and non-check-in subgroups of students who engaged with the LMOOC

Figure 6

Table 6. Learning behaviors of the check-in and non-check-in subgroups of students who engaged with the LMOOC