Lesson 11 - Massive Open Online Courses

5.6. What Do Students Learn in MOOCs?

This is a much more difficult question to answer because so little of the research to date (2019) has tried to answer this question. (One reason, as we shall see in the next section, is that assessment of learning in MOOCs remains a major challenge). There are at least two kinds of study: quantitative studies that seek to quantify learning gains; and qualitative studies that describe the experience of learners within MOOCs, which indirectly provide some insight into what they have learned.

Conceptual Learning

At the time of writing, the most quantitative study of learning in MOOCs has been by Colvin et al.(2014), who investigated ‘conceptual learning’ in an MIT Introductory Physics MOOC. Colvin and colleagues compared learner performance not only between different sub-categories of learners within the MOOC, such as those with no physics or math background with those such as physic teachers who had considerable prior knowledge, but also with on-campus students taking the same curriculum in a traditional campus teaching format. In essence, the study found no significant differences in learning gains between or within the two types of teaching, but it should be noted that the on-campus students were students who had failed an earlier version of the course and were retaking it.

This research is a classic example of the no significant difference in comparative studies in educational technology; other variables, such as differences in the types of students, were as important as the mode of delivery. Also, this MOOC design represents a behaviourist-cognitivist approach to learning that places heavy emphasis on correct answers to conceptual questions.

The Student Experience

here have been far more studies of the experience of learners within MOOCs, particularly focusing on the discussions within MOOCs (see for instance, Kop, 2011). In general (although there are exceptions), discussions are unmonitored, and it is left to participants to make connections and respond to other students’ comments.

However, there are some strong criticisms of the effectiveness of the discussion element of MOOCs for developing the high-level conceptual analysis required for academic learning. There is evidence from studies of credit-based online learning that to develop deep, conceptual learning, there is a need in most cases for intervention by a subject expert to clarify misunderstandings or misconceptions, to provide accurate feedback,  to ensure that the criteria for academic learning, such as  use of evidence, clarity of argument, and so on, are being met, and to ensure the necessary input and guidance to seek deeper understanding (see in particular Harasim, 2017).

Furthermore, the more massive the course, the more likely participants are to feel ‘overload, anxiety and a sense of loss’, if there is not some instructor intervention or structure imposed (Knox, 2014). Firmin et al. (2014) have shown that when there is some form of instructor ‘encouragement and support of student effort and engagement’, results improve for all participants in MOOCs. Without a structured role for subject experts, participants are faced with a wide variety of quality in terms of comments and feedback from other participants. There is again a great deal of research on the conditions necessary for the successful conduct of collaborative and co-operative group learning (see, for instance, Lave and Wenger, 1991, or Barkley, Major and Cross, 2014), and these findings certainly have not been generally applied to the management of MOOC discussions. 

Networked and Collaborative Learning

One counter-argument is that cMOOCs in particular develop a new form of learning based on networking and collaboration that is essentially different from academic learning, and cMOOCs are thus more appropriate to the needs of learners in a digital age. Adult participants in particular, it is claimed by Downes and Siemens, have the ability to self-manage the development of high-level conceptual learning.  cMOOCs are ‘demand’ driven, meeting the interests of individual students who seek out others with similar interests and the necessary expertise to support them in their learning, and for many this interest may well not include the need for deep, conceptual learning but more likely the appropriate applications of prior knowledge in new or specific contexts. All MOOCs do appear to work best for those who already have a high level of education and therefore bring many of the conceptual skills developed in formal education with them when they join a MOOC, and therefore contribute to helping those who come without such prior knowledge or skills.

The Need for Learner Support

Over time, as more experience is gained, MOOCs are likely to incorporate and adapt some of the findings from research on smaller group work to the much larger numbers in MOOCs. For instance, some MOOCs are using ‘volunteer’ or community tutors. The US State Department organized MOOC camps through US missions and consulates abroad to mentor MOOC participants. The camps included Fulbright scholars and embassy staff who lead discussions on content and topics for MOOC participants in countries abroad (Haynie, 2014).

Some MOOC providers, such as the University of British Columbia, paid a small cohort of academic assistants to monitor and contribute to the MOOC discussion forums (Engle, 2014). Engle reported that the use of academic assistants, as well as limited but effective interventions from the instructors themselves made the UBC MOOCs more interactive and engaging.

However, paying for people to monitor and support MOOCs will of course increase the cost to providers. Consequently, MOOCs are likely to develop new automated ways to manage discussions effectively in very large groups. For instance, the University of Edinburgh experimented with an automated ‘teacherbot’ that crawled through student and instructor Twitter posts associated with a MOOC and directed predetermined comments to students to prompt discussion and reflection (Bayne, 2015). These results and approaches are consistent with prior research on the importance of instructor presence for successful online learning in credit-based courses.

In the meantime, though, there is much work still to be done if MOOCs are to provide the support and structure needed to ensure deep, conceptual learning where this does not already exist in students. The development of the skills needed in a digital age is likely to be an even greater challenge when dealing with massive numbers. However, we need much more research into what participants actually learn in MOOCs and under what conditions before any firm conclusions can be drawn.