Change 11 SRL-MOOC study: initial findings

As you will remember the Caledonian Academy conducted some research during the recent Change11 MOOC run by George Siemens, Stephen Downes and Dave Cormier.  The study generated a lot of data, which has been sitting on my desk for some months now. The hypothesis for the study was that we would observe different learning behaviours and different approaches to learning in MOOCs among those with different SRL profiles.

What did we do?

SRL Profiles: The first component of the study was to ask participants to complete an SRL profile instrument* we had developed for the study. The instrument was adapted from a number of pre-existing SRL self-report  instruments (full details, and a copy of the instrument are here), most notably the Motivated Strategies for Learning Questionnaire (Pintrich et al 1991) and a more recent Self directed Learning Orientation scale developed by Raemdonck (Gijbels et al , 2010). Within the instrument there are a number of sub-scales measuring specific SRL components (task analysis, goal setting, self-reflection etc. from the list of sub-processes described in Zimmermann, 2005). 35 respondents completed an SRL profile.

Interview: Everyone who completed an SRL profile was invited to participate in a semi-structured interview. The interview explored a number of factors, including motivation, previous experience of MOOCs, goal-setting, knowledge sharing, and participation patterns.  We conducted 29 interviews generating 26 hrs of interviews.

Emerging themes from the study

I plan to go into detail about many of the findings of the study through a series of posts,  but thought an initial post outlining some of the key things we turned up would be of interest. We identified a number of themes from the interviews, and I shall discuss four briefly here:

• patterns of engagement: we saw different patterns of engagement. In addition to an expected cluster of lurkers who purposefully did not engage with other course participants, we identified two further groups: one group of passive participants, who expected ‘to be taught’, and viewed the course as a source of information, attempting to capture all the ideas being exchanged within the Change 11 community; and a final group, more active participants, who set their own goals, established connections with other learners and linked these connections with their existing personal learning network. These overall patterns of engagement are mediated by a number of factors including: previous participation in other MOOCs, motivation for participating in the MOOC, makeup of existing personal learning network, SRL profile, and confidence. Lurkers will be present in any online course, and the final group are archetypal cMOOC participants, but those more traditional learners in the middle group really wanted more guidance than they were given.
• patterns of tool use: we have written before about what we call “the 4c’s“: consume, connect, create and contribute, as different learning behaviours that individuals use as they learn in networked environments. You can consume content, organise it by connecting content together and to people, create new content (knowledge) of your own, and contribute that new knowledge back into the community. We have collected data about the different tools in use and the types of behaviour they support. To quote one example, of resource management. We collected data about people using a range of tools to collect resources (bookmarks). While some respondents in our study were using their (single user) browser bookmark menus (consume), most were using some social service, such as delicious or diigo (connect).  But we also saw people who were interpreting/providing commentary on the content they were collecting using tools like scoop.it (create) and using such tools to ‘find their own voice’ in the community (contribute).
• goal-setting behaviour: Goal setting is an important task in participation in MOOCs as, without a closely defined curriculum, it is incumbent on the learner to set and monitor their own goals. Most participants from the medium and high SR Scores groups set clearly defined goals, though we found no difference in the nature of the goals set with each group setting both participation and performance based goals. Those participants in the Low SR scores group were less likely to have set goals and were more likely to articulate goals categorised as ‘vague’ when prompted. Several participants in the study described how their goals had changed as they progressed through the course, showing evidence of reflection. We also uncovered evidence of other factors that might contribute to goal setting behaviour. Four participants in particular argued that the nature of this MOOC course (with a broad and only loosely defined curriculum) was incompatible with the idea of ‘goal-setting’ and had therefore explicitly chosen not to set goals.
• motivation to participate: one problem with our study which we hadn’t anticipated (but perhaps should have) was that individual participants might have quite different (conflicting?) reasons for signing up. While some participants signed up for the content of the course, others (the majority) were primarily or exclusively interested in experiencing the Change 11 MOOC as a learning environment, often because they wanted to implement some of the features of a MOOC in their own practice. Several of the participants had previously participated in other MOOCs and when discussing their goals, these participants remarked that their goal –setting behaviour had been different in courses with a more defined topic (eg mobiMOOC and an ePortoflio MOOC, but others too).

What these results tell us about moocs

For me, the recurring theme from this research was that massive courses do need management (of learners, and their expectations), or at least a recognition of the diversity of learner backgrounds, preferences, expectations and motivations that come together in a MOOC, that is then reflected in the design of the learning space which is constructed. I suppose the prevalent (c) MOOC philosophy is that learners should be left to their own devices and they will find their place in the emerging learning networks(anywhere on the spectrum from lurking to leading). We certainly saw interesting evidence of self-organisation, especially among those who engaged with the course through the facebook group, and the twitter chats. But our findings indicate that some users either didn’t find these emerging networks (or at least didn’t identify a network that suited them), or didn’t recognise the central role that these networks play in leveraging the value of the course. While I don’t advocate creating rigid structures, I do think there are some simple things that could be done to make sure MOOCs such as Change11 are accessible by the full range of prospective participants.

Reflections on this study

I am still analysing the data from this study and we hope to publish the significant findings.  The specific nature of the Change 11 mooc (attracting many participants who wanted to ‘experience a mooc delivered by Downes, Siemens and Cormier’) complicates the analysis we can make about SRL behaviours as the different overall goals people held (primarily interested in ‘content’ or ‘process’) influenced their learning behaviour to such a great extent. Stay tuned for further blog posts after the holiday.

Bibliography

Gijbels, David; Raemdonck, I.; Vervecken, D. – Influencing work-related learning: the role of job characteristics and self-directed learning orientation in part-time vocational education – In: Vocations and learning:3(2010), p. 239-255.

Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). Ann Arbor: University of Michigan, National Center for Research to Improve Postsecondary Teaching and Learning.

Zimmerman, B. (2005). Attaining self-regulation: A social cognitive perspective. In Boekaerts, M.,Pintrich, P., & Zeidner, M. (Eds.), Handbook of Self-Regulation . San Diego: Academic Press.

—-

*While we didn’t validate this instrument, we are reasonably confident of its validity as it is based so closely on existing validated instruments and was adequate for our purposes (in a related ongoing project, we are actually going to validate an SRL instrument with around 400 people).