Change 11 SRL-MOOC study: initial findings

Kelvingrove

As you will remember the Caledonian Academy conducted some research during the recent Change11 MOOC run by George Siemens, Stephen Downes and Dave Cormier.  The study generated a lot of data, which has been sitting on my desk for some months now. The hypothesis for the study was that we would observe different learning behaviours and different approaches to learning in MOOCs among those with different SRL profiles.

What did we do?

SRL Profiles: The first component of the study was to ask participants to complete an SRL profile instrument* we had developed for the study. The instrument was adapted from a number of pre-existing SRL self-report  instruments (full details, and a copy of the instrument are here), most notably the Motivated Strategies for Learning Questionnaire (Pintrich et al 1991) and a more recent Self directed Learning Orientation scale developed by Raemdonck (Gijbels et al , 2010). Within the instrument there are a number of sub-scales measuring specific SRL components (task analysis, goal setting, self-reflection etc. from the list of sub-processes described in Zimmermann, 2005). 35 respondents completed an SRL profile.

Interview: Everyone who completed an SRL profile was invited to participate in a semi-structured interview. The interview explored a number of factors, including motivation, previous experience of MOOCs, goal-setting, knowledge sharing, and participation patterns.  We conducted 29 interviews generating 26 hrs of interviews.

Emerging themes from the study

I plan to go into detail about many of the findings of the study through a series of posts,  but thought an initial post outlining some of the key things we turned up would be of interest. We identified a number of themes from the interviews, and I shall discuss four briefly here:

  • patterns of engagement: we saw different patterns of engagement. In addition to an expected cluster of lurkers who purposefully did not engage with other course participants, we identified two further groups: one group of passive participants, who expected ‘to be taught’, and viewed the course as a source of information, attempting to capture all the ideas being exchanged within the Change 11 community; and a final group, more active participants, who set their own goals, established connections with other learners and linked these connections with their existing personal learning network. These overall patterns of engagement are mediated by a number of factors including: previous participation in other MOOCs, motivation for participating in the MOOC, makeup of existing personal learning network, SRL profile, and confidence. Lurkers will be present in any online course, and the final group are archetypal cMOOC participants, but those more traditional learners in the middle group really wanted more guidance than they were given.
  • patterns of tool use: we have written before about what we call “the 4c’s“: consume, connect, create and contribute, as different learning behaviours that individuals use as they learn in networked environments. You can consume content, organise it by connecting content together and to people, create new content (knowledge) of your own, and contribute that new knowledge back into the community. We have collected data about the different tools in use and the types of behaviour they support. To quote one example, of resource management. We collected data about people using a range of tools to collect resources (bookmarks). While some respondents in our study were using their (single user) browser bookmark menus (consume), most were using some social service, such as delicious or diigo (connect).  But we also saw people who were interpreting/providing commentary on the content they were collecting using tools like scoop.it (create) and using such tools to ‘find their own voice’ in the community (contribute).
  • goal-setting behaviour: Goal setting is an important task in participation in MOOCs as, without a closely defined curriculum, it is incumbent on the learner to set and monitor their own goals. Most participants from the medium and high SR Scores groups set clearly defined goals, though we found no difference in the nature of the goals set with each group setting both participation and performance based goals. Those participants in the Low SR scores group were less likely to have set goals and were more likely to articulate goals categorised as ‘vague’ when prompted. Several participants in the study described how their goals had changed as they progressed through the course, showing evidence of reflection. We also uncovered evidence of other factors that might contribute to goal setting behaviour. Four participants in particular argued that the nature of this MOOC course (with a broad and only loosely defined curriculum) was incompatible with the idea of ‘goal-setting’ and had therefore explicitly chosen not to set goals.
  • motivation to participate: one problem with our study which we hadn’t anticipated (but perhaps should have) was that individual participants might have quite different (conflicting?) reasons for signing up. While some participants signed up for the content of the course, others (the majority) were primarily or exclusively interested in experiencing the Change 11 MOOC as a learning environment, often because they wanted to implement some of the features of a MOOC in their own practice. Several of the participants had previously participated in other MOOCs and when discussing their goals, these participants remarked that their goal –setting behaviour had been different in courses with a more defined topic (eg mobiMOOC and an ePortoflio MOOC, but others too).

What these results tell us about moocs

For me, the recurring theme from this research was that massive courses do need management (of learners, and their expectations), or at least a recognition of the diversity of learner backgrounds, preferences, expectations and motivations that come together in a MOOC, that is then reflected in the design of the learning space which is constructed. I suppose the prevalent (c) MOOC philosophy is that learners should be left to their own devices and they will find their place in the emerging learning networks(anywhere on the spectrum from lurking to leading). We certainly saw interesting evidence of self-organisation, especially among those who engaged with the course through the facebook group, and the twitter chats. But our findings indicate that some users either didn’t find these emerging networks (or at least didn’t identify a network that suited them), or didn’t recognise the central role that these networks play in leveraging the value of the course. While I don’t advocate creating rigid structures, I do think there are some simple things that could be done to make sure MOOCs such as Change11 are accessible by the full range of prospective participants.

Reflections on this study

I am still analysing the data from this study and we hope to publish the significant findings.  The specific nature of the Change 11 mooc (attracting many participants who wanted to ‘experience a mooc delivered by Downes, Siemens and Cormier’) complicates the analysis we can make about SRL behaviours as the different overall goals people held (primarily interested in ‘content’ or ‘process’) influenced their learning behaviour to such a great extent. Stay tuned for further blog posts after the holiday.

Bibliography

Gijbels, David; Raemdonck, I.; Vervecken, D. - Influencing work-related learning: the role of job characteristics and self-directed learning orientation in part-time vocational education - In: Vocations and learning:3(2010), p. 239-255.

Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). Ann Arbor: University of Michigan, National Center for Research to Improve Postsecondary Teaching and Learning.

Zimmerman, B. (2005). Attaining self-regulation: A social cognitive perspective. In Boekaerts, M.,Pintrich, P., & Zeidner, M. (Eds.), Handbook of Self-Regulation . San Diego: Academic Press.

—-

*While we didn’t validate this instrument, we are reasonably confident of its validity as it is based so closely on existing validated instruments and was adequate for our purposes (in a related ongoing project, we are actually going to validate an SRL instrument with around 400 people).

About these ads

Learning researcher based in Glasgow, and living in the Loch Lomond National Park.

Tagged with:
Posted in change11, research
18 comments on “Change 11 SRL-MOOC study: initial findings
  1. [...] As you will remember we conducted some research during the recent Change11 MOOC run by George Siemens, Stephen Downes and Dave Cormier.  The study generated a lot of data, which has been sitting on…  [...]

  2. [...] As you will remember the Caledonian Academy conducted some research during the recent Change11 MOOC run by George Siemens, Stephen Downes and Dave Cormier. The study generated a lot of data, which has been sitting on my desk for some months now. The hypothesis for the study was that we would observe different learning behaviours and different approaches to learning in MOOCs among those with different SRL profiles.  [...]

  3. alison says:

    Reblogged this on connectivité and commented:
    This is important work. I’m certainly going to be watching for the full analysis!

  4. [...] As you will remember we conducted some research during the recent Change11 MOOC run by George Siemens, Stephen Downes and Dave Cormier. The study generated a lot of data, which has been sitting on…  [...]

  5. Colin, I would be particularly interested in your thoughts related to this statement:

    “While I don’t advocate creating rigid structures, I do think there are some simple things that could be done to make sure MOOCs such as Change11 are accessible by the full range of prospective participants.”

    How do well designed structure, processes and resources aid in self-organization and connectivity? Hope you explore this in further detail. Thanks.

  6. [...] Change 11 SRL-MOOC study: Initial findings, Blog: Learning in the Workplace [...]

  7. [...] previous post describing initial findings from our SRL-MOOC study got a spike of views yesterday suggesting that [...]

  8. [...] In 2011-2012 as part of the Change11 course (a connectivist course, or cMOOC, facilitated by George Siemens, Dave Cormier and Stephen Downes), the Scottish group Caledonian Academy was given access for surveys and follow-up interviews to help understand the student population in a research study. [...]

  9. [...] is often seen as a vital part of learning this is a major advantage. However, early studies (Milligan, 2012) suggest that while some people engage actively, others lurk (purposefully not engaging), and [...]

  10. [...] SRL-MOOC study has been getting some attention thanks to a post by Phil Hill which built on the categories I [...]

  11. [...] is often seen as a vital part of learning this is a major advantage. However, early studies (Milligan, 2012) suggest that while some people engage actively, others lurk (purposefully not engaging), and [...]

  12. [...] Of course I am talking about cMOOCs here (xMOOCs will have to wait for another day, but Dave Cormier has a related post from last weekend which explores similar issues). My comment continued with this observation (from participation in cMOOcs, and from researching Change11) [...]

  13. [...] Follow-up research on Change MOOC on how learners acted has revealed different groups of learners ranging from lurkers, passive participants (wanting a traditional learning experience with goals being set by course) and active learners (setting their own goals, creating artifacts, social networking). While these are broad (and contestable) categories (Hill, 2013), the research indicated it is these ‘passive participants’ that struggle to find direction and location in a connectivist MOOC (Kop 2011) and for whom a Coursera approach might be more attractive. [...]

  14. [...] and strategies of knowledge workers when they are done. Collin has already written about some initial findings on his [...]

  15. nana yao says:

    i am an M.Ed IT student from cape coast university, ghana, africa.
    i am currently researching in integrating MOOC course ware in universities in ghana and need information.

  16. […] [47] http://worklearn.wordpress.com/2012/12/19/change-11-srl-mooc-study-initial-findings/  […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

This is me

I'm a workplace learning researcher at the Caledonian Academy, Glasgow Caledonian University, in Scotland. When not in work, I live in the beautiful Loch Lomond National Park.

Flickr Photos
Joseph at the Lake

P7080860

P7080857

More Photos
Follow

Get every new post delivered to your Inbox.