Productive MOOCs

OLYMPUS DIGITAL CAMERA

One of my frustrations with MOOCs is that while they provide value for the individual, their (lack of) collective value is a missed opportunity. Although I’ve had a blog post in the works for almost a year, I could never work out how to write it concisely. A recent post by Rachel Harris of Inspire Research  provided a slightly different angle to address the issue.

Rachel is evaluating the ALT ocTEL MOOC and one of her evaluation questions is ‘Can new knowledge be generated via critical discourse in a MOOC?’. With our other research interests here at the Caledonian Academy we are really interested in knowledge creation and collective knowledge. So,  in the comments I said:

I’m firmly of the belief that knowledge creation is a key component of learning, and successful MOOC learners will (through writing and responding to blog posts) create new knowledge and advance their thinking.

Of course I am talking about cMOOCs here (xMOOCs will have to wait for another day, but Dave Cormier has a related post from last weekend which explores similar issues). My comment continued with this observation (from participation in cMOOcs, and from researching Change11)

But one of my concerns in observing other MOOCs is that while each individual might advance their knowledge, there is little in the way of collective knowledge advancement. What this means in practice is that there is lot of duplicated knowledge produced during the MOOC.

Wouldn’t it be great if cMOOcs could be made more ‘productive’ – instead of advancing many people’s knowledge a little by re-creating the same (or similar) new knowledge again and again, can MOOCs be structured to stimulate the creation of new knowledge in a more coordinated way. Can you bring the learners together to produce something entirely novel as they learn? This is in the true spirit of connectivism.

Aside from my desire to see new knowledge created, there is actually an important learning observation here. As I went on to say in the comment on Rachel’s post:

 To the active participants, this can be frustrating as the consequence is that there is too much ‘noise’ in the system.

You might read a number of blog posts by your fellow participants, each making the same point. In the Change11 MOOC, some participants found this boring, others (particularly the less confident) decided not to articulate their own view because they thought it had already been presented and didn’t want to ‘just repeat’. But articulating what you have learned is important so this is a missed learning opportunity.

On a more positive note, the emergence of self-organised spaces in the Chage 11 MOOC (the facebook group, the One Change a Day group) showed that co-creation could succeed, but it was only a minority of participants who managed to engage in this coordinated way.

Solutions. I’m not sure I have any,  but here is one idea**: Can MOOCs be used to encourage interdisciplinarity? An interdisciplinary*MOOC could bring together experts from different fields to explore a topic from different perspectives and adopting each other’s methods etc. The ultimate product could be the new knowledge created as a result, perhaps a series of working papers. The immediate flaw in this is that it doesn’t seem very massive (or at least the productive element might be the result just of the collaboration of the instructors), but if you can structure the MOOC participants as disciplinary ‘tribes’ then you can focus groups on creation of specific knowledge – e.g. initially discipline sub-groups could cooperate to co-create a disciplinary view, then be re-assorted into inter-disciplinary groups to explore the overlaps and tensions. oriented towards a more specific goal that matches their existing knowledge and motivation,  they can work together to co-create something that can be contributed back to the group. A structure such as this might also help counter the tension/imbalance between homophily and heterophily that I think is inherent in MOOCs.

* by interdisciplinarity, I mean ‘integrating knowledge and methods from different disciplines, using a real synthesis of approaches’ – taken from http://www.arj.no/2012/03/12/disciplinarities-2/ which in turn is from a paper by Stember (1990)

**I’ll dig out the notes for the original blog post and write more at a later date.

Advertisements

Designing MOOCs

Image

My previous post describing initial findings from our SRL-MOOC study got a spike of views yesterday suggesting that someone pointed at it (thank you, whoever you are). One comment, from Felicia Sullivan asked me to expand on the following statement from the conclusion of that post:

“While I don’t advocate creating rigid structures, I do think there are some simple things that could be done to make sure MOOCs such as Change11 are accessible by the full range of prospective participants.”

I’d been reflecting on what our study has told us about the design of MOOCs and an observation that (in Change11) while some participants found and joined networks without problems, others didn’t seem to find their place in the MOOC community so easily.

There’s an implicit assumption in this: that you need to find a network to succeed in a cMOOC. In fact I don’t necessarily believe this – we found lurkers – who chose not to attempt to interact with others. These lurkers used the MOOC as a source of knowledge, contributing back at their own level, but not expecting any particular level of engagement with other learners.  The group of participants I am most worried about are those who wanted to find a community, but didn’t. They’d write blog posts and get frustrated that no one responded, or attempt to engage with fellow MOOCers through commenting on blog posts, but get no reply. After a few failed attempts to interact, these participants gave up trying: the weak ties of the MOOC were too weak At the extreme, were those who expected the organisers to facilitate far more actively*, and those who didn’t contribute themselves (through blog posts, comments, or even tweets) but who still expected to benefit from the contributions of others.

What was the difference between those who found and didn’t find networks?

That’s complex. As the name might suggest, our study hypothesised that an individual’s ability to self-regulate their learning might impact their participation in a MOOC. We found that while ability to self-regulate is a factor, a number of other factors are also at play:

  • previous experience of MOOCs: people learn how to learn in MOOCs. We certainly saw a ‘type’ of participant who knew what to expect from the course, knew what they wanted from the course,and knew how to make their participation a success. Given its size, Change11 probably wasn’t a good ‘first mooc’ for people to experience.
  • pre-existing networks: one key element that people who had taken previous MOOCs brought to Change 11 was their pre-existing networks. When these people blogged, they already had an audience for their views, because they were part of a network that had been developed through previous courses. These networks though weak (you might never have met the people who read you blog, and don’t communicate with them at all regularly), are resilient: you’ll read blog posts from people in your twitter network, because that content has already been ‘filtered’ for you – your network is trusted. There’s another interesting observation here: that your network doesn’t necessarily have to consist of people who are studying on your course. We saw different patterns of engagement from respondents who didn’t actively try to set up new networks, but let them grow organically through the course, as well as those who were focused on creating an internal network of fellow participants.Both approaches were successful.
  • expectation and motivation: even among those who had never studied in a MOOC before, there were those who knew what to expect, and who could self-motivate and engineer learning networks. This is partly to do with technology, and partly learner disposition. I suppose the great unanswered question of our study (we still hope to answer it) is the nature of the inter-relationship between between a learner’s ability to self-regulate (planning, self-motivating, managing and reflecting learning) and digital literacies (being able to leverage digital tools and network to support ones learning).

Going back to MOOC design: how can MOOC designers create environments to accommodate the diversity (in background, motivation, skills, expectations) of learners who participate in these massive open courses?

So how could you achieve this?

I think the cMOOC concept and philosophy is great, but my observation from a number of MOOCs is that by definition, Massive courses bring in learners with a range of backgrounds, previous experience and skill levels, and it is therefore incumbent on the organisers to design a learning experience that accommodates these diverse learner profiles. I think this is particularly critical at the start of the course (in fact I would say that if you get the start right, then the cMOOC model should work once initial networks have established). The start of a MOOC is a big scary place, and providing some hooks for participants to hold onto might be all that is necessary. Here are a few suggestions.

  • cater for different interest groups: even in our relatively small sample (it was primarily a qualitative study, and we interviewed 29 participants) we saw strong evidence of people looking for people like themselves. This was particularly the case with different types of educators: on the whole, the HE participants tended to have the loudest voices (more used to blogging etc), and we saw evidence of k-12 educators becoming disillusioned with what they perceived as ‘noise’ on the network – they couldn’t find peple to identofy with, because their voices were drowned out by ‘confident’ participants form different domains. Creating spaces where people can find others they can identify with would be a really simple step – it might only be a set of hashtags: #change11-k12, #change11-workplace etc … but it might make all the difference in helping people find other’s who speak their language – reducing the initial complexity of the MOOC space. Of course you need to guard against homophily – where the great benefit of learning in a Massive Course is not realised because everyone is talking to people who are just like them – it is important to have cross-fertilisation of ideas from k-12 to higher education and vice versa, but this can come later in the MOOC, once people have found their footing.
  • goals: going one step further, finding others who have the same expectations of the course as yourself is key to continued motivation. Some of our other work has explored the possibility of using shared goals as a mechanism bringing learners together and fostering peer-learning and peer-support. Although some of our study participants expressed some resistance to defining goals, there were clear goal types and patterns evident and these could be used to seed self-organising communities. Getting people to articulate their goals is key to allowing them to find each other.
  • orientation:  Clearly, many people participating in MOOCs still need to learn how to ‘participate in a MOOC’. The Change11 mooc did provide some orientation, and there are good resources out there (I’m thinking for instance of this excellent youtube video from Dave Cormier), but this wasn’t enough for some MOOC participants, and I think a different approach might be useful. Encouraging participants to seek out others with similar backgrounds or goals (as above) would be one way of doing this, another would be to engineer interaction by setting tasks which demand that participants contribute to the course and interact with others. While these tasks might feel a little artificial (akin to icebreakers at dinner parties) they are essential in helping participants realise the importance of connecting, creating and contributing, in addition to consuming, in a cMOOC.

Returning to Felicia Sullivan’s comment, she asks:

“How do well designed structure, processes and resources aid in self-organization and connectivity?”

Last week we had a colleague Hans de Zwart (Senior Innovation Advisor for Global HR Technologies at Shell) visiting the Caledonian Academy. Hans is interested in DIY Learning (including MOOCs) and one key principle he espouses is that we should put as much effort into designing ‘experiences’ as we do to designing content. Learning is so much more than the content, and it is vital that in MOOCs, organisers create an environment where learning can occur for all those who want to learn, not just for those who already have the skills and literacies at the outset.

I hope this answers the question (or at least takes the debate forward). I’m currently writing a paper on this aspect of our study, so writing up some of the themes and implications here was useful (cathartic!) for me.

* I think for Change 11 (though not necessarily for MOOCs in general) that there is something in this. By having different presenters each week, the course lacked coherence. A greater degree of facilitation by the organisers wouldn’t have gone amiss. 

nb: this was quickly written … and as a blog post rather than a journal article, so please forgive any looseness – particularly in my near interchangeable use of the terms community and networks.

Change 11 SRL-MOOC study: initial findings

Kelvingrove

As you will remember the Caledonian Academy conducted some research during the recent Change11 MOOC run by George Siemens, Stephen Downes and Dave Cormier.  The study generated a lot of data, which has been sitting on my desk for some months now. The hypothesis for the study was that we would observe different learning behaviours and different approaches to learning in MOOCs among those with different SRL profiles.

What did we do?

SRL Profiles: The first component of the study was to ask participants to complete an SRL profile instrument* we had developed for the study. The instrument was adapted from a number of pre-existing SRL self-report  instruments (full details, and a copy of the instrument are here), most notably the Motivated Strategies for Learning Questionnaire (Pintrich et al 1991) and a more recent Self directed Learning Orientation scale developed by Raemdonck (Gijbels et al , 2010). Within the instrument there are a number of sub-scales measuring specific SRL components (task analysis, goal setting, self-reflection etc. from the list of sub-processes described in Zimmermann, 2005). 35 respondents completed an SRL profile.

Interview: Everyone who completed an SRL profile was invited to participate in a semi-structured interview. The interview explored a number of factors, including motivation, previous experience of MOOCs, goal-setting, knowledge sharing, and participation patterns.  We conducted 29 interviews generating 26 hrs of interviews.

Emerging themes from the study

I plan to go into detail about many of the findings of the study through a series of posts,  but thought an initial post outlining some of the key things we turned up would be of interest. We identified a number of themes from the interviews, and I shall discuss four briefly here:

  • patterns of engagement: we saw different patterns of engagement. In addition to an expected cluster of lurkers who purposefully did not engage with other course participants, we identified two further groups: one group of passive participants, who expected ‘to be taught’, and viewed the course as a source of information, attempting to capture all the ideas being exchanged within the Change 11 community; and a final group, more active participants, who set their own goals, established connections with other learners and linked these connections with their existing personal learning network. These overall patterns of engagement are mediated by a number of factors including: previous participation in other MOOCs, motivation for participating in the MOOC, makeup of existing personal learning network, SRL profile, and confidence. Lurkers will be present in any online course, and the final group are archetypal cMOOC participants, but those more traditional learners in the middle group really wanted more guidance than they were given.
  • patterns of tool use: we have written before about what we call “the 4c’s“: consume, connect, create and contribute, as different learning behaviours that individuals use as they learn in networked environments. You can consume content, organise it by connecting content together and to people, create new content (knowledge) of your own, and contribute that new knowledge back into the community. We have collected data about the different tools in use and the types of behaviour they support. To quote one example, of resource management. We collected data about people using a range of tools to collect resources (bookmarks). While some respondents in our study were using their (single user) browser bookmark menus (consume), most were using some social service, such as delicious or diigo (connect).  But we also saw people who were interpreting/providing commentary on the content they were collecting using tools like scoop.it (create) and using such tools to ‘find their own voice’ in the community (contribute).
  • goal-setting behaviour: Goal setting is an important task in participation in MOOCs as, without a closely defined curriculum, it is incumbent on the learner to set and monitor their own goals. Most participants from the medium and high SR Scores groups set clearly defined goals, though we found no difference in the nature of the goals set with each group setting both participation and performance based goals. Those participants in the Low SR scores group were less likely to have set goals and were more likely to articulate goals categorised as ‘vague’ when prompted. Several participants in the study described how their goals had changed as they progressed through the course, showing evidence of reflection. We also uncovered evidence of other factors that might contribute to goal setting behaviour. Four participants in particular argued that the nature of this MOOC course (with a broad and only loosely defined curriculum) was incompatible with the idea of ‘goal-setting’ and had therefore explicitly chosen not to set goals.
  • motivation to participate: one problem with our study which we hadn’t anticipated (but perhaps should have) was that individual participants might have quite different (conflicting?) reasons for signing up. While some participants signed up for the content of the course, others (the majority) were primarily or exclusively interested in experiencing the Change 11 MOOC as a learning environment, often because they wanted to implement some of the features of a MOOC in their own practice. Several of the participants had previously participated in other MOOCs and when discussing their goals, these participants remarked that their goal –setting behaviour had been different in courses with a more defined topic (eg mobiMOOC and an ePortoflio MOOC, but others too).

What these results tell us about moocs

For me, the recurring theme from this research was that massive courses do need management (of learners, and their expectations), or at least a recognition of the diversity of learner backgrounds, preferences, expectations and motivations that come together in a MOOC, that is then reflected in the design of the learning space which is constructed. I suppose the prevalent (c) MOOC philosophy is that learners should be left to their own devices and they will find their place in the emerging learning networks(anywhere on the spectrum from lurking to leading). We certainly saw interesting evidence of self-organisation, especially among those who engaged with the course through the facebook group, and the twitter chats. But our findings indicate that some users either didn’t find these emerging networks (or at least didn’t identify a network that suited them), or didn’t recognise the central role that these networks play in leveraging the value of the course. While I don’t advocate creating rigid structures, I do think there are some simple things that could be done to make sure MOOCs such as Change11 are accessible by the full range of prospective participants.

Reflections on this study

I am still analysing the data from this study and we hope to publish the significant findings.  The specific nature of the Change 11 mooc (attracting many participants who wanted to ‘experience a mooc delivered by Downes, Siemens and Cormier’) complicates the analysis we can make about SRL behaviours as the different overall goals people held (primarily interested in ‘content’ or ‘process’) influenced their learning behaviour to such a great extent. Stay tuned for further blog posts after the holiday.

Bibliography

Gijbels, David; Raemdonck, I.; Vervecken, D. – Influencing work-related learning: the role of job characteristics and self-directed learning orientation in part-time vocational education – In: Vocations and learning:3(2010), p. 239-255.

Pintrich, P. R., Smith, D. A. F., Garcia, T., & McKeachie, W. J. (1991). A manual for the use of the Motivated Strategies for Learning Questionnaire (MSLQ). Ann Arbor: University of Michigan, National Center for Research to Improve Postsecondary Teaching and Learning.

Zimmerman, B. (2005). Attaining self-regulation: A social cognitive perspective. In Boekaerts, M.,Pintrich, P., & Zeidner, M. (Eds.), Handbook of Self-Regulation . San Diego: Academic Press.

—-

*While we didn’t validate this instrument, we are reasonably confident of its validity as it is based so closely on existing validated instruments and was adequate for our purposes (in a related ongoing project, we are actually going to validate an SRL instrument with around 400 people).

SRL MOOC Study Update and Background

Image

Many of you will remember that the Caledonian Academy planned to conduct some research on the Change11 MOOC which ran some months ago.  There is some background to the study here. It’s taken a while (there have been other priorities like our Mock REF and various papers) but we’re now making good progress with the data analysis. We always said we would be open with the results we produced and in preparation for a series of blog posts to be posted over the next few weeks, I thought I would put up a quick post today describing some of the details of the study.

The study (given Ethical approval by an internal GCU Research Ethics Committee) comprised two components. First, participants were asked to complete a survey which asked them to respond to a number of statements regarding how they learn and plan their learning. Second, participants were invited to take part in a semi-structured interview exploring their participation in the Change11 mooc and their learning practices within it.

  • A version of the SRL survey instrument is available on dropbox: here.
  • The interview script for the semi-structured interview is available on dropbox: here.

We managed to recruit 35 participants for the study, and conducted interviews with 27 of these. The sample seems robust – with representation from many countries, different education sectors (and beyond) and different motivation for participation. We have 26 hours of interviews (394 pages of text)

Posts over the coming weeks will concentrate on: the srl profiles, emerging themes from the interviews, what these results tell us about moocs, what we can say about srl within moocs and many more things. Please get in touch if you want to know anything about the study or its findings and I will try to include it in a forthcoming blog post. Of course we also plan to publish the studues once we have done more extensive analysis.

Mendeley and Rhizomes

If you look at my participation in Change11 you might notice that it tails off just right around the time that the word rhizome is mentioned for the first time. Is this a coincidence?

I’ve known of the concept of Rhizomatic Learning since Dave Cormier’s Innovate article back in 2008 but it was always on the periphery of our work at the Caledonian Academy. I think we are coming at the same issue (learning when knowledge changes so fast) from different directions: our work  is focused on supporting learners in the workplace (where learning is a  secondary activity to work), while Dave’s focus seemed more on improving the value of  formal education – preparing learners to better cope with the modern knowledge workplace.  [apologies for any over-simplification]. In the back of my mind I knew I had to get to grips with Rhizomatic learning, but it never got to the top of the pile. Dave’s change11 week should have provided a good opportunity, but it seemed to get a little bogged down by discussions on terminology. While I should have engaged with the topic, I found myself a little de-motivated by the discussions which were ongoing. My change11 participation never really recovered.

A few months ago, Martin Weller wrote a post where he questioned the practical value of  the rhizomatic concept in helping us as educators.  Martin helpfully used the words expert and novice in his post  and his clear and useful examples (e.g. blogging) helped me relate the ideas of Rhizomatic learning back clearly to our own work.  In the comments to that post I tried to defend the un-teachibility of rhizomatic learning by  focusing  instead on the way learning specialists can create an environment which supports learners in interacting with knowledge and harnessing the knowledge held in their learning network (and the places where new knowledge (the useful stuff) will be created) . The interest in Rhizomatic learning has – gladly – returned.

Anyway, Dave has now set up a Mendeley group about Rhizomatic Learning and is inviting people to pitch in to gather research articles around rhizomatic learning, with a view to developing a literature review on the topic. He has blogged about the motivations for taking this plunge here.  Undeterred by my previous experiences of the discourse surrounding this topic I decided to join the group and add a couple of papers which I have come across in the last few months.

The first: [Wood, M., and Ferlie, E., (2003) Journeying from Hippocrates with Bergson and Deleuze. Organization Studies 24 (1) 47-68. doi:10.1177/0170840603024001680 ] is concerned with knowledge exchange in healthcare settings and uses  Deleuze and Guattari’s notion of the ‘rhizome’ to explore how in contexts such as healthcare, discrete perspectives such as research and practice are necessarily broken down and there is continual exchange and interplay. while not explicitly about learning, the paper helps us think about the nature of knowledge flows in complex networks where different groups interact – clearly relevant to discussions of networked learning.

The second: [Fenwick, T., (2008) Understanding Relations of Individual-Collective Learning in Work: A Review of Research. Management Learning 39 (3) 227-243. doi:10.1177/1350507608090875 ] was actually the place I found the Wood and Ferlie paper. Whilst it says little beyond stating that Wood and Ferlie argue “for a ‘rhizomatic’ understanding of knowledge circulation within activity”, the review may be of real value to the Mendeley group as it shows where ‘Rhizomatic’ concepts might sit within our current understanding(s) of workplace learning.

Hopefully the Mendeley group will provide a place for some clear development of the concepts of Rhizomatic Learning. I’ll try and stay engaged this time.

Self Regulated Learning in MOOCs: study update

I thought it would be useful to give an update on the research study we are conducting alongside the change11 mooc. The study was intially announced last December, and is summarised as follows:

This study aims to surface, describe and systematise the activities and strategies that adult learners use to self-regulate their learning in the context of a massive open online distributed course, in particular Change 2011 MOOC.  The term self-regulation here refers to “self-generated thoughts, feelings and actions that are planned and cyclically adapted to the attainment of personal goals” (Zimmerman, 2005, p. 14). Our interest is specifically in professionals’ actions – practices and strategies that they use to plan and attain their learning goals.  The project will not examine the affective aspects of self-regulation (feelings) nor will it investigate cognitive processes (thoughts, self-beliefs) underpinning self-regulatory strategies.

The study design includes a survey (a modified SRL questionnaire) and semi-structured interview. We recruited a pool of 40 participants, of whom 34 were able to complete the SRL questionnaire. We’ve now collected all these responses and will analyse them shortly. These 34 participants have been contacted for interviews and our colleague Lou McGill  is currently conducting interviews by skype. Again, no analysis yet, but I am enjoying listening to the interviews and reading the transcripts.

We intend to write up the research, but hope also to share the findings through this blog, as well as those of the rest of the research team: Anoush Margaryan and Allison Littlejohn. Please make contact if you ant any more information.

Many thanks are due to everyone who has participated in the study, to Lou for conducting the interviews, Susan, our transcriber and to those who helped to pilot the SRL questionnaire, and the participant who acted as guinea pig for the first interview.

Research Study: Self-regulated learning in massive open online courses

Massive Open Online Courses are still very new and it is important to conduct research to try to understand how they support different types of learning. At the Caledonian Academy, we are interested in Self-regulated learning outside formal learning contexts and we have designed a study, which aims to surface, describe and systematise the activities and strategies that adult learners use to self-regulate their learning in the context of the Change 2011 massive open online course (MOOC). Our interest is specifically in professionals’ actions – practices and strategies that they use to plan and attain their learning goals.

We are looking for volunteers to participate in this study. Anyone who has signed up for the Change11 MOOC is welcome to participate. Participation in the study will involve completion of an online questionnaire (in January/February 2012) and participation in a telephone or Skype interview (in or around March 2012).

Data collected will be accessible only by the research team at Glasgow Caledonian University (Professor Allison Littlejohn, Dr Anoush Margaryan and Dr Colin Milligan). All data will be anonymised prior to publication and participants will not be identified (or identifiable). Participants are free to withdraw from the study at any time.

If you would like to participate in the study, please enter your name and email on the page linked here: https://www.surveymonkey.com/s/R2GK6W2. Alternatively, email me at colin.milligan@gcu.ac.uk, or tweet me @cdmilligan We will contact you in the new year with details about how to participate.

Further information on the research design: The study will examine how learners in Change 2011 MOOC plan, implement and reflect upon their learning goals, analysing similarities and differences in the use of SRL strategies between learners who are positioned on different points on the spectrum of SRL skills.  In identifying the SRL activities and strategies used by the participants, we are specifically interested in finding out how individuals draw upon available resources, such as other people and artefacts, to plan and attain their learning goals, and what tools do they use to do so. The study is guided by the following key research questions:

  1. How do participants plan, implement and reflect upon their learning goals within Change MOOC?What strategies do they use to self-regulate their learning?What tools do they use to self-regulate their learning?
  2. How do participants draw upon collective knowledge – people and other environmental resources – when planning, implementing and reflecting upon their learning goals within Change MOOC?
  3. What are the environmental factors, in particular those related to the coherence of the information space and structure of the MOOC, that constrain or enable SRL?
  4. What are the similarities and differences in the use of SRL strategies between learners who have diverse self-regulatory profiles? For example, do learners who score higher on self-regulatory skill measures use significantly different goal planning, implementation and reflection strategies than participants who score lower on the SRL measures?

Please sign up to participate in the study (here is the link again: https://www.surveymonkey.com/s/R2GK6W2). If you have comments or questions about the study, please email me at colin.milligan@gcu.ac.uk or leave a comment below.