PL-MOOC study reflection

Image

 

We’ve now reached the end of the funding period for PL-MOOC and, while there is still some analysis to conduct and some papers and outputs to write, the final report is written and the main findings synthesised. Allison and I have tweeted about the recommendations and the main findings but I thought i would take an opportunity to reflect more broadly on the study over here.

Professional Learning
Our research interest at the Caledonian Academy is in professional learning – the learning for work that an individual undertakes to support their practice. We’re interested in the increasing responsibility that professional learners must take for their learning, monitoring their learning needs continually to maintain his or her expertise. Our focus is primarily on informal learning, and the learning networks that professionals maintain. We tend to ignore more formal learning activities but we understand that formal learning has a role to play and indeed how formal learning fits with organisational systems within which professional learners are operating. In her writing on ‘Integrative Pedagogy’, Tynjälä has described four different types of knowledge that make up professional practice – the formal theoretical knowledge you can get from books; the practical knowledge you get from doing the job, the socio-cultural knowledge you get from being part of a community of professionals, and finally the self-regulative knowledge you use to keep on top of the continual learning you have to do. A critical element of each professional’s self-regulated learning is to assimilate learning of all four types of knowledge through an ‘integrative pedagogy’

MOOCs
We’ve seen the rise in popularity of MOOCs over the past few years and thought a lot about how MOOCs might serve professional learners: they provide an excellent mechanism to bring learners with common interests together, they are flexible and open, providing opportunities for close integration with practice. But can they support professional learning. While our previous study ‘SRL-MOOC‘ looked at learning in a cMOOC (Change 11), for this study we focused on an xMOOC, the format favoured by the major mooc providers (EdX, Coursera, Futurelearn). We explored the learning behaviours of health professionals studying the ‘Fundamentals of Clinical Trials’ MOOC. The MOOC was designed by Harvard Medical School, Harvard School of Public Health, and Harvard Catalyst, The Harvard Clinical and Translational Science Center and offered through the edX initiative founded jointly between Harvard University and MIT. Our reason for selecting this MOOC was because a) the course was likely to attract a high number of participants working in the health domain, allowing us to examine how professionals learn and b) the course design was typical of the so-called ‘xMOOCs’.

Findings
Its probably best to read the report and formal summaries to get a full picture of the findings, but in a nutshell, we didn’t see much evidence of professional learning occurring in the Fundamentals of Clinical Trials MOOC. There was little in the way of integration of theory and practice that is essential for effective professional learning to occur. Activities not central to the course (discussing in the forum,or elsewhere) were given less priority than viewing videos and completing assessments. Evidence of impact on practice was limited. Now we should point out that the Fundamentals of clinical trials course doesn’t claim to offer professional learning, and the course participants overwhelmingly like the course. But somethings going on here. Health professionals signed up for the course … so there seems to be a demand for learning from the community. We asked professionals about their expectations/motivations and can demonstrate that these were articulated in terms of practice – meeting professional learning needs. Yet when we asked about goals (4 weeks into the course), these were articulated not in terms of practice, or even in terms of the learning content, but instead were articulated in terms of consuming content, passing assessments and completing the course. I think the design of the course shaped their expectations.

But if there is a demand for professional learning, can xMOOCs (Downes is right, we’re not talking about cMOOCs, which  would more naturally support it – though our study of change11 suggests that it is not so simple) be designed to support it. Building on the recommendations provides some pointers:

  1. MOOCs bring together learners with a great diversity of motivation, expectation, and prior knowledge and experience. Don’t expect everyone to start from the same point – nor indeed finish at the same point. Even if the course is content focused, you should allow your learners to personalise their learning to ensure that they are focusing their attention on what they need and want to learn. Personalising learning (and goals) also enables the learners to clearly link their learning to practice.
  2. Help your learners to find others to learn alongside – they might want to find people who speak the same language, or who work in similar contexts, or who have a similar level of experience. In our studies we have  learners expressing frustration at not finding peers (of each of these types and others) to learn alongside, often because they don’t have the confidence to look for them themselves – in a MOOC they are probably there.
  3. Capitalise on the real world experience that professionals bring to their learning by getting them to specify problems and provide solutions. Get learners to create content and contribute it back to the course as resources for other learners (now and in the future).
  4. As well as fostering networks within the course, remember the pre-existing networks that the learners already have outside the course. In our study of Change 11 we saw that some learners kept their internal and external networks separate – seeing Change 11 as a course, while others linked their networks, recognising that the course cohort was merely an extension of their existing network.

We’ve put our findings and ideas out there, but of course the study was by its nature limited and more work needs to be done, not least trying to put them in practice. We welcome responses and reactions to the recommendations we formulated, and hope others can take the instruments we developed and use and refine them.

I really enjoyed working on this study, the compressed timescale meant that we had to make decisions quickly, and work through challenges. Thanks to Allison for the many discussions we had throughout the last 9 months, to Obiagele Ukadike from Harvard for being our contact within the course team, and to Lou McGill for conducting the interviews.

This study was funded by the Bill and Melinda Gates Foundation as part of the MOOC Research initiative led by George Siemens. Thanks.

And of course a special thanks to the participants of the course for answering our survey, and agreeing to be interviewed.

References
For a readable summary of Integrative Pedagogy, try Tynjälä, P. & Gijbels, D. (2012) Changing World: Changing Pedagogy. In P. Tynjälä , Stenström, M-L and Saarnivaara, M. (eds.), Transitions and Transformations in Learning and Education. pp 205-222. Dordrecht, Springer.

 

Tagged with: , , ,
Posted in mooc, Uncategorized

MOOC Research Conference

2013-08-28 17.39.35

Back a few days from mri13 and now that the experience has settled in somewhat I thought I would try and reflect on the conference and what I learned about the current mooc landscape and its impact on our research here at the Caledonian Academy.

First off, thanks to George Siemens and Co. for bringing together a diverse group of people with key knowledge and perspectives and no shortage of ideas and opinion. I thought the conference was about perfect size, though the attendees were sometimes a little thinly spread due to the many parallel sessions. This also meant I only saw a minority of presentations from other MRI grantees (even though I prioritised these over presentations which might have had more general appeal to me), so I need to go back to the abstracts and hope I can find some online presentations for the ones I missed (ours [Professional Learning through Massive Open Online Coursesis here). 

The conference started off on a high with a charismatic performance from Jim Groom describing his work supporting learners as creators at UMW.
In the CETIS PLE project (Milligan et al, 2006, Wilson et al, 2007) we described a utopia of learner-centred learning in a mature online landscape but my own experience at institutional level has taught me how difficult this is to achieve at scale (is it a coincidence that my work has moved into more informal contexts – professional and workplace learning). Jim’s work has shown that learner-centred learning can be supported within formal settings. However, opening the conference with a presentation showing what learning could be like backfired a little for me because over the next few days I saw too much evidence of learning taking second place to other considerations – content, delivery, administration, data. At points I was left wondering whether the legacy of the MOOC bubble of the last few years will be a million hours of talking heads video lectures.

In some presentations I was left wondering whether the whole canon of literature on online and distance learning has been cast aside. A key element of our project has been to try and understand the MOOC we are studying as an online learning event (before then going on to explore the learning occurring within it). Having a robust theoretical base is essential for us as we go forward to design tools and recommendations intended to inform the design of future MOOCs. Another disappointment was that some of the presentations seemed a little dumbed-down. In one presentation, we got the ‘MOOC: every letter is negotiable‘ line, while one presenter asked us to think when a cMOOC would be best and when an xMOOC was appropriate (like I’m not allowed to consider a million other ways of delivering a course online). Surely a MOOC research conference can support  a base level of dialogue beyond this.

Having complained (I did enjoy the conference), I should say that I saw some really thought-provoking presentations that I will follow up on (see abstracts for the MRI grantees here - sorry i couldn’t find presentation links).

  • Bruno Poellhuber described an MRI study based across a number of Canadian Universities using SRL to explore motivation and engagement in MOOCs. Over the years, our work has been focused increasingly on motivation and I think it is a key issue in MOOC research, and one which has to be addressed by any MOOC provider or designer.
  • Rebecca Eynon and Nabeel Gillani from the University of Oxford presented the most interesting SNA focused talk I attended. Part of their study focused on the vulnerability of networks, removing nodes to show how those networks dissolved or degraded. This is a mixed methods study, combining big data analysis with qualitative approaches (interviews) to really understand what is going on in the course they are studying.
  • Martin Weller presented data from two complementary studies. In the first, his colleague Katy Jordan had collected and analysed a large amount of data on MOOC participation. The emerging visualisations (some with large error bars) begin to highlight some key patterns. For example: it looks like you can now start to predict participation rates in MOOCs. [update: see Martin's blog post on Completion Data ] In the second, Martin showed how he had adapted course analysis approaches used at the OU in an attempt to describe the form of a range of MOOCs. [update: see Martin's blog post on Learning Design of MOOCs] Although this work is still in its early stages (I liked the fact that the findings presented by all the MRI grantees at the conference were ‘interim’) I think it shows great promise.
  • Although not a MRI grantee, I liked Shirley Alexander’s presentation describing the signature pedagogy (Learning 2014) being implemented at UTS in Australia. I suppose that brings us back to Jim Groom and learner centred learning. And away from MOOCs.

I’m looking forward to the MRI projects coming to fruition. I hope they make their research results easy to find online.

Personally, it was great to think and talk about research for an extended period of time (both at the conference and at the airport/on the flights at either end). It was also great to meet other researchers in the community and makes some personal connections. The main lesson I learned: I realised that while we don’t do ‘big data’ analysis at the Caledonian Academy, it shouldn’t stop us thinking of how we can try to complement our studies with data-driven studies: I think that in the long run, combined approaches (cf the Oxford study) are going to be the most informative.

References

Tagged with:
Posted in mooc, research

Productive MOOCs

OLYMPUS DIGITAL CAMERA

One of my frustrations with MOOCs is that while they provide value for the individual, their (lack of) collective value is a missed opportunity. Although I’ve had a blog post in the works for almost a year, I could never work out how to write it concisely. A recent post by Rachel Harris of Inspire Research  provided a slightly different angle to address the issue.

Rachel is evaluating the ALT ocTEL MOOC and one of her evaluation questions is ‘Can new knowledge be generated via critical discourse in a MOOC?’. With our other research interests here at the Caledonian Academy we are really interested in knowledge creation and collective knowledge. So,  in the comments I said:

I’m firmly of the belief that knowledge creation is a key component of learning, and successful MOOC learners will (through writing and responding to blog posts) create new knowledge and advance their thinking.

Of course I am talking about cMOOCs here (xMOOCs will have to wait for another day, but Dave Cormier has a related post from last weekend which explores similar issues). My comment continued with this observation (from participation in cMOOcs, and from researching Change11)

But one of my concerns in observing other MOOCs is that while each individual might advance their knowledge, there is little in the way of collective knowledge advancement. What this means in practice is that there is lot of duplicated knowledge produced during the MOOC.

Wouldn’t it be great if cMOOcs could be made more ‘productive’ – instead of advancing many people’s knowledge a little by re-creating the same (or similar) new knowledge again and again, can MOOCs be structured to stimulate the creation of new knowledge in a more coordinated way. Can you bring the learners together to produce something entirely novel as they learn? This is in the true spirit of connectivism.

Aside from my desire to see new knowledge created, there is actually an important learning observation here. As I went on to say in the comment on Rachel’s post:

 To the active participants, this can be frustrating as the consequence is that there is too much ‘noise’ in the system.

You might read a number of blog posts by your fellow participants, each making the same point. In the Change11 MOOC, some participants found this boring, others (particularly the less confident) decided not to articulate their own view because they thought it had already been presented and didn’t want to ‘just repeat’. But articulating what you have learned is important so this is a missed learning opportunity.

On a more positive note, the emergence of self-organised spaces in the Chage 11 MOOC (the facebook group, the One Change a Day group) showed that co-creation could succeed, but it was only a minority of participants who managed to engage in this coordinated way.

Solutions. I’m not sure I have any,  but here is one idea**: Can MOOCs be used to encourage interdisciplinarity? An interdisciplinary*MOOC could bring together experts from different fields to explore a topic from different perspectives and adopting each other’s methods etc. The ultimate product could be the new knowledge created as a result, perhaps a series of working papers. The immediate flaw in this is that it doesn’t seem very massive (or at least the productive element might be the result just of the collaboration of the instructors), but if you can structure the MOOC participants as disciplinary ‘tribes’ then you can focus groups on creation of specific knowledge – e.g. initially discipline sub-groups could cooperate to co-create a disciplinary view, then be re-assorted into inter-disciplinary groups to explore the overlaps and tensions. oriented towards a more specific goal that matches their existing knowledge and motivation,  they can work together to co-create something that can be contributed back to the group. A structure such as this might also help counter the tension/imbalance between homophily and heterophily that I think is inherent in MOOCs.

* by interdisciplinarity, I mean ‘integrating knowledge and methods from different disciplines, using a real synthesis of approaches’ – taken from http://www.arj.no/2012/03/12/disciplinarities-2/ which in turn is from a paper by Stember (1990)

**I’ll dig out the notes for the original blog post and write more at a later date.

Tagged with:
Posted in change11, mooc

From consumption to curation – but where are we headed.

Image

We have previously described four learning behaviours which we feel describe the ways in which an individual engages with their personal and work learning environment/ personal learning network. See Littlejohn, Milligan and Margaryan, 2011 and Milligan, Littlejohn and Margaryan, 2012 for academic papers). We term the behaviours the 4c’s: consume, connect, create and contribute. We tend to argue that learning networks and personal learning environments facilitate connection creation and contribution, whereas traditional learning environments (and their associated pedagogies) were more focused on consumption.

Although we’ve settled on 4c’s, a fifth option: curation describes an important and emergent behaviour in social networks which has clear overlap with these behaviours. This blog post (long and hastly written) captures some of my current thinking about tools in this area.

In the good old days we saved bookmarks. To our own computers. Then along came del.icio.us allowing public sharing of bookmarks, and through tags a great mechanism for discovering trusted knowledge sources and gaining an insight into how other people in our networks conceptualised and structured knowledge. I was an avid user of delicious from its early days – and I think it is an incredibly powerful social tool for learners and for learning. But I hardly use delicious anymore …

I used to have a simple workflow – find things via Google Reader, and save everything useful to delicious, organising with different tags. I shared stuff using the special for:username tag to individuals in my close personal network. and I used to make notes about the resources as i saved them in delicious. This is incredibly powerful in learning terms … Going back to the 4c’s I consumed content, I connected it to my existing knowledge structures (by tagging) and to people in my network (using the for:username tag), I created new knowledge associated with the new resources I’d identified (appending notes) and making all this public through an open social tool represented my contribution back to my learning network, and beyond.

Sometime in the last two years or so, my delicious tagging started to slow considerably and my sharing fell to zero. Sorry network. The reason: the other tools in my personal learning environment evolved, and new ones arrived, while delicious (and Google Reader) stagnated. These new tools have different affordances, but bring some limitations.

In the last 7 or 8 years, the web (and how we interact with it) has moved on, not least with the advent mobile devices and apps. My simple discovery and share workflow has evolved – but not for the better. Now I tend to put things in silos – I save recipes to ‘Pocket’ because they are then easily accessible when I am cooking and have my tablet with me. I store research papers to Mendeley or Zotero where I can search, music videos go on tumblr, tips and tricks articles to Evernote. I’ve gradually migrated many of my newsfeeds from Google Reader to Flipboard where the reading (consumption) interface is much nicer. Managing /curating and consuming my content streams is infinitely easier than it was before. But there’s been a silent cost. In some of these new silos I have retained networks, but not in all. Specifically. FlipBoard and Pocket feel very personal to me and not social at all (they support consume, but not so much connect).

And what about create and contribute … I really like Pocket, and if I am honest I don’t really mind that it is not ‘social’ (in the short run, I don’t suffer) but one thing that does frustrate me is that I can’t annotate the things I keep there. I want to be able to write ‘halve the quantities’ or ‘use butter not margarine’ somewhere to remind me when I make a recipe again, but this online version of a recipe folder doesn’t give that option (it doesn’t support create). And with it not being social, there is little option for me to contribute. That’s a real shame.

Likewise with FlipBoard. Up until now there was no coherent sharing within FlipBoard (you can ‘tweet out’ single articles from the Flipboard app) So it was interesting to read today that FlipBoard is adding a social layer to its app. This immediately made me think – hey, this is a great way to share stuff I discover – I can curate a bundle of content I’ve found which is interesting to me and share it with anyone else who is interested (connect). But a quick look at lunchtime confirmed that their view of social still doesn’t extend to giving me a ‘voice’to annotate my curated selection – there is no facility for creation.

So, where are we? it seems that our social web spaces are increasingly driven by content consumption and curation … and while tools supporting these actions have evolved, this has been at the expense of functionality for creation and contribution. If we are to make use of social web tools for learning, we have to recognise that the functions of the tools we find may not match with the balance of functions we would like our learning tools to have.

If anyone knows of a good tool that gets the balance right then I’d love to know. Scoop.it seems to get creation right (allowing annotation/commenting on resources), but I’m not sure about the others – though in truth i haven’t yet used myself, only seen other’s scoops.

Bibliography
Littlejohn, A., Milligan, C., & Margaryan, A. (2012). Collective knowledge: Supporting self-regulated learning in the workplace. Journal of Workplace Learning, 24(3) 226-238.

Milligan, C., Margaryan, A., & Littlejohn, A. (2012). Supporting goal formation, sharing and learning of knowledge workers. In Ravenscroft, A. et al. (Eds.), Proceedings of European Conference on Technology-Enhanced Learning (EC-TEL), LNCS 7563 (pp. 519—524). Heidelberg: Springer. Postprint (under green archiving arrangements):https://dl.dropbox.com/u/6017514/milliganetalLNCS_archive2012.pdf

Tagged with:
Posted in charting, research

Lurkers, lurking and labels

Conic Hill from Drymen primary school

Our SRL-MOOC study has been getting some attention thanks to a post by Phil Hill which built on the categories I had assigned to learners in Change11 that were emerging from our data analysis. There’s been some discussion going back and forth in the comments (where I think it ought to be), but a comment from Debbie Morrison last night on my use of the term ‘lurkers’ might benefit from a more extensive discussion, so I have written this post.

Debbie wrote:

I realized from reading your comment your choice of the term of lurker was not in a negative context – however the term does have a negative connotation associated with it – to ‘lurk’ according to the Webster’s dictionary, ” to lie in wait in a place of concealment especially for an evil purpose”, though I realize a modern definition has been added recently, “to read messages on an Internet discussion forum (as a newsgroup or chat room) without contributing”.

And it’s this second definition here that is critical. The term ‘lurker’ is already used in the academic literature. Discussing lurking in traditional online courses, Rovai (2000) describes lurkers as “… learners who are bystanders to course discussions, lack commitment to the community, and receive benefits without giving anything back”. This is exactly what we observed, so while I thought about whether to adopt a different term (I played around with lurking, but found it unworkable).  I eventually concluded that Rovai’s definition described what we were observing.

Connectivist MOOCs (such as Change11) must accommodate learners of all types to satisfy Downes’ (2009) ‘Diversity’ and ‘Openness’ criteria. In practice, as long as there is a balance of these different types of learner, then lurkers can be accommodated, and the evidence from this study is that lurkers can learn effectively in connectivist environments: taking the knowledge they acquire to their external networks.

In contrast, in xMOOCs, where structures of peer-grading are put in place, or people are placed in small groups, then lurkers can be disruptive if they don’t participate in the designed learning environment.

Downes, S (2009) Connectivist dynamics in communities. http://halfanhour.blogspot.co.uk/2009/02/connectivist-dynamics-in-communities.html  Available online February 26 2013. [changed - had previously posted the wrong Downes ref]

Rovai, A. P. (2000). Building and sustaining community in asynchronous learning networks. The Internet and Higher Education, 3(4), 285-297.

Posted in mooc

LCL: The maps of My Childhood

Image

The activity for session two of the LCL course is to read the forward to Seymour Papert’s Mindstorms (1980) entitled The Gears of My Childhood and ‘write about an object from your childhood that interested and influenced you‘.

My object is an Atlas. I have been fascinated by maps for as long as I can remember and though I didn’t follow a career path focused on geography, or become an explorer, that early and constant love of maps has provided a framework on which I have hung so much other knowledge in my life.

The atlas was only the centrepiece of an obsession with maps and flags which extended to jigsaws of maps, road atlases, the Ordnance Survey (see the picture of my wall in my old flat above) and the maps form the National Geographic. As a young child (of 7 or 8, see later in the post why I can be sure) I used to take the Atlas, and a table lamp and trace outlines of countries and continents on a glass coffee table we had. I’d then colour in each country with the corresponding flag. Different continents lent themselves to this activity to different degrees: Europe was a bit rubbish, as all the countries were small; Africa much more pleasing with lots of big countries; North America too boring with only 3 countries and South America good, but with the nearly insurmountable problem of fitting the flag of Chile.

This is a nice innocent activity, a good way for a not very artistic child to be creative. But the real value of the activity was that the constant repetition (I must have done each continent dozens of times in my pursuit of perfection) cemented in the positions and names of countries (along with lots of incidental detail) along with their flags. Having this knowledge of maps helped me begin to understand world events – I’ve got a strong memory of following the Angolan Civil War (1975 – so when I was 7 or 8), and recognising the Communist/Marxist imagery (a cog representing workers, the sickle representing the land, etc). These symbols, repeated across Africa in the flags of the other countries such as Mozambique and Zaire, gave me an awareness of the role which armed struggle had played in these countries’ emergence from colonialism (and that while much of the map was pink – British empire – large swathes of Africa had been Portuguese or Spanish. I also saw patterns in the use of Red, Yellow and Green in the swathe of countries further north, and how these were similar to those of many nations in the West Indies and surrounding the Caribbean  and how these in turn gave way to a different palette and set of symbols in the Muslim countries of North Africa and the Middle East.

From the combination of maps and flags I got a framework to hang new knowledge on but also a ‘way in’ to the political and historical changes that had occured not just in Africa but across the whole world – without ever reading a formal textbook or history book.

Tagged with: , ,
Posted in medialabcourse

LCL2: Interest-based Learning

moon_llnp

Writing deadlines mean I am a week (or so) behind in the LCL course, but I do still hope to keep up. Session 2 was about Interest-based learning and was based on the following readings:

We were asked to structure our reflections based on the following two questions:

  • What did you find most interesting or surprising in the readings? I liked all the readings for this week, and was particularly enthused by the very readable Papert piece (I’ll save my comments on that for a separate piece on ‘the gears of my childhood‘. My research interests are informal learning, so although I didn’t know the work I know the ideas, so it is interesting to reflect on the generalisability of the underlying concepts. For example, I’m struck by how the readings this week either explicitly or implicitly emphasise ‘affect’ as a key factor. In Joi Ito’s formal vs informal blog post, he asks: ‘is there a way to support and acknowledge the importance of informal learning …?’ I think the answer is yes (I’m thinking about Higher education contexts here). One way to motivate learners is to make learning meaningful for them – so rather than imposing a curriculum, try and negotiate it (as far as your quality procedures allow) and certainly personalise it – allow each individual student to own their own learning. That’s easy to say, but what does it mean in practice? Well, you could start by helping the students think about why they are doing the course, what they want out of it, how it will contribute towards their overall goal. Then, don’t just produce content for your students to consume – get them to contribute ideas, create new knowledge, make connections. Design assessments that allow them to demonstrate what they have learned (how they have changed during the course) and what it means to them in terms of their own learning and goals. This might be more work in terms of assessment, but you will have motivated your students and given them a learning experience that is valuable beyond exam time. I also liked the ideas in the Dubai post: unpredictability drives learning – it makes it easier to identify and articulate gaps in your knowledge and therefore to start planning to address them.
  • What did you disagree with or have questions about? In Papert’s ‘Gears of My Childhood’ piece, I loved the overall idea, but I was a bit concerned that he had overstepped the mark when talking about body knowledge, and the sensorimotor schemata of the child. I think I’d have to read the rest of the book to find out, but although it worked for his ‘gears’ it wouldn’t work for the thing that spurred my interest-based learning … maps.
Posted in medialabcourse
Follow

Get every new post delivered to your Inbox.