Paper notes: Beheshitha et al 2015.

An interesting paper from Dragan Gašević & colleagues presented at LAK15. This quantitative study looked at SRL behaviours – complementary to our qualitative approach.

Beheshitha, S.S., Gašević, D., & Hatala, M. (2015) A process mining approach to linking the study of aptitude and event facets of self-regulated learning. Proceedings of the Fifth International Conference on Learning Analytics and Knowledge, 16-20 March, 2015, Poughkeepsie, NY, USA http://dx.doi.org/10.1145/2723576.2723628

Summary

SRL can be thought of as aptitudes (observed differences in individuals – commonly measured by self-report) or events (the actions learners execute). We can analyse trace data and associate srl events with sequences/combinations of actions in online environments then relate these to SRL aptitudes (self-report). RQ1 Can we identify groups of learners with different aptitudes (looked specifically at deep/surface learning, and goal orientation). RQ2 Do these groups behave differently? Design: Filled in questionnaires, then used nStudy (tasks monitored included: bookmark and organise resources, highlight and quote key points, take notes, define terms, write report. These were then classified as rehearsal, elaboration, and organisation). N=20. Cluster analysis allowed classification of learners as deep or surface, but was not able to identify different types of goal orientation. Comparing surface and deep learners, different patterns were seen – deep learners chose a strategy and stuck to it, and their strategies were more focused on elaboration. Surface learners used more organisation, and were more likely to adopt different strategies.

Reflection

Interesting complement to our PL-MOOC work, which also seeks to measure (self-reported) aptitude, then link it to SRL behaviours though in our case behaviours are also self-report and collected via interview (as opposed to mining of trace data). Very preliminary results, but shows promise as a method, and lit. review provides pointers to some interesting recent SRL research.

Follow up

Hadwin, A.F., Nesbit, J.C., Jamieson-Noel, D., Code, J. and Winne, P. H. (2007) Examining trace data to explore self-regulated learning,” Metacognition & Learning, 2, (2–3), 107–124.

Bannert, M. Reimann, P., and Sonnenberg, C. (2014) Process mining techniques for analysing patterns and strategies in students’ self-regulated learning,” Metacognition & Learning, 9, 161–185.

PL-MOOC study reflection

Image

 

We’ve now reached the end of the funding period for PL-MOOC and, while there is still some analysis to conduct and some papers and outputs to write, the final report is written and the main findings synthesised. Allison and I have tweeted about the recommendations and the main findings but I thought i would take an opportunity to reflect more broadly on the study over here.

Professional Learning
Our research interest at the Caledonian Academy is in professional learning – the learning for work that an individual undertakes to support their practice. We’re interested in the increasing responsibility that professional learners must take for their learning, monitoring their learning needs continually to maintain his or her expertise. Our focus is primarily on informal learning, and the learning networks that professionals maintain. We tend to ignore more formal learning activities but we understand that formal learning has a role to play and indeed how formal learning fits with organisational systems within which professional learners are operating. In her writing on ‘Integrative Pedagogy’, Tynjälä has described four different types of knowledge that make up professional practice – the formal theoretical knowledge you can get from books; the practical knowledge you get from doing the job, the socio-cultural knowledge you get from being part of a community of professionals, and finally the self-regulative knowledge you use to keep on top of the continual learning you have to do. A critical element of each professional’s self-regulated learning is to assimilate learning of all four types of knowledge through an ‘integrative pedagogy’

MOOCs
We’ve seen the rise in popularity of MOOCs over the past few years and thought a lot about how MOOCs might serve professional learners: they provide an excellent mechanism to bring learners with common interests together, they are flexible and open, providing opportunities for close integration with practice. But can they support professional learning. While our previous study ‘SRL-MOOC‘ looked at learning in a cMOOC (Change 11), for this study we focused on an xMOOC, the format favoured by the major mooc providers (EdX, Coursera, Futurelearn). We explored the learning behaviours of health professionals studying the ‘Fundamentals of Clinical Trials’ MOOC. The MOOC was designed by Harvard Medical School, Harvard School of Public Health, and Harvard Catalyst, The Harvard Clinical and Translational Science Center and offered through the edX initiative founded jointly between Harvard University and MIT. Our reason for selecting this MOOC was because a) the course was likely to attract a high number of participants working in the health domain, allowing us to examine how professionals learn and b) the course design was typical of the so-called ‘xMOOCs’.

Findings
Its probably best to read the report and formal summaries to get a full picture of the findings, but in a nutshell, we didn’t see much evidence of professional learning occurring in the Fundamentals of Clinical Trials MOOC. There was little in the way of integration of theory and practice that is essential for effective professional learning to occur. Activities not central to the course (discussing in the forum,or elsewhere) were given less priority than viewing videos and completing assessments. Evidence of impact on practice was limited. Now we should point out that the Fundamentals of clinical trials course doesn’t claim to offer professional learning, and the course participants overwhelmingly like the course. But somethings going on here. Health professionals signed up for the course … so there seems to be a demand for learning from the community. We asked professionals about their expectations/motivations and can demonstrate that these were articulated in terms of practice – meeting professional learning needs. Yet when we asked about goals (4 weeks into the course), these were articulated not in terms of practice, or even in terms of the learning content, but instead were articulated in terms of consuming content, passing assessments and completing the course. I think the design of the course shaped their expectations.

But if there is a demand for professional learning, can xMOOCs (Downes is right, we’re not talking about cMOOCs, which  would more naturally support it – though our study of change11 suggests that it is not so simple) be designed to support it. Building on the recommendations provides some pointers:

  1. MOOCs bring together learners with a great diversity of motivation, expectation, and prior knowledge and experience. Don’t expect everyone to start from the same point – nor indeed finish at the same point. Even if the course is content focused, you should allow your learners to personalise their learning to ensure that they are focusing their attention on what they need and want to learn. Personalising learning (and goals) also enables the learners to clearly link their learning to practice.
  2. Help your learners to find others to learn alongside – they might want to find people who speak the same language, or who work in similar contexts, or who have a similar level of experience. In our studies we have  learners expressing frustration at not finding peers (of each of these types and others) to learn alongside, often because they don’t have the confidence to look for them themselves – in a MOOC they are probably there.
  3. Capitalise on the real world experience that professionals bring to their learning by getting them to specify problems and provide solutions. Get learners to create content and contribute it back to the course as resources for other learners (now and in the future).
  4. As well as fostering networks within the course, remember the pre-existing networks that the learners already have outside the course. In our study of Change 11 we saw that some learners kept their internal and external networks separate – seeing Change 11 as a course, while others linked their networks, recognising that the course cohort was merely an extension of their existing network.

We’ve put our findings and ideas out there, but of course the study was by its nature limited and more work needs to be done, not least trying to put them in practice. We welcome responses and reactions to the recommendations we formulated, and hope others can take the instruments we developed and use and refine them.

I really enjoyed working on this study, the compressed timescale meant that we had to make decisions quickly, and work through challenges. Thanks to Allison for the many discussions we had throughout the last 9 months, to Obiagele Ukadike from Harvard for being our contact within the course team, and to Lou McGill for conducting the interviews.

This study was funded by the Bill and Melinda Gates Foundation as part of the MOOC Research initiative led by George Siemens. Thanks.

And of course a special thanks to the participants of the course for answering our survey, and agreeing to be interviewed.

References
For a readable summary of Integrative Pedagogy, try Tynjälä, P. & Gijbels, D. (2012) Changing World: Changing Pedagogy. In P. Tynjälä , Stenström, M-L and Saarnivaara, M. (eds.), Transitions and Transformations in Learning and Education. pp 205-222. Dordrecht, Springer.