Ongoing quest for research instrument to log online learning
There is no license.

Methodology Design Narrative Template

(Information about the template at


  1. Situation


 As a connectivist MOOC organiser of the past (MobiMOOC, a MOOC on mobile learning), and now a PhD student driven MOOC researcher, I am investigating how experienced online learners learn in a MOOC (use of technology, in- and outside MOOC content searches, indivivdual and collaborative learning...). The MOOC environment from which I gather research participants is FutureLearn. During my pilot study I investigated what influences MOOC learning in general (daily life, unforeseen circumstances, professional demands) in order to get an idea of all the influences and see what would be the overlap with other learning, and more importantly what could be considered as 'new MOOC related learning'. The results from the pilot study showed that more experienced learners tend to have different skills and abilities that they deploy to engage in a MOOC. There also seems to be a 'stretch', a personal flexibility that each learner is willing to go to get answers to questions they pose themselves during their engagement in a MOOC (eg. could peer X help me to find a solution for a real case challenge I have related to the MOOC topic?). 

These findings and past experiences have pushed me to investigate expert learners and how they self-determine their learning in a MOOC. 


  1. Task

 In order to map the learning actions that experienced online learners take during their MOOC learning journey, I needed to find a research instrument that would feel intuitive to the MOOC environment, and would be easy to use by the MOOC participant. The research instrument needed to enable self-reported logging to enable learners to share what they did in terms of learning action, where they surfed to, who they contacted... This means that a lot of MOOC learning exists outside of the MOOC environment, and could be described as informal learning. 

Up till now I have not found a succesfull solution to this task, but I do want to share the journey up until now, with its failures and renewed challenges. 



  1. Actions (How did you try to address the issue?)


Find online tools that would enable self-reported learning actions. 

Activity goal(s) or intended outcome (“X”)

 Finding an instrument or method that will enable me to see what MOOC participants do in terms of learning, get a deeper understanding of how fare they are willing to go to find solutions for learning challenges they feel they are faced with. In order to be able to do that, I need to have an instrument that will detect and trace informal learning. 



Activity materials

Using self-reported learning logs: delivered to the MOOC participants with a clear demand to fill them in during the MOOC, and return them after the MOOC was finished (at the latest). These learning logs were provided as online questionaires, word documents, paper documents. 

Trying out existing tools: Pearltrees, bookmarking tools.

Trying out new tools that are in development: xAPI and learning lockers. 


The “how” (steps A, B, C, that lead to Y or the activity outcomes)


Step 1:  In my pilot study I used self-reporting learning logs. The MOOC participants did fill them in, but a lot of them complaint that it took too much time on top of their existing MOOC tasks (which they indicated were already on top of their daily life's). This feedback pushed me into searching new ways to ask people to track their learning. 

Step 2: I looked for easy to use tools that could be used as self-reporting instruments: bookmarking tools seemed logical, as they would enable informal sides to be bookmarked with additional open descriptions on why this was bookmarked or used. Another option was Pearltrees: easy to use, fairly similar to bookmarks, but allowing a more connected learning track to be visualized. However, these tools demanded MOOC participants to add yet another tool to their set of MOOC tools. This was not ideal as it was not part of the MOOC environment, or their 'native tool set' for some.

Step 3: get in contact with xAPI developers, and Learning locker developers.  A combination of xAPI and learning locker provided a self-reporting system for informal learning. This looked promising. But the learning locker developer was not very willing to collaborate in any form to build a research instrument for the purpose I described. Unfortunately he refused to get back on several occasions, although his xAPI colleague urged me to keep on trying. Step 3 failed due to personal unwillingness, which could have been for the best reasons, but was never made explicit.

Step 4: changing the learning log approach, frequency, and time. After going through the forementioned steps I returned to the first option. As feedback pointed out that keeping learning logs for the whole duration of the MOOC was too time consuming for MOOC participants, I searched for a solution that would only ask participants to share their learning actions for short moments: those moments would be restricted to 3 days of self-reported learning, then two weeks later 2 days of self-reported learning.  

    The self-reported learning was also made easier by providing online checklists that shared learning actions as they were described by MOOC participants during the pilot study. This way the MOOC participant can quickly indicate what type of learning action it is. 

Activity outcomes (“Y”) (positive)






  1. Reflection

 As MOOCs and MOOC research are still evolving, the tools and understanding of how to understand and grasp MOOC learning are lagging behind. On the other hand, it is important to understand how experienced online learners add to their learning while being engaged in a MOOC, as this can enhance MOOC roll out and design. 

As such researching MOOCs demands quick iterations in terms of research instruments, approaches and understandings, and patience from all parties. 



Author(s): Ignatia_dW
Published at: 08 Jul 2014 10:10 GMT
Original link (login required):