Massive Peer Assessment is a Killer Application. Part One

In this blog post you wil learn: 

  • What is massive peer assessment
  • Its implications for student feedback

Imagine  a university professor, sitting by himself in  a cozy  café. In front of him lay three dozen student projects. He looks at them and sighs. They will take him all weekend to read, comment and grade.

It is hard work, but someone has to do it.

Imagine a different situation now. What if the teacher were not faced with three dozen projects, but  three hundred ?

How about three thousand?


Enter Coursera HCI Mooc

This was the problem faced by the staff at Coursera and Stanford  during 2013’s Human Computer Interaction Mooc. They had to grade and provide feedback to homework submissions from over 3,000 students.

They solved it by outsourcing the classmates assesment to other classmates. Every student submitted a homework and  received specific instructions and a short training on ‘how to evaluate a peer’s  homework’. If they succeeded the training,  they were  trusted with grading five of their peer’s papers.

How reliable was it? 

Some papers were also evaluated by staff at Stanford, so as to provide some ‘ground truth’At the end of the assessment the absolute difference between peer and staff grades was 3%. Under- and over-grading sort of balanced out.

Not perfect, but still amazing.for the first time peer assessment had been used a such a massive scale.

Ever since,  calibration techniques have been devised to improve graders’ accuracy. Massive Peer Assessment (MPA from now on) is now used in hundreds of Moocs in several platforms.

Call it ‘crowdsourcing of feedback and grading’

Call it ‘assessment by the students, for the students’

I call it a killer application. 

To see why this is the case, let’s fist talk about student feedback.

Student Feedback

One of the most interesting aspects of the technology is its implication on student feedback. It  has the potential to make it more timely. And a lot, a lot more abundant. These are exciting news for several reasons:

  • First, feedback is precious.     In the now  classic 2-Sigma paper ,  Benjamin B. Bloom observed that one-on-one -tutoring using  effective techniques can help any student raise to the top 2% of her class. Yes, any student. The availability of immediate, personalized feedback is at the core of the value added  by a tutor, and can indeed make or break a student.
  • Respective to motivation, feedback ist crucial. In  Flow Theory, Mihaly Csikszentmihalyi established that immediate feedback is  a crucial element for a person to concentrate fully on a task , a state of enjoyment also called being  in the zone.   There is no better state of mind for learning  than being in the zone. 
  • On the flipside, lack of feedback is dangerous. Research has found that ignoring a person’s efforts is almost as bad for motivation as destroying their work.


Massive Peer Assessment and student feedback.

Indeed, I believe most learning today is largely  feedbackless task. Classrooms and training needs are too large.  Incredibly, not even after major educational milestones like final exams do students get substantial feedback. Whatever the context,  feedback for students is extraordinarily scarce.

I believe there are  very juicy, low hanging fruits in terms of student achievement and motivation if we get more timely and abundant feedback. Massive peer assessment has now made those fruits accessible.

I believe MAP can allow educators to leverage abundant, quality feedback into the creation of  learning experiences  that approach Bloom’s ‘One-on-one tutoring’ in terms of achievement and that are engineered in a manner that students can really enjoy and get in the zone.


But that  is not all. Oh no, that is not all.


MAP might be precious for the student receiving it, but the same is true for the students generating it.


Indeed, assessing someone else’s work and explaining things to others are first-class learning experiences. Massive Peer assessment democratizes such experiences.

Coming soon:

Part 2: Why assessing peers is a  world-class learning experience  


Read more about Coursera and Stanford’s use of MPA here

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s