I tried out CSU English Success’s Calibrated Peer Review (CPR) a bit. The report isn’t positive, but I hope the system will eventually improve. For now, pen and paper are better than using computers here.
The amount of time it would take to explain the CPR system to the students is much greater than the educational benefit the system would provide. Dan wrote a good posting about this ratio. Think about it. I’m trying to figure out how to explain this system very quickly to my students. If I can do that, I’ll actually use it in an effort to help improve the system and learn to work within the constraints.
The Process
- Students write their essay in response to the prompt.
- Students read three “calibration” essays, with properly gaging student understanding of the rubric as the goal.
- Students move on to peer review. Since students have been calibrated, they ideally leave appropriate marks on three peer essays.
- Now that students have assessed six essays, they finally evaluate their own.
Troubles
While writing their response to the prompt, students have to type P tags at the beginning of each paragraph or else their response will run together as one paragraph. There’s surely a simple fix (probably something with nl2br), yet adds a burden to students who should focus on their ability to write, not their ability to properly render HTML tags. The essay is enough to occupy 100% of their intellectual energy.
Students need to calibrate within a certain margin of error from the correct marks. If a student’s calibrations do not match the correct answers, the student is put back through that loop twice. Not a bad idea and I’m glad that’s not an infinite loop. A better solution here might be to give the student an explanation of how close they came to expectations after the first time through the calibration.
No troubles that I can see with the peer review phase of this. In fact, this is a nice system for gathering this kind of information. Students can see how their peers evaluated their writing.
The rubric has six rows, so each of the first six questions of the evaluation are yes/no questions about each of those rows. By its very nature, a rubric imply shades of gray, making yes/no questions the wrong ones to ask. “Does the response have an effective, fluent style marked by a clear command of language?” If a paper isn’t very strong in this area, but then again isn’t entirely lacking, what’s the right answer, yes or no?
The seventh question of each evaluation is “Does this essay earn a 4 or higher on the rubric?” A fair question, but that question includes directions for how to answer the eighth question. Why are there directions for how to answer the next question here? Why aren’t those directions, I dunno, on question eight?
To make matters worse, those directions could be completely unnecessary. Question eight is “how would you rate this text?” The rubric uses a six-point system so you’d think students would just have to select a score of one through six, right? Sadly, question eight offers scores of one through ten. The students are just told not to mark a score higher than a six. Remember that they were told this key info back on question seven.
Benefits
This system is all online, allowing students to complete this work whenever they have time. Personally, I want discussion about the samples so students really understand why they receive the scores they do, but it’s not a bad thing for students to read these essays outside of class.
A big thing this system allows for is students walking through the peer review period at their own pace. Teachers are secure in the knowledge that there’s another paper waiting for the student to review because it’s automated. It saves the teacher from having to manage who gets what paper and what to do with the students who finish earlier than others, sitting without a paper to read.
Another big thing is that this system frees up class time. If the different steps of this process were required to be completed by certain dates outside of class, class time is available to cover other matters and students are still expected to think critically about their writing.
That Said…
If more development happens I might change my mind, but I’m voting “no” on this one right now. Save time with pen and paper. If there was a way to bypass the calibration phase, assuming that this would happen in the classroom, I’d use it more readily.