Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.


Peer Assessment with Calibration - UNSW Enhanced Workshop Activity in Moodle

This Prezi will help students get a better understanding of what to expect when participating in a UNSW Enhanced Workshop activity in Moodle.

John Paul Posada

on 5 May 2016

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Peer Assessment with Calibration - UNSW Enhanced Workshop Activity in Moodle

Submission Phase
Peer Assessment with Calibration
UNSW Enhanced Workshop Activity in Moodle
(51 × 6 + 67 × 18 + 59 × 8 + 78 × 12)/(6 + 18 + 8 + 12) = 66.3636363636

Setup Phase
Calibration Phase
Assessment form
Assessment Phase
Grading evaluation Phase
Instructor view only
Closed Phase

Thanks for your time.
I hope this presentation was useful.
John Paul Posada
Educational Technologist
University of New South Wales
The UNSW Enhanced Workshop activity allows you to assess and be assessed by your peers. If selected you’ll also be calibrated as a marker by marking sample submissions and assessed on how good a marker you are based on how close you come to the experts assessment of that same submission.
The UNSW Enhanced Workshop activity functions in phases. It is very important to pay close attention to any timings and deadlines associated to these phases. The current phase of the activity is highlighted while the other phases are greyed out.
The initial setup phase will not require any action on your part. It should provide you with some details about the particular activity and possibly information, or links to information about the assignment.
The three most important actionable phases of a workshop activity will be:

The submission phase
The calibration phase (if used)
The assessment phase

We’ll review these in more detail throughout this prezi.
The submission phase is the first phase of this sample UNSW Enhanced Workshop activity.
Here you should be presented with detailed instructions of what you’re expected to submit and possibly how to submit it.
You may also see the timings of when submissions are opened and any deadline submissions are expected by.

Once you’ve submitted, your submission will appear under the ‘Your submission’ section. You can change your submission at anytime during the submission phase.
The next phase in this example is the calibration phase. This phase can either appear before or after the submission phase depending on the activity’s setup.

During the calibration phase you’ll be asked to assess some sample submissions. These samples will be similar to what you’ve been asked to submit. The assessment method and form used to assess these samples will also be the one you and your peers will use to assess each others submissions.

Your calibration score is determined by how accurate and consistent a marker you are based on stringencies determined by your instructor. This is determined by how close your marking of the sample submission is to the expert marker of that same sample submission and how consistently close you are across all the sample submissions you mark.
The calibration phase does not have any timings or deadlines, however it is very important that you try and complete marking your assigned example submissions during this phase in order for the system to determine your calibration score. Your instructors should communicate when they would like you to complete this phase by, or you can determine this yourself if the next phase after calibration has an open date assigned to it.
The calibration score you receive helps determine accurate and appropriate allocation of markers to submissions in the assessment phase where you’ll review and mark your peers work. If all calibration scores are calculated the system will attempt to make sure each of your submissions get assigned at least 1 reviewer who has a high calibration score and therefore is considered a good marker.
The next phase in this UNSW Enhanced Workshop activity is the Assessment phase. Your instructor should provide detailed instructions as to what’s expected of you in this phase.

Here you’ll be assigned some of your peers submissions for you to assess and provide feedback on. You may also be asked to honestly asses your own submission.

In some cases the submissions you assess will be anonymous and your identity as a reviewer will also be anonymous to your peers, but this depends on your instructors pedagogical design. They will determine whether or not anonymity is used.
During the Assessment phase you may re-assess your assigned submissions while the phase is active. Just be sure you’ve completed all your assigned assessments prior to the phase ending.
From this point on you can review your calibration score and determine how good a marker you are based on the stringencies for accuracy and consistency set by your instructor.

It’s a good idea to review these to understand where your marking may have differed from the expert mark. This can give you a better understanding of what the instructor was expecting for the assignment based on the assessment criteria.
Also note that your calibration score is also your assessment mark, but it is only awarded to you if you complete all your assigned assessments in the Assessment phase. If you only complete 2 out of 4 assessments then you’ll only receive 50% of your calibration score as your assessment mark.

The next phase is the Grading and evaluation phase.

During this phase your instructor will be reviewing all your submissions and assessments prior to releasing your marks.

Note that any unfair marking of submissions will be automatically highlighted for the instructor to easily focus on.

This highlighting happens with any assessments that are at least 2 standard deviations away from other assessments of that same submission, so please try and mark fairly and honestly.
The last and final phase is the closed phase where you can now see your grades for Submission and Assessment.

The grades you receive for your submission are weighted based on the reviewers calibration scores.

So this particular grade for submission of 66.25 can be approximately calculated with the following equation:

The minuscule differences in calculations are due to rounding errors as the internal Moodle numbers are much more accurate, but this equation should get you very close to your actual grade and an understanding as to how it was determined.

Artist - BOCrew
Full transcript