Loading presentation...

Present Remotely

Send the link below via email or IM


Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.



No description

Matt Wingfield

on 23 November 2015

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Sociology_ACJ_Nationals_Pilot

For more information please contact:
People are good at:
Relative Judgements;
Ranking work.

People are less good at:
Absolute Judgements, e.g. Scoring.
People & Assessment
The result:
reduced assessment reliability; and
increased inconsistency.
Illustrating the issue...
Measuring without subjectivity – adaptive scaling without needing scores;

Based on The Law of Comparative Judgement (1927);
A different approach...
Comparative Judgements
TAG has created a web-based ADAPTIVE comparative judgement solution for assessing student work...
Adaptive Comparative Judgment
Student A
Student B
The system runs successive 'rounds' of judgement to order the work in rank order...
ACJ - behind the scenes
Resulting in a secure, reliable rank order that has been generated through collective Professional Consensus...

Which can then be graded/scored.
Reduced inconsistency; and
Increased reliability (typically 0.93 – 0.98).
The result...turns traditional assessment on its head!
The assessment problem:
Typically as low as 0.5-0.6
An innovative approach to
assessment and standardisation

Aileen Duffy
College Development Network

Matt Wingfield
Managing Director
TAG Assessment

Our Customers: Institutions & Agencies
Our Customers: Accreditation Bodies
Matt Wingfield
Managing Director, TAG Assessment
e-mail: matt.wingfield@tagassessment.com
Tel: 0203 176 0394
Aileen Duffy
Advisor, College Development Network
e-mail: Aileen.Duffy@collegedevelopmentnetwork.ac.uk
Tel: 07920 824 793

Presentation at: http://tinyurl.com/ACJ-Nationals
The Pilot:
Focus on short-answer and essay responses;
To inform assessment options for Nationals & Highers;
HN Sociology was chosen as the focus;
Six Colleges took park:
Edinburgh College;
Dundee and Angus College;
(Dundee & Angus Colleges)
North East Scotland College;
(Aberdeen, Banff & Buchan)
Ayrshire College;
Scope of Pilot:
Pilot run over 2 month period;
Each college contributed scanned hand-written scripts – 72 in total, which were then assessed by the group;
Sample included 'open' & 'closed' book responses;
Scripts pre-marked using traditional criterion referenced markschemes to provide direct comparison to ACJ.
Results and Conclusion:
A total of 12 'Rounds of Judgement' were completed;
Each judgement took an average of 2.42 mins;
Achieved an assessment reliability coefficient of 0.96;
However, 0.95 reliability achieved after just 8 'rounds';
Judges reaction was very positive;
Saw huge potential for the Nationals & Highers assessment & standardisation.
Where next?
Currently progressing a national pilot to run in two phases from June & September-December 2014;
Focus on capturing & assessing 'creativity';
College Development Network;
Creative Scotland;
Education Scotland; and
Aiming to include 13 school/college/ CLD/Industry partnerships for the pilot.
Full transcript