Send the link below via email or IMCopy
Present to your audienceStart remote presentation
- Invited audience members will follow you as you navigate and present
- People invited to a presentation do not need a Prezi account
- This link expires 10 minutes after you close the presentation
- A maximum of 30 users can follow your presentation
- Learn more about this feature in the manual
Do you really want to delete this prezi?
Neither you, nor the coeditors you shared it with will be able to recover it again.
Make your likes visible on Facebook?
Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.
Matt Wingfieldon 20 May 2014
Transcript of Sociology_ACJ_Nationals_Pilot
People are good at:
People are less good at:
Absolute Judgements, e.g. Scoring.
People & Assessment
reduced assessment reliability; and
Illustrating the issue...
Measuring without subjectivity – adaptive scaling without needing scores;
Based on The Law of Comparative Judgement (1927);
A different approach...
TAG has created a web-based ADAPTIVE comparative judgement solution for assessing student work...
Adaptive Comparative Judgment
The system runs successive 'rounds' of judgement to order the work in rank order...
ACJ - behind the scenes
Resulting in a secure, reliable rank order that has been generated through collective Professional Consensus...
Which can then be graded/scored.
Reduced inconsistency; and
Increased reliability (typically 0.93 – 0.98).
The result...turns traditional assessment on its head!
The assessment problem:
Typically as low as 0.5-0.6
An innovative approach to
assessment and standardisation
College Development Network
Our Customers: Institutions & Agencies
Our Customers: Accreditation Bodies
Managing Director, TAG Assessment
Tel: 0203 176 0394
Advisor, College Development Network
Tel: 07920 824 793
Presentation at: http://tinyurl.com/ACJ-Nationals
Focus on short-answer and essay responses;
To inform assessment options for Nationals & Highers;
HN Sociology was chosen as the focus;
Six Colleges took park:
Dundee and Angus College;
(Dundee & Angus Colleges)
North East Scotland College;
(Aberdeen, Banff & Buchan)
Scope of Pilot:
Pilot run over 2 month period;
Each college contributed scanned hand-written scripts – 72 in total, which were then assessed by the group;
Sample included 'open' & 'closed' book responses;
Scripts pre-marked using traditional criterion referenced markschemes to provide direct comparison to ACJ.
Results and Conclusion:
A total of 12 'Rounds of Judgement' were completed;
Each judgement took an average of 2.42 mins;
Achieved an assessment reliability coefficient of 0.96;
However, 0.95 reliability achieved after just 8 'rounds';
Judges reaction was very positive;
Saw huge potential for the Nationals & Highers assessment & standardisation.
Currently progressing a national pilot to run in two phases from June & September-December 2014;
Focus on capturing & assessing 'creativity';
College Development Network;
Education Scotland; and
Aiming to include 13 school/college/ CLD/Industry partnerships for the pilot.