Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Assessing Assessment: weaving the golden thread of alignment

No description
by

on 2 December 2013

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Assessing Assessment: weaving the golden thread of alignment

The concrete
experience

Abstract Conceptualisation
Active experimentation
Assessing Assessment: Finding the golden thread of contructive alignment
Setting the scene
Class Context
Student context
Module: Substance misuse
E-learning module: conducted entirely online
Post graduate level
Compulsory for: Msc Children Families and
Young People
Elective for other health streams (Pub Health; Nursing; health care studies)

13 students
10 female; 3 male
1 in Cyrpus
1 from Estonia
12/13 currently engaged in work within the health field

Individual Feedback & grade
Graded and returned within 2 weeks
Students provided with mark / 30
Written feedback in moodle
Approximately a paragraph or two outlining strong points; points for improvement; overall comment; suggestions for future
Formative focus – flow on to second assignment (70%)

General Feedback
2 weeks post submission, generic feedback posted to moodle
Focus on what students did well
Emphasise importance of linking concepts to one another
Emphasise importance of references (up to date; in context; relevant)
Suggestions / tips for future academic work including second assignment

Feedback on the feedback
Three weeks post submission, administration of brief survey to gather student views on entire process including:
Relevance of assignment
Timeliness of feedback
Satisfaction with individual feedback
Satisfaction with generic feedback
Suggestions for improvement

Why?
To gauge student understanding of concepts in module
To "feedforward" for second assignment [Biggs, constructive alignment]
To provide feedback for future module development
On the road to effective feed forward
Assessment: Summative task – formative intent
Due: week 6
Weighting: 30% final grade
Can be used as basis for subsequent 70% assignment

“The Epidemiology topic gives some indication of what we do know about the epidemiology of alcohol and drug harms. But it’s usually the case that there are also important questions that we do not know the answers to – that we do not have any research evidence for.

Basic epidemiological questions would usually take the following forms: What is the extent of [disease/behaviour/harm] in [population] in [setting].

Your first assignment, contributing 30% of the overall mark for this module,  is to come up with one basic epidemiological question that points up an important gap in our knowledge of the epidemiology of substance misuse. In your assignment you should state the question clearly, followed by a c.
400 word
justification of why you think this is an important gap in our knowledge.

This justification should be referenced as per the guidance on referencing”.

Student Self Assessment form
Submitted with assignment
Please reflect on your learning under each of the following headings and submit together with your coursework

Presentation and product
Knowledge base and perception
Diversity
Critical analysis and judgement
Creativity
Evidence of learning
What have I learnt during this module
What future learning needs do I have and how might I achieve them?

Source: student self assessment form
8 / 13 students submitted

Strong theme emerged: difficulty with
small word count


Assignment
challenged their thinking
in relation to substance misuse

Reported a
gain in understanding
of epidemiological concepts

Establishing a
connection
between work practice, knowledge and theory
Source: Feedback Survey
Student reflections
"(critical analysis and judgement) … I have tried to include this, although with a short assignment it is difficult to say what the research says and critique it within the word allow"

“I would have loved more words to make a stronger argument here..”

"I am learning to reconcile my experiential knowledge with the available body of researched knowledge including policies"
"The word length made me very critical of the research- I went straight to the point."
"I found it difficult to stick within the word count and get the pertinent facts in"
"At the start of the module, I felt confident with my substance misuse experience and knowledge- from various training and professional experience. After week one I realised I had a lot of learning ahead of me. The module has challenged my thinking"
(critical analysis) ...I found this impossible in the word count – trying to present an argument based on some level of fact left me no scope to really critique
- I already feel I have a strong knowledge of substance misuse but I have certainly learned to write concisely and have found this assignment a really nice refresher back into the field
But ...
3 / 13 students responded - all 3 had completed self assessment exercise

Feedback was received in timely manner
Individual
feedback was of
limited
benefit: ‘helpful’ (2) and ‘neutral’ (1)

General
feedback less helpful: 2 ‘neutral’ 1 ‘helpful’)

What could improve the feedback process?
"A provision of the marking grid as prescribed by the university. Otherwise, the grading might seem a bit subjective."

... A general grading criteria is provided in the course materials - however this is not explicitly linked to the assignment
Reflective Thinking
Personal Reflections
Teacher context
Co teacher
Supportive role
No structural changes to be made to course
yet
* Standard process for this module not a new initiative
What worked well
Freedom of choice for assignment question: made learning relevant & contextual for the student

Self assessment: encouraged reflection and provided both student and tutor with feedback

400 word lightly weighted essay: allowed for brief 'check in' on student learning
On the flipside...
Freedom of topic choice made harder to mark [no common 'standard' for contrast / comparison ]

Self assessment lacked explicit objective [i.e. was it directly related to the assignment or to the module overall?] - hard to assess / draw themes as a result

400 word limit: restrains 'deep learning'; stifles the potential for development of genuinely effective feedback



Identifying the need for the golden thread
I struggled in providing only short feedback - usual practice is to mark in text and provide 1/2-1 page of notes
Next assignment is 70%: is 400 words with brief feedback adequate enough in terms of constructive alignment needs and effective formative feedback processes?
Feedback on the feedback 'flopped' - but I still view it as important to get the student voice
Feeling: where does this feedback go? Weakness in no 'results' until next assignment (which is heavily weighted)
Dialogue with peers
Despite broad range of assignment topics and broad marking guidelines - grading and feedback was almost identical to colleague
Suggested my feedback may be too detailed for such a short assignment
I focus too heavily on strength and adequacy of references for such a short task [idea is to gauge where students are at and provide feedback]
supportive of suggestion to provide general feedback document
... marking 'guidelines' were used not 'criteria'
Why are assessment & feedback important?
One the 6 key areas of UK professional Standards Framework (UKPSF)

2012 National Student Survey (NSS) indicates students least satisfied with this area

'Firm evidence that assessment is not successfully meeting the needs of students, employers, politicians or the public in general' (HEA, 2012)
The assignment links to ...
graduate attributes
of: academic literacy; research literacy; critical self awareness and personal literacy; digital and information literacy

Relevance and transferability
(Rust, 2002): Students could choose own question related to their own discipline / experience / culture to make it relevant

Employability skills
(HEA, 2012): more than simply repeating knowledge – students applied it to different contexts, professional settings, thinking critically. Many students drew from their own working experience / environment in development of their question

Links to wider module
learning objectives
: knowledge and understanding; disciplinary and professional skills; transferable skills
Wider module learning objectives
Knowledge and understanding
describe how substance misuse can be conceived and understood in a historical and societal context, including the extent of substance misuse.  
express and examine the psychological, social and policy factors and issues relating to substance misuse.
Disciplinary and professional skills
build skills in research utilisation, in critical analysis of research and in exploring the applicability of research to work for children young people and family wellbeing in the context of substance misuse.
Transferable skills
report clearly, accurately and unambiguously on the findings and implications of work to specialist and non-specialist audiences.
Manage time effectively
select and use appropriate academic skills (research, analysis, synthesis).

Can summative assessment have formative function?
Assessment of learning (summative) vs assessment for learning (formative)

‘… In practice, to varying degrees, most forms of assessment
probably try to do both
’ (Rust, 2002, p 1.)

Assessment & activities resulting in feedback that is used for
modification of teaching
and learning based on
identified needs
can be considered formative assessment (Black & William, 2001).

Formative assessment / feedback can:
Produce important learning gains, particularly among the “low attainers” (Black & William, 2001);

Be effective across a
range of contexts
including educational level, skills, content (Nicol & MacFarlane-Dick, 2006)

Allow tutors to
test
student constructions and progress of knowledge development (Higgins et al, 2002)

General feedback
allows for a ‘depersonalised’ look at strengths and weaknesses (ASKe, 2008)

Identified as an essential graduate attribute (HEA, 2012); (Brookes Assessment Compact 2009)

Allows for greater insight into requirements of task, in turn improving students own work (Rust, 2002)

Good ‘feedback’ strengthens students capacity to self regulate their own performance (Nicol & MacFarlane, 2006)

Attainment of goals and / or improvement of outcomes
rely strongly
upon student
understanding
/ comprehension of
criteria and standards
(Nicol, Macfarlane-Dick, 2006; Sadler, 1989; Black and William, 1998; Price et al 2008)

Without this, there is the risk that students will
not attach meaning
to the feedback (Hounsell, 1997 as cited in Nicol-Macfarlane-Dick, 2006).




Without clear learning objectives and explicit marking criteria / goals related to the task, have we run the risk that students will not attach meaning to the feedback?
'Feedback on the feedback' indicates that this might be ... yes.
Self regulation is extremely important for this module as it's entirely online
Examining the procedural process
1.2 To be effective the relational nature of the assessment and feedback process needs to be emphasised, particularly in terms of the need for dialogue between students and staff

reflective observations and critical analysis of the task and module reveal that the link between learning objectives, tasks, assessment and feedback is not clearly defined here - there is no "golden thread"

2.2 The relationship between the learning outcomes and assessment tasks should be made explicit

Currently, the learning objectives and intended learning outcomes are not made explicit at all

... In addition, clear assessment criteria should be provided

general grading criteria are provided but there is not set criteria for this assignment
Examining the feedback process
Seven Principles of good feedback: facilitating self-regulation: Nicol & Macfarlane-Dick (2006) p 205.

1. Helps clarify what good performance is (goals, criteria, expected standards);
[There is a clear question
but
no set goals or learning objectives linked directly with the task]
2. Helps facilitate the development of self-assessment (reflection) in learning;
[achieved through self assessment form
but
self assessment questions need to be explicitly related to task OR clearly targeting generic feedback. Current format is unclear of intent/aim. Are students assesing their learning of the assignment or assessing their learning of the overall module? ]
3. Delivers high quality information to students about their learning;
[attempted with individual feedback and general feedback post –
but
not valued too highly]
4. Encourages teacher and peer dialogue around learning; [attempted to capture with feedback survey – little response / engagement with process]
5. Encourages positive motivational beliefs and self esteem;
[written feedback framed in positive manner with suggestions for future academic work]
6. Provides opportunities to close the gap between current and desired performance; [indirectly through feedback and constructive alignment in using first assignment to inform / build upon content for second assignment]
7. Provides information to teachers that can be used to help shape teaching [review of student self assessment; responses to feedback survey. Plenty of opportunity - but effective enough?]

For feedback to be effective ... (Sadler, 1989)
1. Students must know what good performance is
2. Students must know how their current performance relates to good performance
3. How to close the gap between the two
Without explicit learning objectives, intended learning outcomes and assignment specific marking criteria it is unclear if the students have this understanding
Track how students use feedback for the second assignment (feed this in to the wider module review)

Invite students to submit an abstract draft submission for the second assignment to allow for 'proper' formative feedback

Add self assessment task (with clear objectives) to submission of second assignment

In line with student self assessment feedback, review and rework the first assignment to include:
Longer word count (to allow for deeper learning and to provide basis for deeper feedback)
Develop and disseminate explicit learning objectives and outcomes associated with the assignment
Develop and and disseminate explicit marking criteria
Conduct a post-module analysis of the process utilising the HEA/Brookes mannifesto in particular asking:

To what extent is assessment for learning given emphasis in relation to assessment of learning
is assessment and feedback planned … to ensure appropriate student preparation and practice before summative assessment takes place
Does the assessment design process ensure valid assessment of the intended learning outcomes
Is there an exploration of the impact of explicit assessment criteria on assessment practice and work of the students
Ensuring student understanding of alignment and attaching meaning to feedback :
providing an example of "good work" (former essay)
outlining explicit learning objectives /outcomes associated with self assessment task (make clear to be done for reflection on assignment 0- not overall module)
create opportunity for formative assessment without attaching marks (submission of draft)
Immediate plan of action
The bigger 'picture': module review
Thinking ahead: weaving the 'golden thread' for constructive alignment
* Limited opportunity to alter the module activities. Discussion with colleague / module leader led to decision to do analysis on the summative assignment
There was no clear explicit learning objective (s) for this assignment: did this hinder the entire constructive alignment / formative feedback process?
The Brookes Assessment Compact
Student self assessment
Full transcript