Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Creating a Structured Electronic Portfolio for your Residency Program from Start to Finish!

Presented at the International Conference on Residency Education (ICRE) 2015.
by

Lyn Sonnenberg

on 30 January 2016

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Creating a Structured Electronic Portfolio for your Residency Program from Start to Finish!

Creating a Structured Electronic Portfolio for your Residency Program from Start to Finish!
The 12 Essential Features
Case Study: Blogger, Moodle & Google Sites
Google Sites Demonstration
https://sites.google.com/a/ualberta.ca/
Your Experience
Platforms
Features
Frustrations
Solutions
Lyn K. Sonnenberg, MD, MEd (HSE), MSc, FRCPC
Program Director, Dev Peds, University of Alberta
Matthew T.W. Prowse, MD, FRCPC
Program Director, PM&R, University of Alberta

At the end of this session, the participant will be able to:

1. Analyse their Program’s objectives of training and select content suitable for inclusion in the portfolio.

2. Adapt this selected content into a portfolio learning and assessment system.

3. Create an appropriate e-portfolio platform suitable for their Program/Learners.

1. Short lecture on determining which content is best suited for portfolio inclusion.
2. Small Group Discussion on experience with adapting content for portfolios.
3. Demonstration of e-portfolio content integration & design using Google Sites platform.

How we'll get there...
Customization/
Functionality
Sharing
Scalability
Incorporation
of Feedback
by Mentor/Faculty
Privacy
Personalization
by Learner
Mobile
Accessibility
Sustainability
Portability
Support
Cost
Ease of Use
CAN I CUSTOMIZE TO ACHIEVE FUNCTIONALITY?
Customization leads to functionality.

There will always be a need for some degree of customization, given that each educational setting is unique

The skill set needed for a surgical resident is quite different than that of a developmental pediatrician
The e-portfolio should reflect the needs of training

Creation of core content that can readily be shared between different residency programs

How easy it is to share the e-portfolio template between departments and even different institutions, in the spirit of collaboration?

CAN IT BE SHARED BETWEEN
DEPARTMENTS AND INSTITUTIONS?
CAN IT BE SCALED FOR USE WITH EACH
NEW COHORT OF LEARNERS?
Each academic year brings with it a new cohort of learners, who each require their own e-portfolio

How much effort is involved in replicating a new e-portfolio for each learner?

It may be practical to replicate an e-portfolio for four learners each year, but may be too time-consuming for a cohort of 30 learners.
HOW CAN MENTOR/PROGRAM DIRECTOR FEEDBACK BE INCORPORTED?
Connecting learners and mentors is often difficult due to the issue of scheduling and different physical locations

Mentors prefer e-portfolios to paper-based portfolios due to its straight-forward navigation and the ability to easily access the portfolio online

The capability of linking a mentor’s feedback directly to the comments or posts made by the learner is essential in order to facilitate dialogue and create growth for the learner.
Driessen, Muijtjens, van Tartwijk & van der Vleuten
Medical Education 2007
Clay et. al Medical Teacher 2007
Privacy of personal information is a universal concern, particularly in the field of healthcare

All information posted in a learner’s portfolio should be anonymized, but personal reflections and research projects require a judicious amount of protection

In order for a learner to truly be open in his or her reflection, there should be the opportunity to “lock down” a particular reflection so that it is not accessible to even the Program Director or mentor.

This does become challenging for most software applications.
WHAT ARE THE PRIVACY SETTINGS AND CONTROLS?
CAN IT BE PERSONALIZED BY THE LEARNER?
Taking ownership and making it your own

Tailoring the platform for greater functionality for the individual learner is also important

The same way we approach a clinical problem from different perspectives, each learner’s approach to the e-portfolio is different as well
IS THE PLATFORM MOBILE DEVICE ACCESSABLE?
Convenience and efficiency are necessities for any platform

The ability to maneuver the platform on a mobile device is the more pressing question to be asked

Another point to consider is offline use, as an internet connection is not always available.
IS THIS PLATFORM SUSTAINABLE?
Each platform comes with its own risk level, which emanates from different levels, essentially creating a “shelf-life” for the product

Is the software company going to “disappear” or are they well established in the industry? Product retirement?

New versions and updates can either improve the platform or they can render it non-functional for your intended purpose.

“How many years is reasonable to expect from this product” given the effort you will need to invest in developing the platform to meet your needs?
WILL THE LEARNER REQUIRE ACCESS
ONCE TRAINING IS COMPLETED?
A portfolio is representative of a learner’s work. It is a summary of their experiences

Does the learner wish to take it with them when they leave the University walls?

Portfolios are continuing to find a place in continuing medical education (CME). Having an established e-portfolio that learners can carry with them into practice is a benefit
Dornan, Carroll, & Parboosingh
Medical Education
2002
WHAT DEGREE OF SUPPORT CAN I GET
WHEN I NEED HELP?
Support for the platform can be divided into local, face-to-face support, and external support, in the form of web resources and call centres

The most important question to ask when selecting a platform in terms of support is, “can I get support for the tool the way I am using it?"
WHAT ARE THE FINANCIAL COSTS?
Fiscal restraints are present at all levels of healthcare and education

When considering a platform, there are potential costs to the University or institution as a whole, to the faculty members and to the individual learner

A guiding principle is to consider what platforms are currently provided by your Institution or Department and to know the timeline of the existing contracts
Navigation
Usability
Each new technological advance requires orientation and has a learning curve, but some products, though, are much more intuitive to use than others

Learners want to be able to get to where they need to be quickly, usually within a “click” or two.

How likely the learner is to use the platform to accomplish the task? Make it too complicated and the learner will be frustrated. Make it too simple and it impacts functionality.

Buy-in from learners is essential and usability is a key factor.
HOW IS ITS EASE OF USE OVERALL?
Objectives
Disclosures & Conflicts of Interest
We do not have an affiliation (financial or otherwise) with a pharmaceutical, medical device or communications organization.

But we would like to acknowledge:

Dr. Sonnenberg's Subspecialty Residents and Fellows: Drs. Al-Jabri, Bobbitt, Ogilvie, Lemeiux, Lacelle-Webster, and Eliason
http://ogilviedevpeds.blogspot.ca/p/introduction.html
https://moodle.med.ualberta.ca/course/view.php?id=182
https://sites.google.com/a/ualberta.ca/susanbobbitt/
CAN I CUSTOMIZE TO ACHIEVE FUNCTIONALITY?
Customization leads to functionality.

There will always be a need for some degree of customization, given that each educational setting is unique

The skill set needed for a surgical resident is quite different than that of a developmental pediatrician
The e-portfolio should reflect the needs of training

Creation of core content that can readily be shared between different residency programs

How easy it is to share the e-portfolio template between departments and even different institutions, in the spirit of collaboration?

CAN IT BE SHARED BETWEEN
DEPARTMENTS AND INSTITUTIONS?
CAN IT BE SCALED FOR USE WITH EACH
NEW COHORT OF LEARNERS?
Each academic year brings with it a new cohort of learners, who each require their own e-portfolio

How much effort is involved in replicating a new e-portfolio for each learner?

It may be practical to replicate an e-portfolio for four learners each year, but may be too time-consuming for a cohort of 30 learners.
HOW CAN MENTOR/PROGRAM DIRECTOR FEEDBACK BE INCORPORTED?
Connecting learners and mentors is often difficult due to the issue of scheduling and different physical locations

Mentors prefer e-portfolios to paper-based portfolios due to its straight-forward navigation and the ability to easily access the portfolio online

The capability of linking a mentor’s feedback directly to the comments or posts made by the learner is essential in order to facilitate dialogue and create growth for the learner.
Privacy of personal information is a universal concern, particularly in the field of healthcare

All information posted in a learner’s portfolio should be anonymized, but personal reflections and research projects require a judicious amount of protection

In order for a learner to truly be open in his or her reflection, there should be the opportunity to “lock down” a particular reflection so that it is not accessible to even the Program Director or mentor.

This does become challenging for most software applications.
WHAT ARE THE PRIVACY SETTINGS AND CONTROLS?
CAN IT BE PERSONALIZED BY THE LEARNER?
Taking ownership and making it your own

Tailoring the platform for greater functionality for the individual learner is also important

The same way we approach a clinical problem from different perspectives, each learner’s approach to the e-portfolio is different as well
IS THE PLATFORM MOBILE DEVICE ACCESSABLE?
Convenience and efficiency are necessities for any platform

The ability to maneuver the platform on a mobile device is the more pressing question to be asked

Another point to consider is offline use, as an internet connection is not always available.
IS THIS PLATFORM SUSTAINABLE?
Each platform comes with its own risk level, which emanates from different levels, essentially creating a “shelf-life” for the product

Is the software company going to “disappear” or are they well established in the industry? Product retirement?

New versions and updates can either improve the platform or they can render it non-functional for your intended purpose.

“How many years is reasonable to expect from this product” given the effort you will need to invest in developing the platform to meet your needs?
WILL THE LEARNER REQUIRE ACCESS
ONCE TRAINING IS COMPLETED?
A portfolio is representative of a learner’s work. It is a summary of their experiences

Does the learner wish to take it with them when they leave the University walls?

Portfolios are continuing to find a place in continuing medical education (CME). Having an established e-portfolio that learners can carry with them into practice is a benefit
WHAT DEGREE OF SUPPORT CAN I GET
WHEN I NEED HELP?
Support for the platform can be divided into local, face-to-face support, and external support, in the form of web resources and call centres

The most important question to ask when selecting a platform in terms of support is, “can I get support for the tool the way I am using it?"
WHAT ARE THE FINANCIAL COSTS?
Fiscal restraints are present at all levels of healthcare and education

When considering a platform, there are potential costs to the University or institution as a whole, to the faculty members and to the individual learner

A guiding principle is to consider what platforms are currently provided by your Institution or Department and to know the timeline of the existing contracts
Each new technological advance requires orientation and has a learning curve, but some products, though, are much more intuitive to use than others

Learners want to be able to get to where they need to be quickly, usually within a “click” or two.

How likely the learner is to use the platform to accomplish the task? Make it too complicated and the learner will be frustrated. Make it too simple and it impacts functionality.

Buy-in from learners is essential and usability is a key factor.
HOW IS ITS EASE OF USE OVERALL?
Platform Comparison
https://sites.google.com/site/devpedstemplate/

Resources
Google Search: Prezi Structured Portfolio Sonnenberg
lyn.sonnenberg@ualberta.ca mprowse@ualberta.ca
http://basmaaljabridevped.blogspot.ca/2013_11_01_archive.html
https://sites.google.com/a/ualberta.ca/jogilvie/
https://sites.google.com/a/ualberta.ca/aljabriportfolio/medical-expert/patient-encounter-log
Blogger
Retasked
Moodle
Flexibility
Mahara Plug-in
Google Sites
Great Flex
Learning Curve
Moodle
Same Version?

Blogger
Complete Reprogramming
Google Sites
Templates!
Easy Peasy
Moodle
New site generated for each learner
Blogger
Site Duplication
Manual Editing
Google Sites
Learner
Generated
Blogger
Blogging Tool
Imbedding and linking comments
Moodle
Comments are more challenging to link
Forums are the way to go here
Google Sites
Commenting built into design
Learner can delete comments
Mentor access to only part of portfolio
Moodle
housed within University firewalls
Blogger
Can be set by user
Google Sites
Site level
Page access
Complicated
Blogger
Adapted well
Strength
Moodle
Some
Not easy
Google Sites
Born for Personalisation
Page customisation
Moodle
Requires server-side extensions
Blogger
Auto detect and
reformats
Google Sites
Latest feature added
Moodle
University Contract
Blogger
Google Product
Google Sites
Google Product
Blogger
Learner-owned
Moodle
Snap Shot on Export
Open Source Software
Google Sites
Learner-owned
Blogger
Retasked
Moodle
Academic Support
Retasked
Google Sites
Forums
YouTube
Blogger
Free
Moodle
Free to download
Support to maintain
Google Sites
Free
Add'l Storage Costs
Blogger
Challenging, even for advanced users
Moodle
Design flexibility at the expense of intuitive navigation
Google Sites
Intuitive
Straight forward to use
Creating & Duplicating the Portfolio
https://docs.google.com/document/d/1-9-DNdpAwIVboFzbmw1HEfoNvZS-ne2Swmpq-veEiio/edit?usp=sharing
Principal Characteristics of a Structured Portfolio
Multifaceted approach to evaluation
Triangulation
Longitudinal
Comprehensive in scope
Includes evidence of learner self-assessment and reflection
Contains learner contribution, demonstrating evidence of growth
Holmboe et al 2006 APM Perspectives
Components for a Structured Portfolio
NB. Needs to include at least one method of evaluation from each broad category
Foundational Evaluations
longitudinal, global, monthly evals; robust composite
Direct Observations
clinical, communication, interpersonal skills
Practice & Data-Based Learning
personal performance data, including self-assessment, reflection and Quality Improvement plan
Multi-Source Feedback (MSF)
patient and non-physician perspectives
Holmboe et al 2006 APM Perspectives
Clear Vision
What is a Structured Portfolio?
Framework and process for:
Collecting
Analysing/Reflecting
Documenting

Requires active engagement of the learner in his/her own assessment processes

Evidence for improved self-awareness, learning augmentation, independent learning, and integration of theory with practice.

Buckley, Coleman, Davison et al BEME Systematic Review Guide no. 11 Med Teach 2009
Donato and George, A Blueprint for Implementation of a Structured Portfolio in an Internal Medicine Residency in Academic Medicine, Dec 2011
Creating Content:
Don't start from scratch
Don't reinvent the wheel
Rome wasn't built in a day, but your portfolio can be!
Creating Content:
Start with your Objectives of Training
Print them off
Grab a pen and a highlighter
Review one by one
Ask yourself:
How am I assessing this skill/trait?
Can it fit in the portfolio?
Does it make sense to include here?
Bonus Features:
Quality Improvement Course (IHI Open School)
Teaching Perspectives Inventory (TPI)
Critical Appraisal of the Literature (CEBM)
Conflict Management Questionnaire
Time Management Resources
Reflection Feedback Form
Oral Presentation Assessment Form
Multi-Source Feedback Form
Stacer-like Assessments
Observed History and Physical Forms
From your Royal College Specialty Committee:
https://sites.google.com/site/PMRresidentportfolio/
https://sites.google.com/a/ualberta.ca/lisalemieux/
Session Evaluation
Go to:
http://www.royalcollege.ca/icreevaluations
to complete the session evaluation.

@lynsonnenberg
lyn.sonnenberg@ualberta.ca
mprowse@ualberta.ca
https://sites.google.com/site/uofadevpedseportfolio
IHI Open School: Quality Improvement Course
http://app.ihi.org/lms/home.aspx?CatalogGuid=6cb1c614-884b-43ef-9abd-d90849f183d4
Full transcript