Loading presentation...

Present Remotely

Send the link below via email or IM

Copy

Present to your audience

Start remote presentation

  • Invited audience members will follow you as you navigate and present
  • People invited to a presentation do not need a Prezi account
  • This link expires 10 minutes after you close the presentation
  • A maximum of 30 users can follow your presentation
  • Learn more about this feature in our knowledge base article

Do you really want to delete this prezi?

Neither you, nor the coeditors you shared it with will be able to recover it again.

DeleteCancel

Make your likes visible on Facebook?

Connect your Facebook account to Prezi and let your likes appear on your timeline.
You can change this under Settings & Account at any time.

No, thanks

Utilizing pre-existing instruments, scales and inventories in student affairs assessment

Presentation for 6-19-12 assessment symposium
by

Shannon Faris

on 12 June 2014

Comments (0)

Please log in to add your comment.

Report abuse

Transcript of Utilizing pre-existing instruments, scales and inventories in student affairs assessment

Let's not reinvent the wheel:
Using existing scales, inventories and items for student learning outcomes assessment in student affairs.

Josie Ahlquist, LMU
Shannon Faris, LMU
Assessment in Student Affairs
Much focus on
knowledge-based
SLOs
However, SAs SLOs are often
development-centered
:
attitudes, values & beliefs
Harder to capture
constructs
indirect measures
Always looking for creative methods!
Very little in the literature on utilizing this method in this way.
Borrowed from:
national/published instruments & items
psychometrics in social sciences, psychology
Potential Advantages
http://www.google.com
Caveat Emptor
Availability
Quality
psychometrics:
validity, reliability
&all that jazz
less time
less money
1. Do your homework:
Who and how was it developed?
For whom was it designed and why?
How has it been utilized?
How are your intentions similar/different from original?
What do you know about psychometric quality?
You are responsible!
2. Pay attention to sampling procedures.
Was it designed to use at the individual or group level?
3. Don't be afraid to try out newer inventories.
4. Measurement of learning is always evolving.
Where to find them:
Repositories of existing instruments:
Mental Measurements Yearbook Online- //buros.unl.edu/buros/jsp/search/jsp
Inventory of Higher Education Instruments: www.stanford.edu/group/ncpi/unspecified/assessment states/instruments.html
www.learningoutcomesassessments.org/surveys/htm
ETS Testlink
Discipline-specific search
References
Questions?
Additional Lessons Learned
Use the scale exactly, even the wording and order of questions
Include demographic information
Set a realistic methodology
Consider a multi-year study to add depth
It is never too early to start preparing
Use the data for not only practice but for grants, conference proposals and even scholarly journal submissions
Challenges
Learning Curve
Creating a Study
Time
Student Reponses
Interpreting the data
Professional Challenge
Training a student how to do research
Reflection & Setting Realistic Methods
Individual outreach with student performers
Revisiting Student Development Theory
Opportunities
Discussion
 Participating students are provided with a small community of fellow performers who nurture the talent of each group member. When a student steps onto the stage at the weekly Open Mic program, they are overcoming an internal struggle with self-efficacy that has been widely studied within the field of Educational Psychology.

The study “Academic Self-Efficacy and First Year College Student Performance and Adjustment” performed by Martin M. Chemers, et. Al, appeared in the Journal of Educational Psychology in March 2001, which studied and established the impact that self-efficacy, or a student’s belief in their own competence, has on first year college students’ actual performance in the classroom.
Correlated to this and other similar studies, are the results of the question regarding student self confidence in the classroom and within extracurricular activities. We found the 84% of Open Mic performers report experiencing increased self confidence off-stage as a result of their experience on-stage at Open Mic.

We find these results incredible important, particularly considering our daily sign-in sheets indicate that 40% of students attending our Open Mic nights are first year students. The aforementioned study identifies first year students as the most vulnerable to shifts in feelings of self-efficacy, whether positive or negative.

The results of this study establish Open Mic as an extremely effective tool for the Division of Student Affairs to influence that shift in self confidence positively in the students’ most formative years.
Two Open Mic performers describe what Open Mic provides to them as,

“Open Mic is a place to freely express myself and to practice in front of a live supporting audience,”
Another stated, “it gives me confidence and an open forum to share my work.”
Overall Significant Findings
The majority, 86% of survey respondents, were performers of music.

When asked to describe in their own words what Open Mic has provided for them, 79% of survey respondents indicated the value of Open Mic was in the opportunity it provides for self-expression.

Using the Rosenberg Self Esteem Scale, it was found that Open Mic performers have a healthy level of self confidence. When asked if they are satisfied with themselves, 93% agreed or strongly agreed and 100% believe they have a number of good qualities.

When asked to scale their confidence level on stage compared to other areas such as in the classroom, as a student leader, or even as a friend, Open Mic participants rated their confidence levels significantly higher as a performer. Specifically, the average confidence level, on a scale from 0-100, performing on stage was 84%, compared to 78% in the classroom and 79% as a student leader & a friend.

Additionally, performers were asked to rate their self confidence prior to entering LMU, after their Freshman Year and their Current grade level. These results were as follows:
Prior to LMU 60%
After Freshman Year 61%
Current Grade Level 79%

Notably, 100% of respondents stated they Strongly Agree or Agree that performing at Open Mic
has increased their self-confidence as a performer.
Rosenberg’s Self Esteem Scale
We included the Rosenberg Self Esteem Scale in order to establish an academically accepted baseline self-confidence rating for this group of respondents prior to delving into questions of the association of their confidence to Artistic Performance. Upon evaluation of the results of this scale (seen in the table below), this group of survey takers scored a 17 on the Rosenberg Scale, indicating that they are solidly within the “Normal Range” of self-esteem as interpreted by the Morris Rosenberg Foundation.
Assessment Method
In the spring semester of 2011, a nine question survey was created using Qualtrics survey software and made available online.
E-mails were sent to students who regularly performed at our weekly Open Mic Nights in the Fall of 2010 and Spring of 2011.
For this study, regular performance is defined as 2 or more performances per semester.
Thirty students qualified as regular performers and were asked via e-mail and in person to participate in the survey.
Of those 30 e-mail recipients, 14 completed the survey.
The survey questions utilized Likert Scale style responses in which the survey respondent replied to statements using a 5 point scale from “Strongly Agree” to “Strongly Disagree,” for both the “Rosenberg’s Self Esteem Scale” as well as our own specifically tailored questions.
Open Mic at LMU
LMU Student Affairs Division Learning Outcomes
Graduates of Loyola Marymount University will:
develop a well-defined sense of self confidence in their abilities and gifts.
adopt lifestyles that reflect a balance of spiritual, physical, emotional and intellectual health.
integrate knowledge and experience in the process of discernment to make life-giving decisions.
take initiative to make meaningful contributions as citizens within and beyond their communities.
demonstrate an understanding of the diversity of the human experience and embody the ideals of interculturalism.
Open Mic Program Learning Outcomes Project
Study was designed to evaluate the impact that regular artistic performance has on the development and nurturing of the self-confidence that student performers have in their unique abilities, talents, and gifts.
This explored confidence both inside and out of the classroom, particularly on-stage at LMU’s Open Mic program, offered through the Department of Campus Recreation, Mane Entertainment.
Open Mic Program at
Loyola Marymount University
Josie Ahlquist
Associate Director, Campus Recreation
(Mane Entertainment)
Also Known as the “Lion Tamer”
Loyola Marymount University
Open Mic Program
Long standing program, over the last two decades.

Offered every Monday at 8pm in the Malone Student Center Living Room during the academic year.

On average, there are 30 Open Mic events per year, with 7-12 performers showcasing 2 songs or 7 minutes on stage.

Open Mic is managed through Mane Entertainment student staff, including Event Technicians, Emcees, Programming Assistants, a Event Student Manager, as well as one professional staff member.

Audience attendance ranges from 20 to 150 LMU community members and performers spotlight talents such as music, comedy, poetry, improv, drama, and more.
Session Goals:
Present an assessment example that utilized a novel methodology.

Provide infrastructure to begin to transform the approach from novel to robust.
Measures
still indirect, but more robust
Resource Efficiency
Full transcript