Send the link below via email or IMCopy
Present to your audienceStart remote presentation
- Invited audience members will follow you as you navigate and present
- People invited to a presentation do not need a Prezi account
- This link expires 10 minutes after you close the presentation
- A maximum of 30 users can follow your presentation
- Learn more about this feature in our knowledge base article
Measuring Mindfulness: A Critical Review
Transcript of Measuring Mindfulness: A Critical Review
MBCT After widespread application of mindfulness
in the field, attention turning to closer study of
the phenomenon itself, largely in the form of
quantitative measurement, and exploration of the
"mechanisms" involved. My concern: We are taking a practice
based largely on Asian-philosophy/religion, transplanting to Western-scientific reductionist framework, in order to quantify what is essentially a practice of "nonstriving" It seems to me a danger of misunderstanding (e.g. something for mentally ill people) and distorting what it is (e.g. reification, operationalization, quantification, comparing) With 8+ quantitative scales for assessment of mindfulness, it is time for a critique. Long-time interest in Eastern philosophy and spiritual practice. Consisting of reading, coursework, personal practice Perceived similarities between existential philosophical
assumptions and Buddhist/Taoist beliefs as applied to
psychology. (Wu wei, nondual existence) My fascination with mindfulness seemed at odds with what
the field at large was doing with mindfulness.Rather than
pursue my own project applying mindfulness, I felt the immediate
need was to turn the magnifying glass back on the research itself. Literature Review Meditation Mindfulness in Buddhism Essential features of Buddhist concept of mindfulness Mindfulness in Psychology 1950's - Zen and yogic meditation Later Literature Mindfulness-based Stress Reduction Mindfulness-based
Cognitive Therapy Dialectical Behavior Therapy Acceptance and
Commitment Therapy mindfulness being one type of "A persistent effort to detect and become free of all conditioning, compulsive functioning of mind and body, habitual emotional responses..." Pekala (1987)
"meditation is the process of turning consciousness upon itself to develop attentional control of the processes and contents of consciousness"
"...involves training the mind to become attuned to specific processes or contents of consciousness" Shapiro and Walsh (1984)
"...a family of techniques which have in common a conscious attempt to focus attention in a nonanalytic way and in an attempt not to dwell on discursive, ruminative thought" Smith (1986)
a combined set of skills, that of developing physical calmness, focused attention, and acceptance of what is Defined: Different types of meditation may focus on different stimuli such as:
- breath, mental images, sense organs, feeling states, gestures, sounds, or ideas (Naranjo & Ornstein, 1971) Noble Eightfold Path, core teachings of Buddhism. Nyanoponika, 1962)
A higher quality of paying attention which we can all experience at some level of development and frequency, but is more strictly seen as a practice and attitude consisting of "Bare Attention in a purely receptive state of mind"
Must be cultivated to enable clearer form of consciousness.
Paying attention to what is happening, internally and externally, with no added content. Attention is directed toward: body, feelings, current state of mind, and contents of consciousness. It means "to adhere, in that moment, to the object of consciousness with a clear mental focus". In other words, there are no degrees of mindfulness. There is or is not mindfulness.
This is one of many approaches, and will become somewhat confused when quantifying mindfulness. Innate Mindfulness -
Always and already present within everyone. No "I" thoughts, all sense of observer absent. Awareness is undistracted from its own nature. Not cultivated, bur realized upon full cultivation of previous stages. can be experienced in degrees! Ethics In Buddhism, mindfulness is cultivated within context emphasizing compassion, nonharm. Realizing "the way things really are" (Nyanoponika, 1962; Rosch, 2007) In Buddhism, this means realization of realities of impermanence (death, but also moment to moment "self"), egolessness and nondual existence (we are not separate from others), and suffering This understanding of what mindfulness reveals will be seen
differently in Western concepts of mindfulness. Even earlier with Jung and meditative symbols. In '50s and '60s, the Consciousness Movement becomes significant force in psychology, including huge interests in Eastern philosophy, as well as altered states, psychedelic drugs, hypnosis, dreams.
Transcendental Meditation and Zen are popularized. Major research appears in
'70s and '80s In the '70s and '80s, major psychological texts on meditation emerge:
Naranjo & Ornstein's (1976) "On the Psychology of Meditation"
- identifies techniques and common elements across different meditation forms.
- attempts to build common language to bridge East/West.
- investigate impact of meditation on human nervous system. Baer (2003); Kabat-Zinn (2003)
Research in 20th century has been severely flawed, and mostly descriptive.
However, enough evidence to suggest that mindfulness has potential utility to justify better studies.
Future studies must reveal mechanisms of change, determine efficacy in treating mental illness. Quantitative measures are of course a major step in this process. Previous studies have also focused on meditation generally. More recently "mindfulness" specifically is of interest. Western definitions and perspectives. Kabat-Zinn (1994)
"paying attention in a particular way, on purpose, in the present moment, nonjudgmentally" Germer (2005)
In psychology, mindfulness can refer to "a theoretical construct (mindfulness), a practice of cultivating mindfulness (such as meditation), or a psychological process (being mindful). Current formalized interventions 8-week program focusing on developing self-regulatory skills. Modeled after MBSR by Segal, Williams, & Teasdale (2002)
Developed as a follow-up intervention to CBT treatment of depression to prevent relapse.
Focuses on seeing thoughts as "just thoughts" not statements about reality, and for early detection and intervention of mood deterioration through heightened awareness.
Responsibility on clients for results. Developed by Marsha Linehan (1993)
Created to treat Borderline Personality Disorder.
Now applied much more broadly.
"Dialectic" refers to reconciling opposing goals of accepting clients as they are and attempting to change them.
Mindfulness is a major component of the "accepting" side of DBT.
Mindfulness "what" and "how" skills of observing, describing, participating, nonjudging, focusing on the present, and being effective are taught by therapist. Developed by Hayes, Strosahl, Bunting, Twohig &
Wilson (2004) with aim of increasing psychological
Assumes that language is at core of many
Thoughts and language are controlled through logical
language in therapy.
Six core processes of ACT: acceptance, diffusion, self
as context, contact with present moment, values, and
Mindfulness in integrated into many of these
components. Mindfulness Measured This study reviews eight quantitative measures of mindfulness, all available at the time of writing Summated Rating Scales Use multiple items, added to obtain total score.
Each item measures specific psychological dimension on a continuum.
Items do not have correct answers and cannot assess factual knowledge.
Items are statements applying to the subject. Reliability and Validity Advantages:
- good reliability and validity
- inexpensive and easy to make
- short assessment time Disadvantages:
- minimum level of literacy required
- requires good grasp of statistics Reliability refers to the scales' ability
to measure consistently. Inter-rater reliability
How well do different scorers arrive at
similar results (great for summated rating scales)? Test/Retest reliability
Consistency of measurement over time Mindfulness is expected to change over time based on context and variations in practice Internal Consistency reliability
Concerned with homogeneity of items comprising a scale. Test items should be assessing the same thing(s). (Spector, 1992; DeVellis, 1991) Items should correlate well to each other and to an overall construct.
Coefficient alpha is measure of intercorrelation.
Increased with better intercorrelation of items as well as larger number of items.
.70 is generally accepted as minimum acceptable alpha score for internal consistency. (Nunnaly, 1978) Validity refers to the extent to which a scale assesses what it purports to measure. Content validity refers to the extent to which items reflect the construct being measured. Criterion-related validity involves comparison of test scores on other scales Concurrent validity involves administering additional scales in order to check relationship of constructs. Predictive validity involves comparing data as well, but other data collected at a later date. Construct measured expected to correlate with other factors (eg, mindfulness with increased well-being, decreased stress and psychological symptoms) Construct validity checks for a theoretical relationship between variables Convergent validity exists when 2 scales measuring the same construct are highly correlated. Discriminant validity exists when 2 scales
expected to differ significantly do not correlate or correlate negatively. Mindfulness Scales Freiburg Mindfulness Inventory (FMI; Buchheld, Grossman, Walach, 2001; Walach, Buchheld, Buttenmuller, Kleinknecht & Schmidt, 2006) Mindfulness as "the moment-to-moment attentional, unbiased observation of any phenomenon in order to perceive and to experience how it truly is, absent of emotional or intellectual distortion". Devised for use in meditation retreats; relatively experienced meditators. A "trait" measure. Specifies specific span of time to consider when answering items (e.g. the last 2 weeks). Acceptable reliability and validity with experienced meditators. Some items too "difficult" for those foreign to mindfulness. Single-factor construct. Short version developed for broader use, but better scales are available for this purpose. Concerns:
- Reliable/valid for experienced meditators only.
- This is questionable also Reliability/validity testing based on pre-test/post-test comparison before and after an intensive retreat.
Practitioners self-assess upon arrival and at the end.
Daily activities varied considerably prior to retreat and in retreat. Confusion between a trait, the developed capacity, as well as the context, the content of one's daily activities and presence of outside stressors. Mindful Attention and
Awareness Scale (MAAS; Brown & Ryan, 2003; Carlson & Brown, 2005; MacKillop & Anderson, 2007) Defined: “an enhanced attention to and awareness of current experience or present reality”. conspicuous absence of nonjudgmental quality A "trait" measure. Single-factor construct. Extensive reliability/validity testing; generally positive. Concerns:
Some difficulty discerning those with or without
a mindfulness meditation practice.*
Mindfulness incompletely defined. Nonjudgment not
assessed. Significant validity issue. Kentucky Inventory of
Mindfulness Skills (KIMS; Baer, Smith & Allen, 2004) A "trait" measure. Skills-based approach to mindfulness a la DBT and ACT; developed for research into effectiveness of specific mindfulness techniques within these therapies. Concerns:
Skills-based approach in tune with DBT/ACT risks portraying mindfulness as something mentally ill lack and "normal" people have? "doing" more than "being" emphasized.
"Mindfulness experts" recruited for initial test item review consisted of 5 psychologists and 6 graduate students trained in DBT. Construct defined according to mindfulness skills of "observing" "describing" "nonjudgment" and "act with awareness". Four-factor construct same as defined skills. Solid reliability/validity testing; internal consistency testing performed separately on each skill. Mixed results correlating KIMS with meditation experience. Cognitive and Affective
Mindfulness Scale – Revised (CAMS-R; Feldman, Hayes, Kumar, Greeson & Laurenceau, 2007) A "trait" measure. Mindfulness defined as having four components: ability to regulate attention, an orientation to present or immediate experience, awareness of experience, and an attitude of acceptance or nonjudgment toward experience. Designed to be quick and easy; short 12-item scale.
Due to length, single factor scale despite four separate components clearly defined. Concerns:
Suitable reliability/validity testing done only on college student population.
Too short to interpret individual components of mindfulness separately. Toronto Mindfulness Scale (TMS; Lau et al., 2006) Mindfulness defined as consisting of “intentional self-regulation of attention to facilitate greater awareness of bodily sensations, thoughts, and emotions”.
as well as a unique quality of attention where the individual attempts to closely observe whatever comes into awareness.
Mindfulness is also characterized as "open, accepting, and curious." This last addition is the undoing of the scale. A "state" or "mode" based scale. (the only one). Administered following a mindfulness meditation session. 2 factors: Curiosity and Decentering Constructed from beginning to end by people with expertise in mindfulness. Expert review not limited to item ratings. Concerns:
While much of reliability and validity testing looked positive, some weak correlations with Curiosity subscale.
Questionable whether curiosity belongs in a definition of mindfulness. Philadelphia Mindfulness
Scale (PhlMS; Cardaciotto, Herbert, Forman, Moitra & Farrow, 2008) Mindfulness defined as “the tendency to be highly aware of one's internal and external experiences in the context of an accepting, nonjudgmental stance toward those experiences”. A "trait" measure. Explicit mention of "present moment" awareness not made Two factor construct: Acceptance and Awareness. Concerns:
2 items correlate poorly with other items and larger construct. Unclear why kept.
Most validity testing done on Acceptance factor. Surprisingly little done on Awareness factor, with no explanation.
Unclear how scale correlates to mindfulness meditation experience.
Items on each subscale are either all positively worded or all negatively worded. Impact of this is unclear. Southampton Mindfulness Questionnaire (SMQ; Chadwick et al., 2008) Mindfulness defined around 4 bipolar constructs:
- awareness of mental events in wider field of awareness versus lost in reaction to mental events.
- keeping attention on difficult thoughts versus avoidance.
- acceptance of self and uncomfortable thoughts and images versus judgment of self and thoughts.
- letting thoughts pass without reaction versus rumination and worry. Scale developed for use with patients with psychosis. A "trait" measure. The only trait measure which appears to look at 'degrees' of mindfulness rather than frequency of mindful moments. Concerns:
While there are signs of good reliability, validity is concern.
Does not correlate well with meditation practice.
Unclear how similar SMQ is to its more thoroughly tested predecessor, the MQ.
Authors declined to respond to requests for information. Five Facet Mindfulness
Questionnaire (FFMQ;Baer et al., 2008; Baer, Smith, Hopkins, Krietemeyer & Toney, 2009; Lykins & Baer, 2009; Van Dam, Earleywine & Danoff-Burg, 2009) Mindfulness is initially defined as “bringing one's complete attention to the experiences occurring in the present moment, in a nonjudgmental, accepting way”. A "trait" measure. FFMQ constructed by combining all previous mindfulness scales, administering and factor analysis for more comprehensive mindfulness scale. Five factors: Observe, Describe, Act with Awareness, Nonjudgment, and Nonreactivity to inner experience. Longest scale at 39 items. Extensive reliability and validity testing, results appearing significantly stronger than older scales. Correlates with mindfulness meditation experience. Concerns:
Differential Item Function - items may be understood differently by different groups, thus answered based on different criteria.
**This issue is identified with FFMQ, but most likely occurs with other mindfulness scales and simply has not been looked for. Easily the strongest of the trait-based measures so far. Summary: FFMQ likely to be dominant scale going forward due to its comprehensiveness,
its more thorough (and strongest) reliability/validity testing, and based on the
fact that it is built on the strengths of its predecessors.
Other scales retain potential utility:
- Freiburg Mindfulness Inventory has strengths due to unique intent for use with mindfulness retreats and experienced meditators.
- Cognitive Affective Mindfulness Scale - Revised may be preferred if frequent, short administrations are needed.
- Toronto Mindfulness Scale is unique as the only "state-based" scale.
- The Southampton Mindfulness Scale is unique in that it focuses on
unpleasant experiences exclusively and is meant for people suffering from
psychosis. It is also the only trait-based approach that appears to measure
"how mindful" rather than "how often" mindful. Problems of
Quantification Are we measuring
mindfulness? Reasons to measure Alternatives to
Quantification Phenomenology Keeping clear the different
"types" of mindfulness FMI's pre and post-test reliability/validity studies. Category confusion. Measuring everyday mindfulness while defining mindfulness as something that is or is not there in any instant. All except TMS and SMQ do this.
Germer (2005) states that, if mindfulness is all or nothing, all components of mindfulness must be present to be mindful. Since we measure freqencies of each component in day to day life, we cannot ever know if and when these components are present at the same time. Also, would it not be more accurate to state that one cannot be more mindful than what the lowest scoring component indicates?
Is this nitpicking, or pointing out legitimate problems in measuring mindfulness and in the approach to measurement most authors chose?
State-based measures face the same issue, but to a lesser extent. At least they have a specific span of time to consider. Process of cultivation of
mindfulness is nonlinear. Kornfield's (1979) phenomenological study of mindfulness notes that the basic growth pattern in mindfulness meditation includes a cyclical process going from progression, to regression, to restructuring and integration, and progression again. This process is unpredictable, likely different
in every person. What happens to a quantitative measure when
an advanced practitioner scores in the middle of a regressive phase, while a relative newcomer has experienced significant growth only so far? Walsh (1977; 1980) notes that with a deepening practice, behavior patterns are eradicated, leading to increased perceptual sensitivity and awareness of previously unknown perceptions, judgments, etc. One can state truthfully that he is not being judgmental, only to find later that this was not the case. Practitioners of different experience levels may answer items similarly based on their own understanding, making them appear similar.
How they differ remains unseen to quantitative measures. Perhaps the experienced meditator is more mindful, or perhaps he had more problems to sort out. "Adequatio": the knower understans only what he is capable of knowing at a given time.
"Grades of significance": the same phenomena can have totally different meaning to different people. Do individuals with significant experience understand the test items the same way? Differing Epistemologies Walsh (1980) Consciousness disciplines arise from Eastern spiritual epistemology, where "normal" experience is viewed as suboptimal, delusional.
Western paradigms view "normal" experience as relatively accurate, and claims of higher levels of consciousness may be discounted entirely. Rosch (2007) observes that current mindfulness scales do not, in fact, measure mindfulness, based on misunderstandings of what is revealed through mindfulness practice. The "realities" revealed are that of impermanence, egolessness, suffering, nondual existence/interdependence/interconnectedness.
Existing quantitative scales do not acknowledge this. Ethical context Mindfulness traditionally taught in ethical context, emphasizing compassion.
Compassion is inevitable result of realization of egolessness and interconnectedness.
These are truths of the reality of our existence, but not available to positivist/quantitative investigation. Differential understanding
of items In psychology, measurement is seen as prerequisite for further study or applied use of technique. Until measurement, "prescientific" (Giorgi, 1970). However, this does not stop practice in reality.
With mindfulness, measurement efforts did not begin until nearly 20 years after the first applied uses. Given the effort involved in developing and publishing these interventions, there are indications that clinicians do not require quantitative verification to confirm their own experiences and successes. Identification of mechanisms and causes of behavior and change. Assumption that human behavior can be explained by single cause (Rychlak, 2000). We shape our world even as we are shaped by it. We are not simply acted upon by outside forces (Fischer, 2001). Isolating a mechanism of change is perhaps not so simple. Elucidation of underlying constructs that
account for behavior. Bugental (1963) states that the basic unit is the complete person. Smaller reductions are abstract constructs.
Cannot control variables and isolate phenomena of interest in a vacuum.
Statistical methods hide ambiguities, individual uniqueness, leading to "false precision". (Fischer, 2001) Phenomena must be observable, thus operationalized. Confirm psychology as "real science" Must be defined according to how to measure it, essentially redefining. In fact, the only thing observable is is the subjective self-report. Everything derived from that, so its somewhat an illusion. Feyerabend (1978)
Faith in single immutable methodology provides sense of security and clarity.
Also restricts what qualifies as data and how to study. Giorgi (1970)
This ideology allows one to mask problems when
they arise (remove as error variance). Modern science not nearly as perfect as it would have us believe. Shapiro & Walsh (1984) "Meditation"
- compilation of most significant scholarly articles and research studies up to that time, covering impact on stress, hypertension, addiction, and comparison with other relaxation strategies.
- phenomenological studies introduced.
- therapeutic effects reviewed Smith (1986) "Meditation: A sensible guide to a timeless practice"
- tries to separate the practice from its spiritual/religious traditions, which serve to distract by setting up expectations that impede practice. Meditation is seen at this time as a path to God or a way to sedate or dissociate.
- Book presents spiritually-agnostic meditation instructions, then presents guide to setting up a meditation training program. West (1987) "The psychology of meditation"
- reviews the research on meditation: comparisons to other relaxation
methods, impact on personality, impact on the brain via EEG, critique of
quality of phenomenological studies
- Practical guide for meditation: when to do it, what it helps with, limitations
of, mechanisms of change. Natural science method is not the only way to credible knowledge (Feyerabend, 1978). Need to broaden what we call "data" and be flexible about methodology. Is a methodology that disavows that which cannot be directly observed the best way to investigate mind/consciousness (Halling, 2008)? The method must be appropriate to the phenomenon of interest (Frattaroli, 2001). "Natural science methods are
not appropriate for internal phenomena" (Giorgi, 1970). Why? Studying a practice which purports to raise consciousness to a level above what we call "normal" (Walsh, 1980). Benefit from including expert practitioners in research from start to finish (Kabat-Zinn, 2003; Walsh, 1980). No reduction to speculative "parts". Stay with the whole person (Bugental, 1963). Focus on lived experience and meaning using close description (Halling, 2008). The same lifeworld of interest is also the basis for natural science investigation, though this is covered up typically. It remains scientific and rigorous while respecting ambiguity and complexity of human experience (Fischer, 2001). Description is an appropriate and adequate approach to the study of mindfulness (Walsh, 1980). The Phenomenology of Consciousness Inventory (Pekala; 1982; 1991b) Built upon Pekala's (1991a) psychophenomenological method. Based on elements of consciousness derived from description, then applies traditional psychological and statistical methods to assess and quantify those elements. Quantified self-report scale. Assesses intensity of various aspects of consciousness. Attempt to apply best of both worlds to
consciousness studies. Does the assessment retain individual complexity and differences? 53-item "state-based" scale. Administered following a specific stimulus condition which is used as basis of assessment. Concerns: Scale designed for study of hypnosis. Useful for mindfulness? Limited ability to conduct known-groups validity test due to test's uniqueness. Needs new reliability/validity testing for each state being studied. Nothing for mindfulness yet. Scale assesses specific dimensions of consciousness, which appear psychometrically stable. The several 'subdimensions' are less internally consistent. Method by Nathan Lorentz, M.A. Administer a selection of mindfulness scales to experienced mindfulness meditators. Conduct qualitative interviews to solicit their experience of and opinions about the scales based on their own knowledge, training and experience. Three scales:
- Five Facet Mindfulness Questionnaire (FFMQ)
- Toronto Mindfulness Scale (TMS)
- Phenomenology of Consciousness Inventory (PCI) Approximately 8 sought, ideally diverse in mindfulness training and other demographics NOT an expert review, just review by those with mindfulness experience and a more than superficial understanding of the practice. Consent forms will be obtained with option to back out at any time for any reason. Interviews will be audio recorded and transcribed, with both kept securely. Interviews Focus groups will be arranged when possible to facilitate discussion among participants in addition to simply answering questions. There will be prepared questions asked of all participants, though it is expected that additional questions may arise within the flow of the interview. Data Collection 2 sessions minimum:
- One to have participants meditate for 20 minutes, followed by administration of the mindfulness scales.
- The second, within 2 weeks later, to conduct interviews/focus groups.
- Potential follow-up interviews for additional data gathering if necessary and possible. Data Analysis
(Qualitative Research) Common Elements Conducted in natural setting.
Researcher is instrument of data collection who analyzes data for meaning.
Data often collected through varied sources (observation, interviews, personal experience, historical data, media)
Attempt made to respect the full complexity of the unreduced phenomenon. Reasons to Use Depends on research question or problem.
Richer description of phenomenon of interest.
"How" and "What" questions as opposed to "why".
When data cannot appropriately be collected in lab setting. Major Types of
Qualitative Methods Biographical Life
History In-depth study of a single individual. Data consists of collections of stories about the individual. Focus on numours significant events in the person's life. Significant focus on setting and social context. Researcher/author is part of the story, sharing experiences that occur during research. Phenomenological
Study Attempt to understand a phenomenon more fully by identifying essential themes and features. (Creswell, 1998; Halling, 2008) (Creswell, 1998) Many examples of the phenomenon are investigated in great detail to identify common features. Begins with exploration of philosophical ideas. Then examines how phenomenon is experienced in concrete situations in the world. Effort made to identify and set aside preunderstandings and biases prior to investigation to be open to what is unexpected. Data analysis serves to reveal a set of significant statements which enhance understanding of the essential structure of the phenomenon and the meaning it has for individuals. Grounded Theory (Creswell, 1998) Focus is on theory building, solidly grounded in qualitative data. Data gathered through interview, observation, and focus groups. Data is analyzed by coding significant statements and organizing into meaningful categories, culminating in a visual model with a central theme. Result shows differing relationships between different themes and categories. Final schema represents the phenomenon studies as occuring in a specific situation in the world. Ethnography Case Study (Creswell, 1998) (Creswell, 1998) Describes in detail and interprets a cultural group, system, or other group engaged in a specific activity. Typically told in story form describing everyday lives of subjects involved. Heavy emphasis on depicting the context. The culture of the group is elicited, including roles, expectations, rituals, and other behavior patterns. Data consists of interviews, physical evidence, and extensive prolonged observation of the group at work. Creates a detailed description of a single event in its context. Provides a chronology of events, analyzing data to discover important themes which emerge. The case is included within specific boundaries of time and place, and that entire context is described thoroughly. Data collected from many sources, including observation, interviews, media, historical documents, etc. Analysis of data reveals themes, which are tied together and related to relevant literature. Method for the Present Study The present study does not easily fit standard research questions, seeking to solicit opinions and reactions to mindfulness scales. Grounded Theory and Phenomenology, regardless of their traditionally intended uses, most closely fit requirement of this study. Interview data will be gathered and analyzed by thorough coding/categorization according to major points as significant for the purpose of the study. Codes will be organized into a hierarchy for meaningful and organized presentation of relevant feedback. There is no attempt at theory-building, however. Initial biases and expectations should be identified and set aside or at least kept in awareness. Potential biases/expectations:
Participants may favor state-based scales.
Participants may observe that people will understand items differently according to their level of mindfulness experience. Result of suspension:
Participants noticed significant flaws in the Toronto Mindfulness Scale which I did not.
Participants also offered unforeseen reasoning to favor trait-based assessment. The experience of participants while filling out the scales, unrestrained by fixed items, will be sought. A set of interview questions is planned in advance, with the possibility for spontaneous questions to follow from the discussion. Results Based on test items, how well do the scales capture mindfulness, based on your own understanding?
Based on your understanding of what mindfulness is, which measure did you prefer and why?
How might attempts to measure mindfulness be improved?
The Toronto Mindfulness Scale and Phenomenology of Consciousness Inventory assess a state (or mode) of being, while the Five Facet Mindfulness Questionnaire looks at more stable, trait-like characteristics. What opinions do you have about the different approaches to mindfulness assessment?
What did you like about the Five Facet Mindfulness Questionnaire?
What did you dislike about the Five Facet Mindfulness Questionnaire?
Given your score in comparison to mean scores in various samples, how accurate did the FFMQ assess you in your opinion?
What did you like about the Toronto Mindfulness Scale?
What did you dislike about the Toronto Mindfulness Scale?
Given your score in comparison to mean scores in various samples, how accurate did the FFMQ assess you in your opinion?
What did you like about the Phenomenology of Consciousness Inventory?
What did you dislike about the Phenomenology of Consciousness Inventory?
Given your score in comparison to mean scores in various samples, how accurate did the Phenomenology of Consciousness Inventory assess you in your opinion?
Is there anything else that you would like to add regarding your experience with the scales, your results on the scales, or about the assessment of mindfulness or this research project which has not yet been covered? Appendix Prepared Interview Questions Eight participants were recruited Participant # Age Gender Tradition Experience
2 29 M Zen 10 years
8 57 M Zen 38 years
10 46 F Zen 16 years
11 28 F Shambhala 5 years
12 27 F Mix (Buddhist) 5 years
13 26 M Vipassana, Zen 3 years
43 23 M Psych. Intervntn 6 months
56 25 F Not Sure 18 months Data Collection Session One 20 minute meditation followed by administration of FFMQ, TMS, and PCI. Between Sessions Tests scored, results prepared for discussion, if any. No more than 2 weeks passed between sessions. Session Two Interviews conducted individually for two participants. Focus groups consisting of two participants each for the remaining six participants. Much of data collection occurred between semesters, making students less available. Schedules were quite restricted even with school in session. Transcription and Coding All interviews were audio recorded and later transcribed. Transcriptions were imported into NVivo Version 8 software for coding. Coding occurred in two phases to ensure adequate rigor. No assistants were available to code the same data. This would have allowed differences to be discussed and reconciled. Instead, the author performed coding twice and reconciled differences that arose. With the completion of "free coding", or the assignment of basic labels for relevant statements, codes were grouped within relevant categories for structure. Unlike typical Grounded Theory, categorization of themes had more to do with meaningful presentation than with theory-building. Outcome Data Overwhelming preference for
Five Facet Mindfulness Scale Five participants provided opinions, all in favor of FFMQ. One initially chose the Toronto Mindfulness Scale, being critical of FFMQ's trait-like, day to day approach.
Her mind changed, deciding the day to day approach reflects the results of a mindfulness practice, or "the why" of practicing. FFMQ praised for its comprehensiveness. Some generally appreciated the trait or day-to-day approach to measurement. The TMS was specifically disliked for its focus on "curiosity". FFMQ Feedback Strengths Concerns Others noted that, given the day-to-day method of assessment, ability to put things into words may be okay and even good; but in meditation itself, you would want to avoid this. (2) Easy to answer simply to look more mindful, but this may miss the complexity of mindfulness. (1) "...there's something to be said about being able to catch yourself finding it difficult to stay focused...That's mindfulness too" (Subject 12, Transcript 4, p. 3) In the FFMQ, it is indicative of mindfulness to feel calm soon after a distressful experience. (2) Participants state that "mindfulness isn't exactly about relaxation or feeling calm, but more about like being able to notice my thoughts, and if mindfulness might make me a little more agitated or anxious...that's fine because it helps me like actually pay attention to what's going on" (Subject 56, Transcript 2, p.2) De-emphasis on "results" Scoring Comments Expectation of greater difference in scores between someone with little experience and someone with extensive experience. (2) "I would just expect for somebody who's so much more experienced...to be...more above average" (Participant 11, Transcript 5, p.14) Echoed by Participant 56, who felt her score was higher than she would expect given her relatively little (18 months) experience. Possibly reflecting that the scale's range may be relatively narrow, from completely naive or mentally ill to someone who has been taught some basic mindfulness skills, finished short term seminar/training, or not mentally ill. Beyond that, not much sensitivity to higher levels (epistemelogical issue?) Nonjudge scale's score lower than participant thought maybe it should be. (1) Appears accurate. (2) Comprehensiveness (specific attention to thoughts, feelings, sensations, external events, attention, judgment) (8) Face Validity; looks like a mindfulness scale (1) Dislike 5-point Likert scale; "I feel like it could be really easy just to be completely neutral". (1) Items inquiring into ability to put experiences into words may distance practitioner from present-moment experience; equivalent to labeling, categorizing, even judging. (3) TMS Feedback Strengths Accessibility. Participants found the Likert scale easy to use and the items easy to understand, and they appreciated that the scale was brief. (4) Comprehensiveness. Participants appreciated the scale's inquiry into thoughts, emotions, and sensations, as well as noticing feelings without identifying with them. (3) "This one was the easiest to answer. It was the most straightforward for me" (Participant 12, Transcript 4, p.7) Concerns Potentially too few items to be reliable. (1) Curiosity as an aspect of mindfulness. Six participants felt that to be curious suggests a pursuit of what enters awareness, investigating, analyzing, and generally taking one away from present moment experiencing and the act of noticing and letting go. One participant understood "curiosity" as "beginner's mind" or unknowing, and did not have a problem with its use in the TMS. These concerns were severe enough to dismiss the TMS as an appropriate mindfulness scale altogether in some. "That's something that I was really reacting to in filling these out, was feeling like, 'Ah! This isn't...what I do'" (Participant 10, Transcript 3, p.9) Scoring Comments Scores appear lower than they should be. (2) "That's interesting. I've been doing this for kind of awhile now" (Participant 8, Transcript 3, p.11). Scores appear to accurately reflect where they are, observing that their scores were low. (3) Also not unexpected, as TMS assess what participants consider not rightly part of mindfulness. PCI Feedback Strengths Concerns Appreciation of the potential ability to compare different states of consciousness. (1) Scale's attempt to be sensitive to changing of awareness to stimuli through meditation. (1) The scale gathers more information than the specific requirements to fit the definition of mindfulness, which could be informative in unforeseen ways. (3) Scale felt repetitive, due to highly similar items. (2) "I felt the most self-conscious...around this one, because I was very aware that I was being asked the same thing multiple times, and I fel like I wasn't...sure what my answer was..." (Participant 12, Transcript 4, p.9) Lack of clarity about what items asked about. (1) Item "I was silently talking to myself a great deal". "I rememeber that one I didn't really know how to answer, because I wasn't really sure what they meant by 'talking'. ...I wasn't talking to myself, but I had thoughts" (Participant 56, Transcript 4, p.10) Content of items not sufficient to assess requirements of mindfulness, and many items unnecessary to do so. Participant 2 in Transcript 5 noted that "to me the measure of mindfulness would be more like, how...aware of these things I really was, so, for example, [item] 24 'conceptually my thinking was clear and distinct'/'conceptually my thinknig was confused and muddled'. The measure of mindfulness is how transparent that is to me" (p.18) The PCI inquires about the presence of certain types of thoughts. To have sexual thoughts, for example, does not negate mindfulness; being unaware of them might. To some extent, if less so, this critique may apply to other scales as well. Likert-scale Comments The PCI has a unique item scoring method, with bipolar statements representing extremes on the scale; participants select the number (0-6) that fits between these extremes. Four participants preferred this unique scoring method. Three participants had a strong dislike for it. Two who liked the item-scoring noted that they tended to answer items either at one extreme or right in the middle. "I noticed too that I was either always at one end or the other, or if I wasn't, I was dead center" (Participant, Transcript 5, p.16) Suggestions for
Mindfulness Assessment Scales should attend more to the tendency people have to judge themselves, as well as to the need to let go when judgments and other thoughts are noticed. (1) Possible incorporation of behavioral data (2). No ideas how; one participant firmly disagreed with this suggestion. Assess technique, not achievement. (1) Participant 2 in Transcript 5 stated that the focus on achievement or goals is "a complete mistake. ...I never had any of my teachers talk about stuff like this. Whenever I would ask them a question, they would refer me always back to technique" (p.9). Begin questionnaire with "overall question...about how effective were you doing this, because some days were not effective even though we try" (Participant 56, Transcript 2, p.6). Trait vs. State Assessment The FFMQ (and many other mindfulness scales not administered) assess mindfulness as a relatively stable, day-to-day construct, though they define mindfulness as something there or not there in any instant. The SMQ is a trait-based scale also, though it treats mindfulness as something that is there in "degrees". The TMS, however, as well as the PCI, assess a state or mode of being, a purposeful activity. Availability of both types of measures allows for a more complete understanding. (4) "...mindfulness is such an open ended construct...the person reading it might be more interested in the trait-like [approach], where another person would be more interested in this [state-like approach] and so would have as a whole...[more] validity, and just emcompass a lot more" (Participant 43, Transcript 1, p.2) Trait and State measures useful for different questions. (1) Concern about variability in state-based measures. (5) Participant 10 stated in Transcript 3 that "if you want to measure someone's mindfulness, you don't maybe want to just focus on one particular 20 minute episode in their lives" (p.16) There appeared to be some self-consciousness among some participants about how well they did during their mindfulness session, revealing significant concern about being "mindful enough". Trait measures more relevant. (2) Participant 12 in Transcript 4 stated that "it's certainly interesting...to look at the specific time period, but...the whole point about meditating isn't to have a good sitting session on the cushion for 20 minutes. The whole point is so you can get off the cushion and live your life more mindfully, and...find ways to relieve suffering in yourself and to be a better person for other people" (p.4) There is significant concern about the end result, or the "why" of meditation. Concerns about Quantification Elusiveness of a single authoritative
definition of mindfulness. (4) Different mindfulness traditions have different focus.
"You are absolutely shaped by the discipline that you are taught and that you follow every day" (Participant 8, Transcript 3, p.9) Two participants felt that varying definitions of mindfulness are the biggest problem with assessment of mindfulness. Ability to Quantify
Mindfulness. Five participants noted in some way that we can only report based on what we are aware of. (5) If we slip into unawareness, it is difficult to be accurate about how much time passes. "What we're aware of and what's actually happening, even in something this simple...could be two very different things" (Participant 10, Transcript 3, p.16). A scale might indicate more mindfulness if one states that he or she is not easily distracted, but it is possible the person does not notice distractions. Self-report scales make it easy for people to be swayed to answer according to what makes them look more desirable. Participant 10 in Transcript 3 observed that "to some extent someone's answers easily might be contaminated by how closely they think...'Okay, this is how mindfulness meditation is supposed to work'" (p.4). Utility of Assessment. Significant difficulty comparing different people, or achieving significant or predictable gains in one's own practice over time (2). Participant 2 in Transcript 5 stated that "its hard to compare someone's baseline to anything. I could tell you what I thought maybe I was like 10 years ago, or what I think I might be like in 10 years, but it also takes a very long time to...notice the fruits of practice in your life and you might not necessarily" (p.7). "You can perhaps gain an increased capacity in one way or another, but you can't predict that. It varies from person to person" (Participant 2, Transcript 5, p. 9). Measurement Implies Judgment. (4) Participant 12 in Transcript 4 stated that "I'm noticing myself having an aversion to the measurement of it altogether because it's...not the point. ...I can see it being helpful to other people who want to study it, but as a practitioner, to be encouraged to judge your practice is...the opposite of what we're working on" (p.8). Participant 2 in Transcript 5 disliked the term "mindfulness", which "made me feel like there was an experience or a state that I was supposed to shoot for.... And that took me away from my ability to stay present with what I was actually doing, or what was actually happening. So...if mindfulness was...described as a certain...experience, chances are I wasn't having that. So to try to have that was to be detrimental" (p.8). By striving for scientific certainty through measurement, do we rob mindfulness of its value or ability to help? Discussion After several decades of interest and research, mindfulness as a topic for psychology is maturing, with many books summarizing research, formalized interventions, and at least eight assessment tools. There have been recent calls to improve the research in this area to take us beyond descriptive results into explanation, insight into mechanisms of change and underlying constructs. Eight recently developed quantitative assessment tools have been developed to help in this area. The purpose of this study was to review these recent efforts and reflect on the direction research is going as well as the adequacy of these measures to assess what they purport to assess and their ability to contribute to the field. Review of Purpose Existing Mindfulness Scales The Five Facet Mindfulness Scale is the strongest scale psychometrically, and the most well-liked amont research participants in this study. Built on the strengths of its predecessors, it appears poised to be the predominant mindfulness scale going forward. Some concerns exist, such as whether the ability to put things into words represents a departure from present-moment experience. This reflects a conflict between a liking for the trait-based approach, while a hesitancy to let go of the process-nature of mindfulness. Another possible concern is the ceiling for scoring may be relatively low, with little difference between someone practicing for several months or years and someone with decades of experience. The Toronto Mindfulness Scale is the only state-based scale, and therefore the only mindfulness scale requiring a meditation period prior to assessment. While the different approach was appreciated, participants almost universally disliked the TMS due to its emphasis on "curiosity", which they feld took practitioners away from the present moment and away from mindfulness. The authors themselves acknowledge this to some degree, indicating that psychology-based mindfulness interventions may encourage reflection more, though they are inconsistent in that they did not expect the TMS to correlate with psychological-mindedness as it did. This approach has potential, but may require re-working. The Phenomenology of Consciousness Inventory is not a mindfulness scale, but was introduced to investigate its potential utility to study mindfulness. Participants generally felt that it had insufficient items to assess mindfulness and many additional items considered unnecessary to do so. Extra items may be useful if the scale is to compare different states of consciousness, but more mindfulness-related questions seem necessary. Inquiries into the presence or absence of certain types of thoughts, such as sexual thoughts, are seen as irrelevant, while awareness of these thoughts and acceptance is imporant. Potential utility of other mindfulness scales not administered in this study (FMI, SMQ, PhlMS, CAMS-R, etc.) Trait vs. State Issue Some participants believe having both provides more complete data.
Others feel that one or the other would be used depending on the research question.
Still others feel the trait-based approach may be best, though part of the reason is based on a probably mistaken assumption that only one or two assessments would be made for any one person. Authors view: It is not easy to say, because an a good example of a state-based scale does not exist (yet). A modified TMS, shifting to, for example, openness, and away from curiosity, might be a good start. Judgment in Measurement Evaluating "how mindful" is tantamount to evaluation, judgment, and harm to one's mindfulness practice. May explain anxieties voiced by participants about their low scores on the TMS because of poor meditation session. Concern about measuring up. Does this indicate participants were already removed from what mindfulness is about?
As the FFMQ is a trait-based measure, these anxieties did not show. Is that a different problem? Do people filling out scales not consider their "bad days" or less proud moments when filling out the scales? Suggestions for Improvement More attention to noticing judgments and distractions and then "letting go". Focus more on technique than achievement. Author's view: Preference for the state-based approach. Some participants favored trait-measures but these opinions are guided in part by a flawed state-based measure. State-based assessments will reflect specific sessions, both good and bad, and say more about the session than the person if administered sensitively. The scales could genuinely be used to assess technique.
This would likely appear less useful for research into underlying constructs and mechanisms of change, but would be less harmful for practice.
Finally, the emphasis would remain on process.
Though some participants favored looking at results, they also had difficulty letting go of the focus on process and practice. Concerns with Assessment The word "Mindfulness" Nonlinear growth in
mindfulness practice (Kornfield, 1979; Smith, 1986) Mindfulness practice leads to periods of progression, regression, integration, and more progression (Kornfield, 1979). Smith's (1986) "meditation deepening cycle": repeating and alternating phases characterized by increased calm, focused attention, and letting be, then increased physical tension, distraction, and greater effort required to be mindful. These phases are necessary for growth. Though there are declines, these "declines" pave the way for further growth and represent increased sensitivity and self-awareness, goals of practice. No recognition of this in assessment literature. Awareness of Awareness Mindfulness scales tend to inquire into individual awareness of external and internal phenomena (thoughts, feelings, judgments) as indications of greater mindfulness. To what extent can we report on that which we are unaware? Growth in practice generally consists of gaining awareness of previously unknown levels of judgments, feelings, beliefs. This suggests that there are layers upon layers of distraction, judgment, thoughts that we are incapable of attending to as we are now, and thus cannot report on them. Perhaps mindfulness scales are sensitive within a single "deepening cycle", sensitive to greater awareness, but not to awareness beyond what we call "normal". This may lead mindfulness scales to show highly experienced meditators to appear only subtly different from new practitioners. Necessary Components
of Mindfulness In most measures available today, mindfulness is defined as
a quality that is or is not present in consciousness at any time. If all or nothing, all components of mindfulness must be
accounted for at a minimum to consider the presence of a
mindful state. Most scales measure day-to-day mindfulness, yet inquire into
frequency of individual indicators of mindfulness. So, it does not follow that these conditions all occur in the
same moments. Also, if all conditions must be present, it makes sense that one
could not be more mindful than whatever is indicated by the
least frequently occurring element, rather than a sum of all. Despite how mindfulness is defined by the various authors,
what is measured seems to be different, more akin to "grades"
of mindfulness. Taken together, the researcher must concur with Rosch (2007) that these mindfulness scales are not really measuring mindfulness, but something similar in many ways. Ineffable, prelinguistic concept
(Hick, 2008). What is left out? Walsh (1980) and Rosch (2007)
on what it means to be mindful. In consciousness traditions, including those which developed mindfulness meditation, "normal" experience is suboptimal, delusional. In Western paradigms,
"normal" experience is considered to be optimal. To see "reality" according to mindfulness traditions is not to be free of mental illness or to be "normal" but to realize the ubiquity of suffering in existence. To discover the nondual nature of existence and our interconnectedness with others and the world, and thus to realize great compassion for all things. It is to recognize impermanence, not just in death but in every moment. To be mindful, in the highest or most pure sense, is to realize limitations of Western
positivist psychological assumptions. It is to realize that what we consider reality and normal is not ideal, nor is it reality. Mindfulness as intentional practice, or a psychological process. Mindfulness as all or nothing, or something which occurs in levels or grades. Easy to confuse these or start referring to one and then the other without making the distinction. Mindfulness scales do this, defining as one, but unable to measure it that way. This does not necessarily take away
their utility, but we should be clear about what we are doing. The word "mindfulness" implies a level of achievement one could have but probably is not. Potentially harmful to practice. May be useful still, for teaching the basics, or studying the initial benefits of a mindfulness practice or of teaching "mindfulness skills". I suggest that these limitations, to be honest, accurate, and transparent, should be clearly stated. The scales should be clear about what they measure, and what they do not. They should measure the same thing they define. The scales at present define mindfulness as "all or nothing" but may not measure an all or nothing construct. The scales, firmly constructed within the cognitivist paradigm, likely do not have validity beyond initial stages of growth in mindfulness practice. Alternatives to Measurement Limitations of this Study Measurement is often considered necessary in psychology to "understand"
a phenomenon, by discovering explanatory power and underlying mechanisms and constructs. This reductionist perspective, however, relies on largely hypothetical constructs and often takes a simplistic single linear causal model for change. There are many examples of useful knowledge arising without use of the natural scientific method. Uncritical use of natural science method can conceal problems in research design. Accepting the complexity of the phenomenon. Walsh (1980) warns against misunderstanding or distorting what is being studied through uncritical transport of a concept from Eastern ways of knowing. Respect the practice for what it is, even if it presents challenges or looks less scientific. He also advocates for collaboration, from beginning to end, with people who are competent with research and with the concept being studied, a "yogi-scientist". This hopefully prevents outside input from being carefully limited to "expert review" portions of a study. These points of advice serve to respect and retain the complexity of what is studied,
to do the work that needs doing, and avoid bias regardless of the method used. Different approaches Rather than advocate for a single method over another and make the research question or problem fit, ask: "What is the research question?" and then "What is the most appropriate way to investigate?" Phenomenology Is rigorous.
Is more adequate for mental (non-material) phenomena.
Acknowledges the complexity of the phenomenon, individual differences, context in which phenomena occur, and the difficulty of adequately establishing cause.
Can avoid dangers of inserting evaluation into a practice that necessarily must be non-evaluative. This dissertation was completed within significant time constraints. With more time:
- more participants could be selected;
- more and larger focus groups could have been conducted;
- follow-up focus groups could have been conducted to gather more data after initial analysis.
- a larger and more comprehensive literature review could have been provided;
- conclusions offered in the study could have been pursued with more depth. Research assistants were not available who could have served to "check" the researcher's data analysis to ensure the most correct conclusions possible were made. This study is therefore limited in a number of ways. However, the researcher hopes that it may serve to help other psychologists and researchers to more critically approach the study and application of mindfulness in the field. Rosch, 2007 Mindfulness is "a simple mental factor that can be present or absent in a moment of consciousness". Stages of growth Lama Surya Das (2007) Average Mindfulness - shortlived heightened attention which arises spontaneously Cultivated Mindfulness -
Developed through mindfulness practice. Intentionally directed toward object of interest to understand it more fully. Nonjudgmental, nonreactive. Abiding Mindfulness -
A well-trained mind can rest attention naturally, without distraction, for a long time, on anything. This is also realized in practice, via discovery of lack of separation between self and others. Hick (2008)
Caution toward psychology research efforts:
- Presence of some components does not imply presence of mindfulness.
- Mindfulness is a preconceptual and preverbal notion; cannot entirely be described in language. Developed in 1982 by Jon Kabat-Zinn for chronic pain. First formalized mindfulness intervention. Used worldwide. Now used for a variety of illnesses. So, are these operationalized definitions sufficiently encompassing what mindfulness is? Does the ethical component represent easily removable aspects of mindfulness unnecessarily added as a result of traditional religious contexts? A significant reason why a score is not important or helpful.