D-KEFS

Sorting Test

Airplane

Duck

Bus

Colour-Word Interference Test

Car

Tiger

Eagle

Verbal Fluency

• Measures concept-formation skills, modality-specific problem-solving skills (verbal/nonverbal), and the ability to explain sorting concepts abstractly
• Two conditions:
• Free Sorting
• Examinee needs to sort 6 cards (each with a word and picture) into two groups based on as many rules as possible
• Scoring based on the description of and sorting of responses
• Sort Recognition
• Examineer sorts the the cards, and examinee must identify the rules by which the examineer sorted them
• Scoring based on description of responses

• Measures inhibitory control and cognitive flexibility
• Utilises the “Stroop Effect” to identify deficits in inhibition of over-learned behaviours
• 4 conditions:
• Colour Naming
• Word Reading
• Inhibition
• Colour names are printed in a different colour - must say the color of the ink, not read the word
• Inhibition/Switching
• Runs the same as inhibition, but some words are in boxes. The examinee must read the word in the boxes, not say the colour of the ink
• Raw score is determined by the number of seconds that the examinee takes to complete each condition

• Measures multiple aspects of verbal behavioural productivity and cognitive flexibility.
• Evaluates effectiveness of novel and semantic search strategies, and assesses flexibility in the implementation of semantic search strategies.
• The process approach enables further evaluation of self-monitoring of information search, as well as difficulties related to initiation and sustaining effort.
• Uses 3 different conditions:
• Letter Fluency
• say words beginning with a specified letter (either F, A or S) as quickly as possible
• Category Fluency
• say words belonging to a designated semantic category (animals or boys' names)
• Category Switching
• Alternate between saying words from two different categories (Fruits/Furniture)
• Scoring: Raw score is the number of correct responses given by examinee and is converted into a scaled score

Ears

Mouth

Toes

HAT

Socks

Shoe

Design Fluency Test

• Measures multiple aspects of visual behavioural productivity and cognitive flexibility. It evaluates visual productivity with and without distracting stimuli, and switching between stimuli. The process approach is used to assess for difficulties with self-monitoring of performance for rule-adherence, and redundant responding.
• Rules are laid out in the stimulus booklet for the examinee to refer to
• There are three conditions where the examinee attempts to produce as many novel designs as possible using:
• filled
• empty
• switching between filled and empty dots
• Scoring: Raw score is determined by number of correctly drawn designs

Trail Making Test

The Subtests

• Applies the process approach to the assessment of cognitive flexibility
• Differentiates deficits in visual scanning, mental sequencing, and motor speed/control from deficits in cognitive flexibility and control
• 5 trials:
• visual scanning
• number sequencing
• letter sequencing
• number-letter switching
• motor speed
• Scoring: Time to complete each condition converted into scaled score and subjected to contrast measures

BackgrounD

D-KEFS

Twenty Questions

• D-KEFS (Delis Kaplan Executive Function System)

• Dean Delis, Edith Kaplan, Joel Kramer, 2001

• Assess executive function components (Dubartey, 2003)

• Executive function: control and regulation of cognitive processes, goal-directed behaviour, future-planning. Regulated by the frontal lobes. (Alvarez & Emory, 2006)

• Neurodevelopmental issues, degenerative disorders, traumatic brain injury, epilepsy with still more populations being tested

• Understand the profile of executive dysfunction. To develop strategies with personal strengths and weaknesses in mind

Proverb Test

D-KEFS

• Measures the ability to categorize, formulate abstract, yes/no questions, and incorporate the examiner’s feedback to formulate more efficient yes/no questions
• Examinee asks the examiner yes or no questions in order to identify a picture the examiner has chosen. The goal is to figure out which picture the examiner has chosen using the least number of questions
• Scoring: Abstraction score determined by number of objects eliminated by first questions. Overall score determined using abstraction score and number of questions used to get correct response

Word Context Test

Tower Test

• measures the ability to form novel, verbal abstractions
• Two conditions:
• Proverbs are read and examinee is asked to interpret orally
• Proverb is given, and examinee must choose from four options that best explains the proverb
• Scoring: Based on abstraction and accuracy of response

• Measures verbal modality, deductive reasoning, integration of multiple bits of information, hypothesis testing, and flexibility of thinking
• Given a made up word (in context) and must determine its meaning
• Given 5 sentences as clues for each item
• Scoring: Number of correct responses, incorrect responses, number of don't know responses, and repeated incorrect responses, quantified and weighted more according to deductive reasoning than accuracy in expressing words.

• Measures spatial planning, rule learning, inhibition of impulsive and perseverative responding, and the ability to establish and maintain instructional set
• Goal is to create the picture in stimulus book in fewest number of moves
• Examinee is given certain rules about how the discs can be used
• Scoring: Points for each item are scored and converted into a raw score according to number of moves used to create tower

About the D-Kefs

Norm population

• Normative sample of 1750 non-clinical individuals aged 8-89 who were nationally representative (composition was based on the US Census 2000. (Homack, Lee & Riccio, 2005)

• The 9 substests were designed to stand alone

• Domains such as cognitive flexibility, cognitive shifting, inhibition, problem solving, creativity, planning, reasoning, abstract thinking, concept formation

• Areas known to be sensitive to executive control dysfunction (Dubartey, 2003)

• Trail Making: Cognitive flexibility, Attention,

Visual-Motor skills

• Verbal Fluency: Fluent productivity (verbal

domain), lexical organisation

• Design Fluency: Fluent productivity (spatial

domain)

• Colour-Word Interference: Inhibition of verbal

response, Processing speed, Cognitive flexibility

• Sorting: Problem solving, Verbal + Spatial

concept formation, Cognitive flexibility

Domains

Norms

Test Norms

Traumatic Brain Injury (TBI)

Autism Spectrum Disorder

- Normative tables for education

and vocabulary level

(Fine, Delis & Holdnack, 2011)

• 20 Questions: Abstract thinking.

Visual attention, Object recognition, Naming

• Word Context: Deductive

reasoning, Abstract thinking

• Tower: Spatial planning, Response

inhibition, Maintaining instructions

• Proverb: Metaphorical thinking,

Verbal abstract interpretation

• Neuropsychological testing has contributed extensively to the rehabilitation of patients with traumatic brain injury (Uomoto, 2010)
• Tests in D-KEFS sensitive to brain damage (Homack et al., 2005)
• i.e. cognitive switching, inhibition, processing
• e.g. 16 rules in the Sorting test
• Repetition of instructions

Those with ASD have difficulty with Executive Functioning such as: paying attention to minor details but not seeing how they fit into a bigger picture; complex thinking that involves more than one train of thought; maintaining attention; or organising their thoughts and actions

Fluency Subtests

Comprehension of Indirect Requests

Lopez, Lincoln, Ozonoff, and Lai (2005)

Kleinhans, Akshoomoff, and Delis (2005)

Examined executive functions in ASD (HFA and Aspergers) in flexibility, fluency and inhibition using the Colour- Word Inteference Test, Trail Making Test, Verbal Fluency Test and Design Fluency Test

• Found that executive dysfunction in ASD was generally mild, with the most consistent deficit found on the measures of verbal fluency
• 10/12 participants were impaired on either letter fluency, or category switching fluency
• Consistently had difficulty only on complex verbal tasks that required generating and initiating efficient cognitve search strategies and problem-solving techniques

• Strong, C., Tiesma, D., & Donders, J. (2011)
• More research on verbal fluency tasks, less on nonverbal fluency tasks, and rare to examine both together in TBI.
• Compared mild-severe TBI to normal controls
• TBI patients performed statistically lower than controls on Verbal Fluency test, particularly on category-switching
• While classification rate was 65%, verbal fluency test regarded as a useful tool to include in assessment of mild-severe TBI
• However, useful to differentiate between moderate and severe brain damage (Finnanger et al., 2013)
• Limited discussion on why there is no discrimination on design fluency tests (length of time after injury?)
• Little discussion on low classification rate (warning not to use single test to evaluate)
• Other useful D-KEFS tests to consider: Sorting Test (Heled, Hoofien, Margalit, Natovich & Agranov, 2011).

• (Evans & Hux, 2011)
• A note on how D-KEFS scores might be related to day-to-day functioning.
• Adults with severe traumatic brain injury benefited from combination of verbal land gestural information when presented with an indirect request. For example, saying ‘Actually…I’m still pretty hungry’ and the gesture is pointing to someone else’s sandwich.
• However, hard to distinguish between whether social cognition deficits, or executive functioning deficits.
• Behavioural and personality domains not assessed by D-KEFS (Dugbartey, 2003) – yet this shows how it may still provide some insight

Examined the relationship between executive functions and restricted, repetitive symptoms of ASD. The aim was to investigate how cognitive abilities with ASD are related to the behavioural symptoms using the Trail Making Test, Design Fluency Test, Colour-Word Interference Test, Tower Test, and Verbal Fluency Test

• In support of previous literature, the study found that the ASD group performed similarly to the control group on measures of working memory and response inhibition, and that there were deficits in the ASD group for cognitive flexibility and planning
• Found a strong positive correlation between cognitive flexibility and restrictive, repetitive behaviours
• this provides some evidence that the tendency to perseverate is related to stereotyped behaviours
• Planning and fluency abilities were not significantly correlated with restricted repetitive behaviours
• Robust correlations between restrictive, repetitive behaviours and working memory and response inhibition
• This suggests that relatively preserved executive processes influence restrictive, repetitive symptoms of ASD and that executive strengths and deficits are related to restrictive, repetitive behaviours

Usage

Scoring

Materials

• Age Range:
• 8-89 years old (except Proverbs Test which is 16-89 years old)
• Administration Time:
• 90 minutes for all 9 tests
• Roughly 15 minutes for individual test
• Who can administer the test?
• Registered Psychologist
• Alternate Forms
• Sorting test
• 20 Questions
• Verbal Fluency
• Practice effects (Ramsden, 2003)

Schizophrenia

• Scoring Software!!

• Scoring Based on the "cognitive-process" theoretical approach to neuropsychological assessment.

• Equal importance on understanding how the examinee attempts to solve the task rather than a single-score method.

• Achievement (or aggregate) and process scores for each subtest.

• Availability of several ways of scoring (including response accuracy, error rates, and response latencies) offer a wide range of options in interpreting examinee performance.

• Interpretation involves conversion of raw scores to age-corrected scale scores – these have a mean of 10 and standard deviation of 3 and percentiles.

• No overall composite score (obviously)

• Examiner’s manual

• Technical manual

• Stimulus Book

• Card sets

• Tower stand with colour disks

• Record Forms

• Design Fluency response booklet

• Trail Making response booklet

• Extras $$$: Scoring assistant

• Paper exploring effects of verbalization on cognitive performance in schizophrenia using tasks from the DKEFS.
• Harvey, Galletly, Field and Proeve (2009)
• Previous studies have found cognitive remediation strategies (verbalizing) assists in improving performance on a task such as Wisconsin Card Sorting Task for people with schizophrenia.
• This study extends previous research by assessing whether verbalizing ongoing actions improves performance on tasks selected from the DKEFS.
• 22 subjects with schizophrenia
• Within subjects design: verbalization vs silence

(Homack et al., 2005)

Results Found

Criticisms

• Results demonstrated selective benefits of verbalization on a neuropsychological task.
• Improvement with verbalization was found on a task requiring multiple executive functioning (eg number-letter switching task)
• Poorer performance was found on tasks requiring single-component cognitive functions (i.e visual scanning and motor speed tasks.
• Found because:
• the number-letter switching task utilizes higher-order executive functions, thus verbalizing ongoing actions may have guided the participants to use a cognitive strategy that allowed for better conceptual organization.
• It also may have decreased distractibility on tasks where greater concentration and attention is required.
• However, on simpler tasks, verbalizing may have been a distraction by disrupting the momentum of motor skills and automatic visual scanning processes.
• Performance on the tower task however, requires multiple functions (eg rule learning, the ability to plan ahead and response inhibition. ). However, performance did not appear to be enhanced by verbalization on this task.
• The authors suggest this is because of the difficulty of the task – it was neither the most difficult or the most easy so cognitive remediation strategies was not detrimental or helpful to their performance

• Small Sample size
• Possible learning – sub-verbalisation.
• Amount of verbalization differed – needed to control or measure that
• However, this study found a new use for the dkefs

Administration

Benefits/Clinical Usefulness

Overall

Validity

Reliability

• Useful for identifying brain damage and diseases that affect cognitive function
• Helps determine how deficits in higher order thinking may impact an individual’s functioning. This then can be used to develop coping strategies and rehabilitation programs tailored towards individual’s profile of strengths and weaknesses in executive functions.
• Complements traditional tests of intelligence and other basic achievement skills.
• The clinical usefulness of the D-KEFS with children is largely unknown.
• Some researchers have criticised the limited clinical data presented in the original publication manual (e.g., Schmidt, 2003), specifically in regards to children.

• Validity evidence is divided
• Schmidt (2003) argues that validity for D-KEFS is poor, due to:
• The authors of the D-KEFS rely on the established validity of previous versions of the tasks to support the validity of the tasks comprising the D-KEFS.
• For the current combination of tests, validity of studies were conducted on only one of the 9 subtests (i.e. sorting). No other validity studies were conducted on eight subtests of the D-KEFS.
• However Delis et al. (2004) pointed out that much of the test development and validity data for the D-KEFS had appeared in the mainstream neuropsychology literature rather than the test manual itself, where it was available for peer review and open to “scientific critique."

• Reliability evidence is divided.
• Process scores demonstrate extremely low reliability (Strauss, Sherman and Spreen 2006).
• Strong age effects. Children at the youngest ages, i.e. 8-10 years generally exhibit the lowest scores
• 53 of 316 reliability values published in D-KEFS manual were above .80 value, “little reliability” (Schmidt, 2003)
• However, this is comparable to Wisconsin Card-Sort Test and Weschler Memory Scale – III. (Delis et al., 2004).
• Attributed to these tests tapping a wider spectrum of complex, effortful cognitive processes, rather than homogenous, fundamental tasks such as Vocabulary or Picture Completion subtests, therefore performance variability or measurement error may be greater for more complex tests.
• Difficult to use split-half reliabilities on tests of executive functions and memory.

Important to note:

D-KEFS should be used with more moderate to severe cognitive dysfunction, because the D-KEFS task

(a) is often helpful in revealing the neurocognitive mechanisms underlying the problem-solving deficit (e.g., due in part to the fact that the patient is asked

to describe his or her sorting strategy); and

(b) tends not to be as frustrating as the Wisconsin (e.g., due to the absent of right-wrong feedback). (Delis et al., 2004).

• A study by Wodka et al. (2008) examined the performance of typically developing children with children with ADHD on four subtests of the D-Kefs (Trail Making, Verbal Fluency, Color-Word Interference, and Tower).
• found: overall children with ADHD did not differ from TD on most optional measures from the D-KEFS.
• When sex and ADHD subtype were considered, children with the subtype of ADHD less common for sex were at greater risk for poorer performance.

• This suggests the optional “process” scores have limited sensitivity in identifying behavioral differences between children with ADHD and typically developing children (TD). The results of the present study, along with our recent findings when assessing primary scores (Wodka et al., 2006), suggest that the primary summary scores, rather than primary contrast scores or optional process scores, appear to be most sensitive in identifying executive dysfunction in children with ADHD.

References

Miller, D.C. (2009). Best Practices in School Neuropsychology: Guidelines for Effective Practice, Assessment, and Evidence-Based Intervention. John Wiley & Sons: USA

Ramsden, P (2003). [Review of the Delis-Kaplan Executive Function System]. In the fifteenth Mental Measurements Yearbook. Lincoln, NE: Buros Institute of Mental Measurements.

Schmidt, M. (2003). Hit or miss? Insight into executive functions (Test Review). Journal of the International Neuropsychological Society, 9, 962–964.

Uomoto, J. (2010). The contribution of neuropsychological evaluation to traumatic brain injury. In Ashley, J. (Ed.) Traumatic brain injury: Rehabilitation, treatment and case management, p. 843-882. Florida, US: CRC Press

Wodka, E.L, Mosofsky, S.H, Prahme, C., Gidley-Larson, J.c., Loftis, C., Denckla, M.B & Mahone, E.M. (2008). Process examination of executive function in ADHD: sex and subtype effects. The Clinical Neuropsychologist, 22 (5), p 826-841

Alvarez, J., & Emory, E. (2006). Executive function and the frontal lobes: A meta-analytic review. Neuropsychology Review, 16(1). doi: 10.1007/d22065-006-9002-x

Delis, D. C., Kaplan, E., Kramer, J., & Ober, B. A. (2000). California Verbal Learning Test-Second Edition (CVLT-II). San Antonio, TX: The Psychological Corporation.

Delis, D.C., Kramer, J.H., Kaplan, E. & Holdnack, J. (2004). Reliability and validity of the Delis-Kaplan Executive Function System: An Update. Journal of the International Neuropsychological Society, 10, 301-303.

Dugbartey, A., (2003). [Review of the Delis-Kaplan Executive Function System]. In the fifteenth Mental Measurements Yearbook. Lincoln, NE: Buros Institute of Mental Measurements.

Evans, K., & Hux, K. (2011). Comprehension of indirect requests by adults with severe traumatic brain injury: Contributions of gestural and verbal information. Brain Injury, 25, 767-776. doi: 10.3109/02699052/2011/576307

Fine, E., Delis, D., & Holdnack, J. (2011). Normative adjustments to the D-KEFS trail making test: Corrections for education and vocabulary level. The Clinical Neuropsychologist, 25(8), 1331-1344. doi: 10.1080/13854046.2011.609838

Finnanger et al. (2013). Differentiated patterns of cognitive impairment 12 months after severe and moderate traumatic brain injury. Brain Injury, 27, 1606-1616. doi: 10.3109/02699052.2013.831127

Heaton, R. K., Chelune, G. J., Talley, J. L., Kay, G. G., & Curtiss, G. (1993). Wisconsin Card Sorting Test manual – Revised and expanded. Lutz, FL: Psychological Assessment Resource, Inc.

Heled, E., Hoofien, D., Margalit, D., Natovich, R., & Agranov, E. (2012). The Delis-Kaplan Executive Function System sorting test as an evaluative tool for executive functions after severe traumatic brain injury: A comparative study. Journal of Clinical and Experimental Neuropsychology, 34(2), 151-159. doi: 10.1080/13803995.2011.625351

Homack, S., Lee, D., & Ricci, C. (2005). Test review: Delis-Kaplan Executive Function System. Journal of Clinical and Experimental Neuropsychology, 27(5). doi:10.1080/13803390490918444

Kleinhans, N., Akshoomoff, N., & Delis, D. C. (2005). Executive functions in autism and Asperger's disorder: Flexibility, fluency, and inhibition. Developmental Neuropsychology, 27(3), 379-401.

Lopez, B. R., Lincoln, A. J., Ozonoff, S., & Lai, Z. (2005). Examining the relationship between executive functions and restricted, repetitive symptoms of autistic disorder. Journal of autism and developmental disorders, 35(4), 445-460.