SPECIFICITY, AND POSITIVE AND NEGATIVE PREDICTIVE POWER FOR
CONTINUOUS PERFORMANCE TESTS
Ron P. Dumont, Ed.D., John Willis, Ed.D., Casey Stevens,
The Gordon Diagnostic System is another in the array of
computerized CPTs (Continuous Performance Tests) that have been used
to help diagnose attentional difficulties. It, like many of the CPTs,
lacks strong validity. I do not deny that CPTs can be useful in a
comprehensive evaluation of children suspected of attentional
problems, but they are not 'proof' in and of themselves. There are a
number of important confounds that could impact upon the score
obtained on this and other computerized measures.
Reviews of research on computerized continuous performance tasks
have generally been favorable, and they are seen as playing a role,
albeit limited, in the evaluation of attention disorders.
Barkley and Grodzinksi (1994), for
instance, evaluated the utility of neuropsychological measures,
including continuous performance tests (CPTs) for distinguishing
children with ADHD from normal controls and children with learning
disabilities. They found CPT measures among the most useful of the
assessment procedures investigated. Nonetheless, they noted that
positive but not negative CPT findings can have diagnostic utility.
Thus, while poor performance on a CPT measure was indicative of an
attention disorder, good performance did not necessarily rule out
attention disorders. These results have also been replicated by
Matier-Sharma and colleagues (Matier-Sharma et.
One recent study, Wherry et al., 1993,
Psychology in the Schools, investigated the validity of the
GDS and the results were fairly poor. These authors stated that "The
results failed to demonstrate the discriminant validity of any GDS
score regardless of the behavior rating used." As Barkley and others
(1994) have noted, in order for a test to be diagnostically useful,
it must be able to not only identify the children with ADHD, but it
must also accurately identify children without ADHD. One very
important issue regarding the typical validity studies is their use
of already identified clients. This does provide some aspect of
validity but it is also necessary to investigate the sensitivity and
specificity of the measures (something typically lacking).
Ellwood (1993) discusses parameters
that can be used to examine a test's diagnostic usefulness. Test
specific parameters include sensitivity, or the proportion of
individuals with a disorder that exhibit the sign (i.e., the
proportion of children with ADHD who receive scores within the
abnormal range) and specificity, or the proportion of individuals
without a disorder that do not exhibit the sign (i.e., the proportion
of controls who receive scores within the normal range). These two
parameters are calculated in the research setting by first knowing
the diagnosis of the children (through test-independent criteria) and
noting how they perform on the test of interest. However, as
Ellwood (1993) points out, this is the
opposite of the way an evaluator uses a test. The evaluator starts
with the test score and attempts to determine the child's diagnosis.
In order to judge the usefulness of a test for this purpose, the
evaluator will need to look at a test's sensitivity and specificity
in light of the disorder's base rate in their referral
For example, if a test was used as a screening measure on a
population of 1000 children in which 4% (40) of the children have
ADHD, and that test gives an abnormal score for 90% of the children
with ADHD (i.e., sensitivity) and gives a normal score for 90% of the
children without ADHD (specificity), the following diagnostic
Calculation of Sensitivity, Specificity, PPP, and NPP
Sensitivity = a/a+c = .90
Specificity = d/b+d = .90
PPP = a/a+b = .27
NPP = d/d+c = .99
Using this table, one can calculate Positive Predictive Power (PPP),
or the chances that a child who receives an abnormal test score
actually has ADHD. PPP = a/a+b = 36/132 = 0.27. A test with 90%
sensitivity and specificity has restricted usefulness as a diagnostic
tool if it is used on a population with a 4% base rate of the
disorder because if the child receives an abnormal score, (s)he is
still much more likely to be a control than a child with ADHD.
The issue of PPP and NPP was from an article we wrote on the use
of the Mesulam CPT (a paper and pencil CPT) that takes about 3
minutes and can be used in an entire class. We found the PPP and NPP
to be similar or better than the Computerized tests.
Barkley, R. A. and Grodzinksi, G. M. (1994). Are tests of
frontal lobe functions useful in the diagnosis of Attention
Deficit Disorders? The Clinical Neurologist, 8, 121-139.
Ellwood, R.W. (1993). Clinical discriminations and
neuropsychological tests: An appeal to Bayes' theorem. The
Clinical Neuropsychologist, 7, 224-233.
Matier-Sharma, K., Perachio, N., Newcorn, J.H., Sharma, V.,
& Halperin, J. M. (1995). Differential diagnosis of ADHD: Are
objective measures of attention, impulsivity, and activity level
helpful? Child Neuropsychology, 1, 118-127.
Wherry, J. N., Paal, N., Jolly, J. B., Balkozar, A., Holloway,
C., Everett, B., & Vaught, L. (1993). Concurrent and
discriminant validity of the Gordon Diagnostic System: A
preliminary study. Psychology in the Schools, 1, 29-36.