|
| |
Preliminary
Observations on the WIAT-II – Some serious, some trivial
-
In general, the graphics and the content of
the items appear to be much improved and excellent.
(Trivial: Note that the graphic designer has still been unable to spell the
word OPEN upside down and backwards on Oral expression, item 14. Might be the
same person who allowed the sun to melt the door handle off the WISC-III PA
item and had the robber place an extra hat box on the shelf in the same
subtest!)
-
The Oral Expression subtest still does
not appear to consider the quality of oral expression in the scoring.
Grammar, length of utterance, vocabulary, and other essential components of
oral expression seem to have little effect on scores. Apparently, a series of
ungrammatical phrases and sentence fragments using primitive vocabulary
(except for certain key words) could score as high as a series of eloquent,
perfectly grammatical, compound, complex, and compound/complex sentences with
embedded clauses and a rich vocabulary. We are hearing reports from
practitioners that Oral Expression seems to be yielding inflated scores,
especially for children with oral language problems. We suspect that the
scoring rules and the heavy use of visual materials might account for such
reports of allegedly inflated scores for children with oral language
problems. The scoring seems in some instances arbitrary. For example, does
the failure to compare the story to ones own experiences or another story
represent a weakness in oral expression or the ability to stay focused on the
assigned task or simply a lack of narcissism? Even the administration example
for the visual passage retell (story about Emily) does not contain any
narcissistic, self indulgent statements. How is a child expected to know that
this is part of the scoring rubric – we never gave an example?
-
Even studying the examples in the manual, Oral
Expression remains difficult to score reliably.
-
One wonderful aspect of the WIAT was the
lovely contrast between Reading Comprehension and Listening Comprehension.
You had two tests, normed on the same sample, that were (except for the memory
demand of Listening Comprehension) nearly identical in format, with one
requiring reading and the other only listening. This contrasting pair was
tremendously helpful in distinguishing reading comprehension problems from
more pervasive language comprehension problems and in documenting the severity
of a reading problem compared to an expectation based on oral comprehension.
That contrast is now lost in the WIAT-II.
-
The Reading Comprehension questions
appear to be good and much better than the ones on the WIAT.
-
The Reading Comprehension scoring rules
in the easel should, we think, provide more guidance on querying. Although
much improved over the WIAT, the acceptable answers still place a premium on
the examinee guessing what the question really is asking, so there are many
possible answers that suggest good understanding but receive no credit and
apparently warrant no query.
-
The Listening Comprehension subtest now
seems to be primarily two very brief vocabulary test (Receptive Vocabulary –
16 items, and Expressive Vocabulary – 15 items) with an extremely brief (10
items) sentence comprehension test thrown in. The formats are very similar to
the PPVT-III, OWLS Listening Comprehension, and EVT, respectively. If this is
an area of concern, we would be more inclined to use those tests to gain a
larger number of items.
-
However, one examiner has reported preferring
the new Listening Comprehension subtest, especially the sentence
comprehension component, and finding it does a better job of documenting
listening problems observed by teachers. Students who do better on the WIAT-II
Listening Comprehension than they do listening in class might be demonstrating
good listening potential impaired by distractibility in class.
-
The Listening Comprehension subtest
has, as noted above, 3 separate parts (sub-subtest?) with continuous numbering
from 1 to 41. There may be some confusion regarding scoring of these parts.
On the record form the discontinuation rule indicates that an examiner stops
after “6 consecutive scores of 0.” Is this for the entire test or for each
separate part? Since the Record form does not indicate that the
discontinuation rule is actually to be applied to “each section”, and because
the numbers imply that the test is all one set of items (1-41), an examiner
may discontinue testing without giving the child a chance on each of the 3
sections. One must reads carefully page 74 in the Examiner’s manual to see
that the discontinuation rule applies to each of the 3 sections.
·
The lack of top and bottom in the norms tables is a concern. We
realize that the test is not designed for gifted adolescents, but the low and
uneven tops on some of the subtests for adolescents could significantly skew
results for a bright (not gifted) adolescent with strengths in those areas with
low tops and weaknesses in other areas. Please see the scores of
Sam McGee
(zero raw scores) and Ginny Genius (perfect raw scores) as well
as WIAT II
Minimum and Maximum Standard Scores by Age and Subtest.
-
The reading speed measure would be a wonderful
addition were it not for the possibility that the student might read some or
all of the stories aloud. Also, measuring reading speed on a comprehension
test is risky. Some students might cautiously re-read the story before
announcing they were done, while others might barely skim it, planning to
review it for detail when they heard the questions.
-
The Reading Rate chart in the record form is
interesting, but difficult to interpret (even assuming the student read
silently, did not obsessively reread, and did not simply skim and wait for
questions). Definitions (e.g., "Far below average to below average") are
given on the page for the quartiles, but not for the various shaded areas on
matrix.
-
Differences between computer data-entry and
hand-scoring for Written Expression has confused some examiners and
caused scoring errors.
-
The Pseudoword Decoding is a wonderful
addition. We wish there were a comparable Pseudoword Spelling.
-
In general, the alternative correct
pronunciations for Pseudoword Decoding are good. Could a /z/ sound
also be correct for "nesal"?
-
There has been a lot of discussion about
Reading Comprehension. Some examiners have been pleased that the required
stopping and ending points diminish frustration. Others have asserted that,
when they tested limits, students have missed many items below the starting
point or passed many beyond the stopping point, leading them to question the
validity of the resulting scores. New rules and new norms were developed by
The Psychological Corporation, which seems prudent, but which will increase
confusion. For further comment see: WIAT-II
Reading Comprehension Changes and Comment
The new rules and normative information can be obtained at the following link:
http://www.PsychCorp.com
-
The optional procedure described in Product
Update No. 1 ["If the student is able to respond correctly to many of the
items in the new (lower) grade level, the examiner may continue to administer
items beyond the stop point for that grade level according to his/her clinical
judgment, but not beyond the original start point."] seems like a very good
idea, but unfortunately a potential cause of inter-examiner inconsistency in
scores (much like the essentially optional basal and ceiling rules on the WJ
III).
-
The inclusion of phonological items in the
WIAT-II is very good, but we have some concern about burying the phonological
items within the Word Reading subtest (rather than making a separate
subtest with a sufficient number of items as was done with the valuable, new
Pseudoword Decoding). We am suspicious of tests that measure two or more
different skills in a single subtest (known as the "Two Handles on a Shovel"
phenomenon, best illustrated by the K-TEA Brief Form and the
MBA). It might prove
difficult, for example, to sort out young children with strong phonological
skills and very weak decoding from children with moderately weak phonological
skills and moderately weak decoding.
-
We really like the inclusion of all 26
lower-case letters in Word Reading. We are not sure what the three
letter-matching-from-memory items contribute, and one of the three pulls for
reversals (d p b).
-
For the Word Reading rhyming items, the
examiner provides a good explanation if the student does not respond,
but not if the student makes an error
(which would seem to doom the next three items). We wish the first
(essentially practice) item (30) were not scored, given the wide variations in
previous experience young children will bring to this subtest. Item 34
(beginning sounds) gets an explanation for either no response or an error,
which seems much more reasonable.
-
The lamination on the Word Reading and
Pseudoword Decoding cards does not seem very strong.
-
On Numerical Operations items 6 and 7
(penny counting), two wrongs may make a right: the child may point wrong
(missing a penny) and count wrong (skipping a number) and come up with the
right answer.
-
The answer spaces for the common-fraction
items on Numerical Operations are confusing. Are the answers to be
placed on the line or does the line separate a numerator form a denominator?
-
There really aren't many items on Numerical
Operations for high school students with dyslexia but decent math skills.
This could be a serious limitation when assessing strengths and weaknesses.
Item 54 (missing hypotenuse of a right triangle with sides of 3 and 4 is (like
the similar WJ-R item that was improved on the WJ III) too easy to guess.
-
Math Reasoning item 53 should, we
think, accept a range of answers, e.g., 70% to 80%, rather than only precisely
75%. Item 60 is very unclear. Does "the least expensive set" refer to the TV
that was least expensive before the sale or the one that is least expensive
after the sale mark-down? Item 66 is hopelessly unclear. "How many ways are
there to arrange four books on a shelf?" You could stack them horizontally,
front cover on top, in any of 24 sequences. You could stack them
horizontally, back cover on top, in any of 24 sequences. You could arrange
them in a line in any of 24 sequences. You could stand up 2 and use 2 for
bookends. You could . . .
-
We read and understood the discussion of the
importance of testing homonyms on the Spelling subtest (page 12, Examiner’s
Manual), but still think that there are too many homonym and near-homonym
items.
-
Many of the Written Expression items
are very good.
-
More practice may help, but even with two
manuals to consult, we continue to find ambiguities in scoring Written
Expression. We are concerned about the scoring accuracy of inexperienced
examiners and examiners who do not work full-time as evaluators.
-
However, one examiner has reported that the
Written Expression subtest does a good job of picking up writing problems
missed by other tests.
-
We really dislike the Parent Report
form. It may bewilder most parents. The greatly elevated normal curve makes
the extreme categories seem much too large in proportion to the average
category. Parents may pay more attention to the heights at which the subtest
abbreviations are written than to their positions along the baseline. Unlike
the more “normal” normal curve provide on the page with Ability-Achievement
Discrepancy Analysis, the parent’s normal curve does not provide any reference
scores below the curve. This would seem a reasonable addition to this
section. The Parent Report form reads as though it is intended to be sent
home with no face-to-face meeting and no narrative report. It strikes us as
woefully inadequate for that questionable purpose, a purpose we view with
alarm and wish were not encouraged.
-
The Princess Summerfallwinterspring phenomenon
(precipitous drops in scores overnight from Fall to Winter or Winter to
Spring) is very small on the WIAT-II. See
WIAT II Norm Changes, based on Grade, Between Seasonal
Norms [It takes some
searching to find the definitions of the seasons on page 40 of the manual.
Fall = August, September, October and November; Winter = December, January,
and February; and Spring = March, April, May, June, and July. “What are the
three seasons of the year?”]
-
The extra Oral Expression norms pages
for nine-year-olds in third grade are unexpected and easy to miss. They are
found on pages 242 and 243 of the Scoring and Normative Supplement for Grades
PreK -12.
-
However, the expected increase in raw scores
from one grade to the next to maintain a standard score of 100 is often two
points or less. Please see WIAT-II
Raw Score for a Standard Score of 100 by Winter Norms.
This low rate of growth makes me worry about the use of the WIAT-II for
re-evaluations. We do not think it is generally wise to use any individual
achievement test for monitoring annual growth, but one of the most popular
uses of such tests is, sadly, for special education teachers to stop teaching
for a couple of weeks each spring and administer individual achievement tests
to all of their students.
-
Nice caveats about grade-equivalent scores on
p. 145 of the manual.
- Information
from page 151 of the Examiner's manual provide some clues as to the effect of
age on the WISC-III scores. See
Wechsler
Intelligence Scale for Children - Third Edition Normative Re-anchoring
(1991-2001)
-
|
