|
| |
Software Review: A
Comparison of Five Interpreter/Report Writers
Ron Dumont - Casey Stevens - Jon Short
Introduction
These authors reviewed 6 commercial software packages currently
available for use in creating psycho educational assessment reports.
All programs were provided by the individual software companies with
no stipulations placed on the authors' comments. (One company
requested a signed contract that allowed for prepublication editorial
input and that software package was not reviewed). IBM compatible
programs (both DOS and Windows based) were reviewed using a Digital
Venturis 466, while Macintosh programs were reviewed on a Power
Macintosh 7100 and a Powerbook 5300cs. The goals of these reviews
were to compare and contrast the programs' ease of use, user
friendliness, information provided, and accuracy of results.
For each of the programs, installation on the appropriate machines
was simple and straight forward. Those familiar with the particular
computer should find no difficulty loading and running either
version. Computer neophytes will find the installation instructions
in the manuals helpful. Once loaded and running, all programs offer
both mouse and/or keystroke navigation.
Each program reviewed acted, to greatly varying degrees, as a
report generator by placing inputted scores (raw or scaled scores
depending on the program) and user chosen descriptors into its own
report format. The resulting narrative is editable either within the
program itself or after the program transfers the results to your
word processor. Most reports included areas for background
information, test results, differing levels and ways of
interpretation, a section for recommendations (program or user
supplied) and some form of tables that include all relevant scores.
Most programs provided a way to input scores from achievement tests
and to then determine if there was an ability/achievement
discrepancy. Differing options for making this determination were
available. Most programs print a 5 to 10 page report. No program
contained a built-in spell checker. Each program offered varying
degrees of error checking of inputted scores. Some programs were
found to have serious errors that can effect the interpretations
generated. Anyone using a computer program must be mindful of the
legal and ethical obligations that accompany their use.
WISC-III
Writer: The Interpretive Software System. The Psychological
Corporation, $295. Available for Macintosh and IBM (DOS and Windows
based).
This program allows for the scoring and interpreting of the
WISC-III and the WIAT. Users are able to input either raw scores or
scaled scores from the tests and the program converts these scores
into appropriate standard scores and IQs. Because this data is
copyrighted by the Psychological Corporation, this is the only
commercial program available that allows such a conversion of scores.
Extensive error checking is available. Scores entered by accident
that do not match the range expected are automatically flagged with a
warning dialog. Once the data are entered, they may be saved to a
database and sorted by a number of options. If the user has graphic
capability, this program also allows for the inclusion of a child's
picture. Once scored, the computer will generate varying reports: a
parent report; an interpretive report; or a set of Tables and Graphs.
Besides the scoring and interpreting of the Wechsler material, the
extensive report writing nature of the program comes from 20
selectable screens that provide the user with check boxes and/or
radio button choices. Areas covered include: Test session behaviors,
Referral questions, Previous evaluations, Medical results, etc.. The
program will tailor the printout based on the information checked in
these windows. The choices made by the user are checked for
contradictions and the program guards against them (in the
development section, if a user checks "was born with no apparent
complications" and then checks "was born prematurely", the program
automatically un-selects the first choice. The report generated is
fully editable on screen before printing. Also included in the
program is a fairly detailed set of "ReportClips". These are
prewritten statements that can be copied and pasted directly into the
current report. These ReportClips cover 11 broad areas including:
Academic, Intellectual, Recommendations, and Speech and
Communications. The program also allows the user to create and save
new ReportClips for later use. This may be especially helpful for
users who wish to save pre-written statements that are used
continually in different reports. Users have the ability to save the
completed reports as report files (separate from the database), as
well as the ability to copy the entire report into a word processing
program for additional editing and formatting. This was useful since
the program does not contain a spell checker. It should be noted that
when the reports were spell checked, no errors were found in any of
the statements generated by the program.
The program offers the user many options for configuring the
interpretive report and analysis. All options are easily changed with
a click of the mouse. A full set of options for Table printouts;
Ability estimates (IQ vs. Indexes); Ability achievement discrepancy
determination (simple difference vs. Predicted difference);
Significance levels for confidence bands and VIQ-PIQ differences;
Headers and Footers are included. Users are also given the option to
change the classification labels attached to certain IQ ranges (i.e.,
80-89 = Low average could be changed to 80-89 = Below Average).
The manual was useful for understanding how to use the program but
especially so for understanding decisions made by the program in the
development of the interpretive report. Chapter 2, Interpretive
Rationale, was an easy to understand description of the decision
making algorithms used by the program. The rationale is similar to,
yet different from, that proposed by Alan Kaufman in Intelligent
Testing With the WISC-III. It should be noted that the Macintosh and
DOS versions of this program do not allow the option of substituting
Symbol Search for Coding (a la Kaufman) but the Windows version does.
Inputting of scores is done on separate WISC-III and WIAT pages.
If the dates between testing exceed a certain time, the program warns
the user in the printout about the validity of interpreting such
results. The computer converts the inputted scores (raw or scaled)
into scaled scores and percentile ranking. The computer generates a
prorated Verbal and/or Performance IQ after 4 subtests from the
appropriate scale are entered. (Will this result in people creating
"short form" IQs without understanding the technical properties of
them?) In order to see interpretive information concerning indices
and subtest score strengths and/or weaknesses, users must create a
report. The input screens are not interpretive. Within the report,
subtest strengths and weaknesses are developed always using the
separate Verbal and Performance test means. Critical values used for
developing subtest strengths and weaknesses are from the average of
all ages in the standardization sample. If the program chooses to
interpret the VCI or the POI, it does not redo the statistical
analysis to report strengths and weaknesses within those indexes. The
printout reports all scores in the body of the narrative as standard
scores, percentiles, and confidence ranges. The descriptive
categories given are for the obtained score only, (i.e., A child
obtaining a FS IQ of 89 would be classified with the words "Low
Average" although the range would be Low Average to Average). The
option to compute "Shared-Ability Composites" is available. Groupings
hypothesized by Bannatyne, Horn, Dean, Kaufman, and Prifitera &
Dersh are printed as standard score conversions and identified as
being a strength or a weakness. Instructions and cautions about their
use are included in the manual.
An extremely useful feature of this program is the option to
generate a "Clinical Review" before printing out the final report.
These reviews offer the interpreter rationale for the decisions made
by the program as well as additional information for the evaluator
(i.e., base rate for obtained V-P difference) not included in the
final report.
Printing the report was easily accomplished. Options for printing
the entire report or individual pages of it are given in the print
alert window.
Pros: Only program that allows the inputting of raw scores. Allows
for extensive error checking. Provides analysis and interpretation of
both the WISC-III and the WIAT. Extensive options for individualizing
the report. Contradictory statements are guarded against. It is the
only commercial program currently available for a Mac.
Cons: For those wanting to substitute Symbol Search for Coding,
only the Windows version has that option. The ability to change
Wechsler classification categories and still be able to print out a
report with the Psychological Corporation's name on it seems
dangerous. Limited to WISC-III and WIAT (although since it is a
totally editable text writer, the user can directly add on to the
report before printing).
(Top of page)
Psych
Report Writer Version 4.1: Psych Support Systems, $199, Available for
IBM.
The Psych Report Writer is a user-friendly program which attempts
to interpret results from a wide assortment of commonly used tests,
including the Wechsler Scales (WPPSI-R, WISC-III, WISC-R, WAIS-R),
the Kaufman tests (K-ABC, K-BIT, KAIT), and the Stanford-Binet Fourth
Edition. The program also has the capability of "hooking up" to the
WJ-R Cognitive and Achievement tests through the separate Compuscore
program. The Psych Report Writer also has the capability of
incorporating into the report results from visual motor tests, the
Goodenough-Harris Draw-A-Man test, and adaptive and independent
behavior scales. This information, along with test session behavioral
observations can be recorded on the Psych Report Writer's Computer
Data Sheet. These sheets are then used to transfer the obtained
scores to the computer at a convenient time. (While it seems a
helpful addition to the software package, the data sheet was found to
be somewhat incomplete. For example, the WISC-III Processing Speed
Index is missing, the WJ-R Cognitive Test and the WIAT are absent
from the page, and only 7 out of 21 subtests are listed under the
WJ-R Achievement heading. A spelling error was found under the K-ABC
Achievement Test heading, with a subtest listed as
"Reading/Understading").
The Psych Report Writer provides options for determining
ability/achievement discrepancies using either the "Federal LD
Discrepancy Formula" (achievement scores greater than 1.5 standard
deviations below the IQ scores are identified as significant), or the
"Indiana LD Discrepancy Model" (a regression method which uses the
correlations between the IQ and Achievement tests to predict
achievement scores). With either formula, the user is able to change
the criterion level each time the program is run.
Users will find the program easy to install, and the manual quite
helpful. Help options are also available on-screen. The program was
easy to navigate with the tab button or the spacebar, however, the
mouse cannot be used to simply "click" on the area(s) of interest.
The user can review/edit the report before printing, and can set
preferences for the header.
The Psych Report Writer does not contain any of the normative data
for the tests it interprets and therefore cannot compute the exact
IQs (Verbal, Performance, Full Scale, Crystallized, Fluid, Composite,
BCA), Index Factor Scores, or Cognitive Ability Clusters. The program
provides little or no error-checking for the scores inputted. If an
examiner accidentally inputs an incorrect score, the program simply
accepts the number and bases interpretation on it. For example, upon
entering high scaled scores on the WAIS-R, the Psych Report Writer
included in the report indications of Superior to Very Superior
abilities on Perceptual Organization and Verbal Comprehension
Factors, and yet reported that the subject would "be expected to
perform at a level which is significantly lower than same aged
peers." It's obvious that these types of results conflict, and the
reason for this is that the 'bottom line' recommendation was based
directly on the Full Scale IQ, which was entered as zero. Also
interesting was that this subject's corresponding classification was
listed as "Profoundly Retarded", a term which is not among Wechsler's
classifications of intellectual ability.
A number of major errors in interpretation were noted by these
reviewers on the Kaufman Adolescent and Adult Intelligence Test
(KAIT). The scores inputted are reported at 90% confidence, with the
confidence interval for the Composite Score given as a band of ±
8 points, while the KAIT manual provides 90% confidence bands of
± 4 points. More disconcerting was the error related to the
development of strengths and weaknesses. The manual for the KAIT
makes it clear that the means used for determining significant
subtest strengths and weaknesses are to be derived from the sum of 3
Crystallized and 3 Fluid subtests. This is true whether or not the
supplemental subtests are given. As stated in the KAIT manual, "The
Crystallized and Fluid IQs are based on the three Core subtests
whether or not the Expanded Battery is administered" (page 42).
During our review, the program consistently gave erroneous means, and
thus erroneous strengths and weaknesses, for the cases we examined.
Additionally, in one case, the program reported a subtest as a
"relative strength" even though it happened to be the lowest score
entered. The KAIT interpretation also provides a list of "shared
abilities." These did not conform to those in the KAIT manual,
occasionally leaving out a subtest that made up the grouping, and
consistently creating means that did not match the subtests listed.
Another problem concerned the interpretation of significant
differences between immediate and delayed memory scaled scores for
both Rebus and Auditory Learning. While a discrepancy of four points
between the scores was seen as significant at the .01 level, and a
discrepancy of three points was seen as significant at the .05 level,
significant differences were not reported when the delayed recall
subtest score was higher than the immediate recall subtest score. In
one case there was a 7 point difference between the relevant scores,
yet the program noted that this "was not significant at the .05
confidence level." No rationale is given for this uni-directional
mode of significance reporting. One further problem was that users
are able to enter scores well over any reasonable limit (300), and
even worse, are able to enter such inflated IQs along with below
average subtest scaled scores. (Users of this program must check
carefully all interpretive findings. These reviewers found serious
errors on the KAIT interpretations but did not analyze every possible
output and test available from this program).
Again, users are cautioned to examine these reports thoroughly for
errors. While no spelling errors were found on the printouts, they
were found on-screen ("WAIS-R Age Corected Scaled Scores), and in the
manual ("SELECT INTELLEIGENCE TEST", Figure 2.4f). Ultimately the
user must be held responsible for the accuracy and spelling on these
reports.
Further cautions: In determining discrepancies between IQ and
Achievement Tests, the examiner is unable to change the level of
statistical significance used (this program is set for 90%
confidence). The program comes packaged with 82 "Remediation files",
although the extensiveness and current applicability of each is
questionable. For instance, remediations are available for students
who obtain low scores on any of the WISC-III subtests (except Symbol
Search). It is unclear why specialized service is required for
children with single, low subtest scores, and even more unclear as to
why professionals treating children with low Block Design scores
would be recommended to use the following remediation: "more time
should be used when introducing new materials to a child with this
disability" (bold added). In this same remediation file,
practitioners are also cautioned that "such a child tends to learn
piecemeal". Furthermore, in a remediation file designed to address
poor anger control in students, judgmental and even possibly sexist
remarks were found. "When the inability to control anger is not
rooted in organic factors, it stems, of course, from family
malpractice. Two aspects of the mother's behavior must be
watched:..." (bold added). No mention of the father's role is
made in this remediation report.
(Top of page)
Kaufman
WISC-III Integrated Interpretive System (K-WIIS), version 1.
Psychological Assessment Resources, Inc. $425.00
The WISC-III Integrated Interpretive System (K-WIIS) was developed
in conjunction with Alan and Nadeen Kaufman and follows closely the
interpretive methods outlined in "Intelligent Testing with the
WISC-III". The software package comes with computer disks, a manual,
and 3 separate checklists which can be used in conjunction with the
program. The checklists (Background Information, Physical
Observations, and Behaviors Observed During Administration of the
WISC-III Subtests) allow the examiner to record basic information
during the testing session that will later be transferred into the
report. The computer screens mimic the checklists and make entry very
simple. Within the program, it is possible to check contradictory
observations. (Checking that the mother reported no health problems
while pregnant will not prevent one from also checking that she had
diabetes, high blood pressure, and toxemia). While running the
program, on screen help is accessible at any time and supported by
the trouble shooting section in the manual. There is also a 1-800
number listed in the manual for technical support which is available
during business hours. The program was user friendly for all levels
of computer users. Those unfamiliar with an IBM DOS based program
will still have no real problem using the program. Navigation through
the program was accomplished easily by either using the mouse or key
strokes.
Although the basic function for this program is to interpret the
WISC-III results, users are able to enter up to 16 achievement
cluster/subtest scores and to compute discrepancies between them and
the WISC-III IQs and Indexes. Any achievement score entered must be
expressed as scores with a mean of 100 and a standard deviation of
±15. The program will determine both simple and regression based
ability/achievement differences. User must supply the reliability
coefficient value for the test they wish compared. Information
regarding ability/achievement discrepancies is not given in the body
of the report, only in the tables appended to the narrative report.
Users can set program preferences for confidence intervals,
significance levels, and for substituting Digit Span for Arithmetic
and/or Symbol Search for Coding. K-WIIS produces a narrative report
based on user entered observations, scaled scores, IQ scores, and
achievement scores. Recommendations are supplied by both the user and
the program. A graph of the individuals reported scores is available,
but must be chosen by the user. There is limited error checking
(certain ranges for scaled scores and IQs). The program assumes that
the user is error free when entering the data. For example, it
allowed a Full Scale IQ of 42, a Performance IQ of 90, and a Verbal
IQ of 78 to be entered without warning. This could happen by striking
the wrong key on the key pad, resulting in an improper report. Text
files of commonly used information can be save in a "Library" file
and used in subsequent reports by simply copy and pasting the items.
Headers and Footers for the report can be altered to suit the users
needs. No spell checking abilities are available within the program
itself (a few errors were found during this review) but an option
does allow the user to save the report in a specified word processing
format and edit the report there.
Pros: Allows for the option of substituting Symbol Search for
Coding and Digit Span for Arithmetic. Extensive choices for
individualizing the report. Review of the final report can be
accomplished on screen before printing and can be saved as a text
file to be opened by word processing software. Software package comes
with hard copies of the checklists which can filled out prior to
running the program. Can be run from DOS or Windows.
Cons: Lack of extensive error checking. Is limited only to the
WISC-III. The Mazes subtest is not included at all in the program. No
achievement interpretation or narrative in the report.
(Top of page)
QuickWriter Version
1.1d, Ewing Solutions, $249, IBM 386 or newer, Windows 95, Windows
3.1
QuickWriter was easily loaded onto the computer and ran well.
Without having read the manual first, we were able to quickly and
easily create reports. This report writing program provides a
comprehensive psychological report. The reports are generated by
choosing options from the "Main Menu" window. This window provides 12
choice buttons, each representing some aspect of the resulting
report. After entering a new student's background information or
selecting a previously saved case, the user navigates the program by
simply clicking the mouse on any of the buttons (keystroke navigation
is also available but less desirable). The navigation buttons take
the user to screens that offer from one to 6 tabbed sublevels. For
example, clicking on the "TEST MENU" button brings the user to a
screen with 4 tabbed choices: Test Session Observation (with 6 more
submenus), Intellectual (6 submenus), Academic-Adaptive (7 submenus),
and Developmental-Visual Motor-Other (6 submenus). Each choice
provides numerous check boxes (for multiple statement choices) and/or
radio buttons (for choosing mutually exclusive options). Many of the
screens also include an active text field in which the user can enter
their own statements or comments. This was an extremely attractive
option for this program. The program does not allow for on screen
editing or viewing of the final report. Reports are either saved to
disk, printed, or automatically exported to a word processing program
the user specifies.
The QuickWriter provides 3 options for determining
ability/achievement discrepancies: a "Standard formula" simple
difference method (ability/achievement difference scores greater than
a user entered cut-off are identified as significant), a "Federal
formula" (a z-score transformation with user defined discrepancy
requirements), and the "Arizona formula ( regression method which
uses the correlations between the IQ and Achievement tests to predict
achievement scores).
No determination of a child's subtest strength or weakness is made
by the program. Most interpretive statements simply describe the
global scores obtained. More interpretive statements were generated
for the WISC-III than any of the other tests available in the
report.
We did note one major error and a number of minor
problems/errors/discrepancies in the program. The major error
involved the interpretation of the WISC-III scores. A statement about
the validity of the measured cognitive ability is reported by
determining if there is a significant difference between the Verbal
Comprehension (VCI) and Freedom from Distractibility (FDI) Indexes.
It was found that instead of comparing the score on the FDI to the
VCI, the program incorrectly compares the Processing Speed index to
the VCI. There are a number of times in the program where, if using
the tab key, certain choices are skipped. In the Intervention window,
an entire column is missed when using the tab key. A number of the
allowable multichoice statements seemed contradictory and might have
been better grouped together with radio buttons. For example, a
child's report can conceivably state "Ron is a male with blond,
brown, black, red, auburn, fair-haired and dyed hair and blue eyes."
A number of spelling errors were noted, most disconcerting was the
spelling of Wechsler as Weschler. If one chooses the WAIS-R, there is
no instruction about the use of Age Scaled Scores vs. Scaled Scores.
The WAIS-R also allows for entering a score on the MAZES subtest!
Missing from the Intellectual assessment choices was the
Woodcock-Johnson-Cognitive. There was little error checking of the
scores inputted. Subtest scores over 19 and IQ scores up to 999 were
allowed.
The manual was not particularly helpful for this program. On the
one hand, the program is so user friendly there does not appear to be
much need for the manual. The manual might better serve if it
included more detailed discussion about the decision making processes
used in the interpretive narratives.
(Top of page)
Educational
Applications of the WISC-III (EAW3) V2.003, Western Psychological
Services, $249, IBM with Windows 3.x
This particular program differs the most from the others reviewed
because it does not generate 'typical' narrative reports. The
program's focus is to "complement the detailed discussion of WISC-III
results presented in "Educational Applications of the WISC-III: A
Handbook of Interpretive Strategies and Remedial Recommendations"
authored by Charles Nicholson and Charles Alcorn. Installation was
simple and flawless. A manual is provided on the disk so that a
person wishing may either read it on screen or print it out for
reading. The manual explains the basics necessary for installing and
using the program but does not provide any explanations for decisions
made within the program. Users wishing this information must purchase
the "Handbook." Use of the tab key to navigate from entry to entry
was easy, and the mouse can be used to point and click into an entry
field. Data for the 3 IQs, 4 Indexes, and 13 subtests of the WISC-III
are entered as standard or scaled scores. If an achievement test was
administered the results of that test can be entered. Although the
program has options for choosing 1 of 7 Achievement tests (including
the misnamed WIAT-R), the entries for achievement scores do not
differ. Users may enter only Reading, Math, and Spelling scores. The
program reports all entered IQ and Index scores as single values with
no confidence intervals. Comparisons between IQ and academic
achievement display standard score differences for VIQ and PIQ only.
The program also provides the "Estimated Mental Age and Expected
Grade Equivalent (GE) Achievement Level". A theoretical GE
Achievement Level at Age 16 is also reported.
The completed "Professional Report", unlike all others reviewed,
consisted of two columns of text. The narrative describes each
subtest, the appropriate scaled score, and a descriptive
classification of the child's performance (for example: low, high,
good, fair). Each subtest is described as if it had measured a
separate, unique skill as opposed to being thought of as a part of a
total battery. No attempt is made to describe any of the scores as
being significantly higher or lower than the mean of the test.
Instead each receives its descriptive category based on its absolute
value in a scale of 1 to 19. (The "Handbook" does provide a
description of how one might go about determining subtest strengths
and weaknesses, but the way this is done is different than most other
programs: Determine the mean of the separate scales, round the
results to a whole number, and determine a S/W based on subtests
being ±2 points away from the mean). Next a description of IQ
and Index Scores is provided. Each score is reported as an obtained
score with no confidence intervals noted and no ranges reported. If
there is a 15 point difference between the scales, the program
provides descriptions of possible reasons for such differences. No
mention of base rate is made. Index scores are next reported, whether
they are relevant or not and whether or not the scores that make them
up differ significantly. The Freedom from Distractibility and
Processing Speed Indexes are given solitary descriptions ("ability to
concentrate" and "ability to complete timed activity involving the
use of nonverbal information"). If an achievement test has been
administered, the program compares both the standard score and grade
equivalent to the Verbal, Performance and Full Scale IQ standard
score and "expected grade level based on" the respective IQ. (The
Mathematics achievement score is alternately called an "arithmetic"
and "mathematics" score in the narrative, even if it is Math
Computation, Math Reasoning, or a Math Broad cluster score). Finally,
the narrative report provides descriptions of up to 33 "Significant
Factors." A caution is given about the validity of these factors and
if more than 4 factors are 'evident', the program provides a warning
to check the scoring of the WISC-III and the entry of data into the
program. At the end of the professional report, the program prints a
set of "Remedial Recommendations" based solely on the age of the
child tested and a perceived weakness (low score) on a particular
subtest. Interpretive descriptions, recommended procedures, and
suggested materials from various vendors are printed for each
suspected weakness.
Changes to the data can be made up to the point of selecting
"Complete" from the menu choices. Special procedures are necessary to
modify data from a completed report. No text editing is available in
the program itself. Files can be saved for editing by a word
processing program. There are a limited number of configuration
options. The program lacks any ability to set and/or report
confidence ranges, to set or report significance levels, and to set
or report subtest strengths or weaknesses. Some error checking is
available, but it is limited and confusing. If you have inadvertently
entered a scaled score that is beyond the range of 19, the program,
when you choose "Complete", displays a warning "Too many scores have
been skipped" or "Too many scores are out of range". The program does
not indicate which score or scores are out or range. The user must
review each individual score for correctness.
Although the program was easy to use, its practical relevance is
questioned. The interpretive rationale for this program seems to lack
much validity. The program's focus on interpreting the exact score
obtained instead of the score in relationship to all scores obtained
seemed a bit dated. The lack of base rate information, confidence
levels (90 or 95%); the lack of confidence bands on global scores;
the use of Mental Age and Grade Equivalents all raise concerns about
the use of such a program by school psychologists.
(Top of page)
Report
Writer for the WJ-R, Riverside Publishing Company, $348, IBM and
Macintosh
The Report Writer for the WJ-R was easily installed onto the
Macintosh computers. Entering data (raw scores) was simple and very
similar to that of the WJ-COMPUSCORE that anyone using the WJ-R is
probably familiar. Moving from screen to screen by clicking on large
icon buttons, the user enters various data or chooses various
options: background information, raw scores for any of the 21
cognitive subtest and 14 achievement subtest (both forms A and B are
available), norms based on age or grade, aptitude/ability achievement
discrepancies (choices ranged from 1.3 to 2.3 SD (SEE) Discrepancy),
report options of Standard, Customized, or Summary and Table of
Scores. With every possible option chosen, the program created a 17
page report that included extensive narrative and a number of pages
of score tables. Standard error checking is provided as the user
enters raw scores. On screen descriptions provide subtest name and
raw score range.
The report is capable of providing detailed descriptions of each
test, cluster, and subtest given. Scores are reported in the
narrative as grade equivalents, percentile ranks, standard scores,
and classification descriptions. In the narrative section of the
report, scores are reported as obtained scores with no confidence
ranges added. Within the printout for "Table of Scores", the standard
scores are reported with 68% confidence bands. There is no option
within the program to change these levels to 90 or 95% confidence
bands. If cluster scores differ significantly from one another, or if
the subtests that make up a cluster differ significantly, the program
notes the differences, but gives no plausible explanation for these
differences. This was true throughout the program. No real
"interpretation" of the scores is made, only the placing of the
appropriate scores within the suitable narrative is done. It appeared
to these reviewers that the emphasis on reporting grade- and
age-equivalents over the more 'appropriate' standard scores, as well
as the limitation of 68% confidence intervals, is unfortunate
The program does provide on screen editing once the program has
generated a report. No spell checking capability is incorporated
although it was simple to copy and paste the completed report into a
word processing program and spell check it from there. (There were no
spelling errors found during this review with the exception of some
questionable use of words: The report produced the following sentence
"Word attack measures Ron's ability in applying phonic and
structural analysis skills to the pronunciation of phonically regular
nonsense words." [BOLD ADDED].
One draw back to the report writer is the lack of graphs that show
error bands for each cluster and each subtest that makes up a cluster
(similar to those provided with the PROFILES program of COMPUSCORE).
These graphs would be useful in determining relevant differences
between clusters and within clusters.
Once a report is customized and displayed or printed, you may not
go back and re-customize choices without reloading the appropriate
case and starting over again. This is not so much a problem as a time
consuming process. There is no capability to save as a report as a
text file.
This program works only with the Woodcock tests. If the Woodcock
Cognitive is given, the only achievement scores and comparison
available is to the Woodcock achievement.
This program does a wonderful job for the tasks it is designed to
do. The biggest limitation of the program is the interpretations
given. The program does a good job developing narrative reports but
the reports lack any true interpretation. No effort to describe
meaningful differences between clusters or subtest s are made and no
real hypothesis generation is made.
(Top of page)
Bottom
Line
Although all the reports are editable in one way or another and
do, to differing degrees, contain error checking, these authors did
find significant differences between the programs' rationale and
output. Each program, in its own right, offers comparable yet
differing levels of interpretation; completeness of reports; and
number of tests available for interpretation. Practitioners will want
to assess their own individual needs when choosing among these
programs. For example, only the WISC-III Writer (TCP) and the Report
Writer for the WJ-R offers a program that can convert the raw scores
to scaled scores for the appropriate tests, but these programs are
limited to the WISC-III and WIAT and WJ-R Cognitive and Achievement
respectively. The Ewing and Psychological Support Systems programs
offer the most comprehensive number of tests allowable, but often do
so at the expense of real interpretive and accurate statements. For
devotees of Alan Kaufman's method of interpretation, his KWIIS is a
good choice, but it too is limited. The most accurate test results
were found in programs published by the companies that also publish
the respective tests, but they are typically limited to those single
tests. (It should also be noted that these reviewers discovered a
mistake in the 'Shared Abilities' report created by the KAIT-ASSIST
program published by AGS.) For a program that offers multiple tests,
the Ewing program provides a large number of option in an attractive,
easy to use program. The EAW3 is very different from most of the
programs reviewed, both in the interpretive methods used and the
format of the report. Users of this program should understand and
agree with the Nicholson and Alcorn analysis that is reported. (The
first author has major concerns about the accuracy and validity of
this method).
No matter what program a school psychologist chooses, the user is
ultimately responsible, both legally and ethically, for the reports
generated. Each program has, either in the manual or in the program
itself, a disclaimer to the user about this point.
These reviews, comments and opinions are those of the authors and
may not reflect that of NASP, the editorial board of the
Communiqué, and/or the affiliations of the authors.
(Top of page)
|