|
Since 2002, 4 million visitors plus:
| |||||||
|
|
This is not meant to be a comprehensive review of the LET-II but merely a summation of questions and concern. The Learning Efficiency Test-II (1992 revision) is published by Academic Therapy Publications and authored by Raymond Wheeler, a Professor of Psychology at East Carolina University in Greenville, North Carolina. It costs $60.00 for a manual (187 pages), stimulus cards, 50 record forms, and a vinyl folder. Norms are provided for ages 5 through 75 and older, with tables presented in one year intervals between 5 and 16 years. The catalog and manual claim that it is a quick and reliable measure of visual and auditory memory characteristics and that it can provide useful information about a person's preferred modality of learning, as well as providing information about the impact of interference on memory storage and retrieval. The LET-II can be administered in 10-15 minutes. It consists of strings of 2 to 9 non-rhyming letters presented either orally or visually. The child/adult responds verbally and subtests raw scores are based on string length. The raw scores are then converted to scaled scores and percentile ranks. Memory on the LET-II is assessed in two modes (visual and auditory) and in three recall conditions (Immediate, short-term, and long-term). The six subtest scores (Mean 10, SD 3) can be converted into Modality scores as well as a Global Memory score (Mean 100 SD 15). Given the way that memory can affect a child's functioning in school, and the need to assess preferred learning modality, the LET-II is a test that we might be tempted to buy. An examination of the LET-II tests material, manual, and catalog raised the following concerns: The manual states that the LET-II "has been empirically demonstrated to be highly predictive of actual classroom levels of performance in reading and mathematics for students with average ability as well as handicapped students" (page 10). The support for this claim is 4 citations. Examining the bibliography at the back of the manual, these four empirical studies turn out to be 3 unpublished Master's degree theses done at East Carolina University in Greenville, North Carolina. The fourth citation is for the LET (1981) manual. Standardization: The LET-II was normed on a sample of 1126 children and adults between the ages of 5 years 0 months and 85 years, 4 months. Data is provided for the age, sex and race of the sample but no Socio-economic status data is provided. The manual states that the "participants came from a broad range" of SES backgrounds. Sex variables are fairly even across the sample with 46% male and 53% females being included. This is very close to the U.S. Census Bureau's 1990 estimates of 48% and 52% respectively. Race variables are less comparable to the Census data. The LET-II reports percentages of 66 versus 33 for Caucasians and Blacks in the total sample. This in contrast to the 77 and 12 percent estimates for the U.S. population. For the ages 5 to 16 the percentages are 58 versus 42 for Caucasians and Blacks. No geographical data is provided. It is my assumption that most came from North Carolina. For each of the one year age intervals, 5 through 16 (not 5 through 15 as stated on page 7 of the manual), the samples average only 55 people, with a high of 84 at age 8 and a low of 40 at age 16. These are well below the sample size of 100 recommended by Salvia and Yssledyke (Assessment in Special and Remedial Education/Third Edition, 1985) for the computation of standard scores. No data about the IQs of the children in the normative sample is provided. Adults in the sample were administered the Peabody Picture Vocabulary Test and anyone scoring below 85 were excluded from the sample. It is further noted that "no known cases of mental retardation as defined by a general IQ score of less than 85 (on either a group or individually administered intelligence test) were included in the sample." (italics added). Two points. Was this the PPVT or the PPVT-R and would the age of the norms have an effect on the IQ? Who defined mental retardation as "a general IQ score of less than 85"? Reliability: Reliability studies using the LET-II were not conducted. Information based on 2 studies using the LET (1981) are included in the manual as measures of the LET-II reliability. One unpublished, "informal" study, involved 55 learning disabled student in grades 4 through 12. Coefficients for this group were found to be .71 to .86 (median .80). No breakdown by age, grade, IQ, or time interval is provided in the manual. Without that information, and the interval between test and retest, this data seems almost meaningless. A second study was done which involved only 40 students identified as having "learning and behavior problems." These students each had IQs above 89 and were test-retested between 1 to 6 weeks. Reliability coefficients ranged from .81 to .97 and are generally considered adequate. However, there are no specific information about age and or grade of the subjects. All are simply labeled "secondary" students in the manual. Since these reliability studies were carried out using the LET and not the LET-II, no coefficients, and hence no Standard Error of Measurements are provided for the two Modality scores or for the Global Memory score. Without this information, there is no way to calculate meaningful differences between the Modality and Global memory scores. The reported coefficients also offer no information from which to draw conclusions about the permanency of the scores since the time limit between the two administrations of the test was fairly short. Validity: Validity in the manual is addressed in a number of ways: content validity, diagnostic validity, and predictive validity. Diagnostic validity was shown by examining patterns of performance among 4 groups of special education students and comparing these patterns to a group of "average" students in grades 4 though 7. The 197 Average students had average Verbal, Performance, and Full Scale WISC-R IQs of 111, 110, and 112 respectively. This raises some question about what might be expected when this 'average' group is compared to a special education population. Average is typically thought of as having a mean of 100, not 112! Won't comparing a group labeled "LD" who have a mean IQ of 93 with a group labeled "average" with a mean IQ of 111 create some confusion when interpreting? My average and this sample's average are different. One confusing aspect of the statistical tables presented in the manual is the arrangement of the categories for the "learning groups." In the tables listing Correlations, Mean recall characteristics, Percent of information loss, Reading and Math achievement, and Stepwise and Simultaneous regression, the order in which the categories are presented change from page to page. The first table lists the order as Average, LD, EH, EMH and SL; yet the very next table lists the order as Average, EH, LD, EMH, and SL. In the next 3 tables, the order changes again and the EMH group becomes an EMR group? One may quibble that this is a minor point, but in a manual that contains other errors, some fairly serious, this does raise questions about the validity of such tables and statistics. Table 2, which presents 4 pages of Means and Standard Deviations for raw scores based on string length by age level grouping, contains either a very curious developmental anomaly or a very serious error. Page 163 is a table for Auditory Memory-Ordered. For the Immediate and Long term trials the table shows the expected increasing lengths of the string as the person's age increases. A child of 5 remembers an average 2.91 letters while a person of 16 remembers 5.41. However, for the column for the Short term recall trial we discover that all ages had mean recalls of approximately 1 letter?? In fact, according to this chart, at age 70-74, a person remembers 1.71 letters but the standard deviation is larger than the mean (2.04)? This is almost certainly a serious error in the table. How many of the other tables are in error? I don't know for sure, but curiously, looking at the conversion tables for raw score to scaled score for a 15 year old (page 99) I found another! As you examine the table you note that the higher the raw score one has, typically the higher the scaled score will be. Not so for age 15, unordered, immediate recall. OOPS! Table 6 (page 167) and Table 7 (page 168) display mean recall characteristics and percent of information lost for each recall trial for each special group. These pages are interesting for the way the information is explained in the text of the manual. The manual explains that special groups show distinct patterns of differences in recall capacity. The tables do show this but one must ask about the use of percentages as the measure of loss. For example, the average child, with an IQ of 112, starts by remembering 4.7 letters and drops to 3.7 on short term recall, a loss of 22.3%. The LD child starts at 3.5 letters and drops to 2.2 on short term recall. A drop of 37%. The percentages are different by such large percentages only because the LD child started at a lower number. The actual loss is about the same, 1 letter! One aspect of validity not addressed in the LET-II manual is that of carry over to real life. The LET-II has items that are discrete units of abstract symbols (letter) delivered in a non meaningful manner. With regard to children in school, much of the material to be learned requires the recognition and use of meaningful items. Krupski (1985) demonstrated that for learning disabled children, memory performance is approximately equal to that of average students if the material being presented is meaningful in nature. If this is true, the LET-II may be measuring some important aspect of non meaningful memory, but tells us nothing about the child's performance in real life situations. Since the LET-II proposes to measure and relate certain learning styles to academic functioning, there should be some studies to show the meaningful relationship of different learning styles to classroom success. The manual does not offer any study to support any claim that academic deficiencies can be attributed to a weaker modality score on the LET-II. Administration and Scoring: Administration of the LET-II should be fairly straight forward and trouble free. Unfortunately, it isn't. Although the manual presents detailed instruction on how to administer and score the LET-II, these instructions are at times unclear and worse, they contain errors! When administering the LET-II immediate recall subtests, the examiner presents the visual stimulus items for 2 seconds while the Auditory stimulus items are read 1 per second. After the child repeats the items back to the examiner, s/he is asked to perform some interference task ("Count from 2 to 12"). The manual states that some children have difficulty with the counting interference task, but examiners are not instructed what to do if the person can't perform the task. Recording responses is addressed on pages 36-37 of the manual. Unfortunately the examples used on these pages contain errors in the scoring. (In fact, there are two errors in this section). When explaining how to score an item for which the child has extended the string beyond the length, the examiner is told to disregard any letter beyond the proper length. The example given is for a child presented the string "Q-R-H-X"and who responds "Q-R-X-H-Y." Scoring is for both ordered (correct placement of letter) and unordered (correct letter, regardless of placement). The manual states: "Scoring of items would end with the fourth item in the student's response. Both Ordered and Unordered raw score would be four." (italics added). Gee, I thought the score would be 2 and 4 respectively? The second error on the page is in the notation of the sequence. Letters that a child gives that are not in the presented string are circled to indicate such an error has occurred. The examples on page 37 have letters that were included in the string circled to indicate that they were not included in the string!! But wait...the same notation error is made again on page 72 for a different protocol! And on page 76, another error, but this time in a different notation. Can anyone get this right? Okay, I'm done with errors, but I stopped looking. How about just wanting to be average. Examining the norms tables for ages 5 to 20, it is interesting to note that a child cannot obtain an "average" scaled score of 10 a large percentage of the time. For example, in the Visual modality tables, there is no raw score to scaled score equivalent of 10 in 58% of the cases. In the Auditory Modality tables the child will fare better. Here there is no raw score to scaled score equivalent of 10 in 56% of the cases. What do raw scores convert to then if not average? Often the converted score is 8 or 11. A 3 point difference based on 1 raw score point. Three points! One standard deviation! So what's my point? I guess it's that you can't tell a test by it's advertising. No matter how good a test looks; or what company publishes it; or that it is published at all, we as examiners are often left to critique the tests we buy. In fact, that is our ethical responsibility. If we use it , we are responsible for defending the reliability and validity of it's use. After spending lots of money on a test, it's too bad how often we can be disappointed by the product. As far as the LET-II is concerned, I'd follow Lennon and McCartney's refrain "Let it be. Let it be." As for all others, Caveat Emptor (Buyer beware). |