|
Since 2002, 4 million visitors plus:
| |||||||
|
|
Comments on the KABC-II and KTEA-II after a review of the tests and their Manuals. These comments are simply questions waiting for answers. KABC-II Tech manual Page 1: The “KABC-II is not intended to be administered in Spanish; except for the Nonverbal Scale, it is for use with children who are proficient in English.”
Page 3: When referring to the 4 to 6 year old groups, Rebus Delayed is listed as a supplemental subtest for age 6 only.
Page 4: “All core subtests, and all but one supplemental subtest, are fully normed on the complete KABC standardization sample. “ QUESTION/COMMENT 3: What does "fully normed" mean? From the manual it appears that Face Recognition (FR) is not normed past 5 years old and Conceptual Thinking is not normed past 7. Story Completion (SC), Rover (RO), Atlantis delayed (ATd), Block Counting (BC), Pattern Reasoning (PR), and Rebus delayed (Rebd) are not available for ages 3 to 4:11. Checking the norms tables shows:
Page 4: Core/Supplementary/Out-of-level QUESTION/COMMENT 4: There doesn't seem to be a good explanation in the manual for the difference between a Supplemental and an Out-of-level (OOL) subtest. Page 6: Table 1.2 QUESTION/COMMENT 5: There appears to be several discrepancies between this table and the record form and norms tables as well as other sections in manual's text. For example, Conceptual Thinking is listed only as a core subtest at ages 3 to 6. The KABC-II norms tables lists it as Supplementary at age 7. Table 1.2 describes both Atlantis delayed and Rebus delayed as Supplementary for 5 to 18. However, the norms tables list Atlantis Delayed as Supplementary for ages 5 to 8 and OOL for 9 to 18. The norms tables list Rebus Delayed as Supplementary for ages 6 to 12 and OOL for 5 and for 13 to 18. Below is a table that might approximates the KABC-II subtest constructs
QUESTION/COMMENT 6: Pattern Reasoning (core 5-6) and Story Completion (supplemental 6) are listed as Gv at those ages but switches to Gf at age 7. How strong is the evidence to support this cognitive shift? It seems very interesting that a test would switch from assessing one type of cognitive ability at one age but change to another type at a different age. Would a 6 year 11 month and 30 day old child administered Pattern Reasoning be using Gv skills, but the next day, at age 7 years, 0 months, and 0 days be using Gf skills? Page 16: “Gs is measured to some extent at ages 7 to 18 by the KABC-II subtests that include time points (Story Completion, Triangles, and Pattern Reasoning).” QUESTION/COMMENT 7: On pages 61 to 71, in the section of the manual that provides the narrow abilities of each subtest, there is no mention or description of any Gs narrow abilities for these or any other subtest? Page 16: “Rover is primarily a measure of Gv because of the visualization that is required, but it is also a measure of Gf…” QUESTION/COMMENT 8: According to page 65 it is also a measure of Gq Page 20: Figure 3.2 QUESTION/COMMENT 9: The description in the text (i.e., “green" = Knowledge/Gc subtests) does not match the colors printed in the figure. There is no green in Figure 3.3. This figure is the same as that on the back cover of Rover Stimulus booklet where the colors do match. Also regarding this figure:
Page 21: “Supplementary subtests other than Atlantis Delayed and Rebus Delayed should be administered after the core subtests have been given. . . “ QUESTION/COMMENT 10: This is true with the exception of Gestalt Closure where, at age 13-18, it must be given before Rebus Delayed. The explanation for giving Gestalt Closure before Rebus Delayed appears to be the need for an adequate time delay. There are 3 core subtests given after Rebus, just like all other ages. At ages 7-12, the three are Triangles, Word Order, and Pattern Reasoning. At ages 13-18, the three are Block Counting, Word Order, and Pattern Reasoning. Why the need to add the fourth, Gestalt Closure? Does Block Counting (possible 35 items with last 16 having time limits from 30 to 60 seconds) take much less time than Triangles (27 items with last 17 with time limits of 30 to 105 seconds)? QUESTION/COMMENT 11: It would have been helpful if the record form had included a note reminding examinees that at age 13-18, Gestalt Closure must be saved for before Rebus Delayed. Page 22: Description of discontinuation and scoring QUESTION/COMMENT 12: Page 21 notes that an examiner can begin at an earlier item. What do you do if a child, started at an earlier start point, arrives at the correct start point and pass 3 consecutive items? Page 22 notes “. . .do not give credit for any items that may have been administered beyond the stopping point dictated by the discontinue rule . . .” In the typical case this makes sense, but what if the examiner chooses to start at the earlier point. Should a child be penalized for an examiner's error in judgment? Page 25: “Except on Triangles and Word order, if a child who starts beyond Item 1 fails one of the first two scored items and as a result does not satisfy the basal rule, . . . “ QUESTION/COMMENT 13: Since the "Basal" rule states "first 3 items passed, or drop back one start point as needed", shouldn’t the sentence in the manual read “Except on Triangles and Word order, if a child who starts beyond Item 1 fails one of the first three scored items and as a result does not satisfy the basal rule, . . . “ Page 26: Time points QUESTION/COMMENT 14: Timing is important on 3 subtests since they allow "bonus" points for rapid correct answers. As with other tests that allow for bonus points (e.g., Block Design), the question arises as to when to stop timing – is it when the child says he or she is through or when the examiner sees that they are done? The easel says “Stop timing when the child communicates, either verbally or nonverbally, that he or she is done.” Nowhere in the directions does it tell the child to indicate when he or she is done. Page 26: Repetition of item: “On some subtest, it is permissible to re-administer easy items to a child who subsequently passes harder item in the same subtest.” “. . . you may return to easy items and give credit for correct responses.” QUESTION/COMMENT 15: This seems problematic both for scoring rule and examiner’s flexibility to change results. It could causes problems with the rule for discontinuation. For example, what do examiners do if they do go back and administer those items that caused discontinuance – can they now go forward again? Page 28: Out of level battery QUESTION/COMMENT 16: The way in which an examiner, using the out-of-level testing, computes the overall scores is incompletely explained. It seems unclear what tables an examiner should use to compute these scores? Take for example a 6 year old who is given the 7-12 year old battery. How would an examiner look up the Standard Scores for Gv, Gf, or Gc clusters? They are not the same for 6 or 7+ year olds. [The manual (p. 28) states “In all cases, as stated, the child’s profile of standard scores is based on norms for his or her chronological age.” This may be true for the subtests, but it can’t be true for the Composite standard scores. At age 6 there are no relevant tables. More explanation would be helpful Page 29: Subtests with Verbal Responses – “Record the child’s responses verbatim on the record form.” QUESTION/COMMENT 17: The text on page 29 suggests that one should record all responses verbatim for the three KABC-II subtests (Gestalt Closure, Expressive Vocabulary, and Riddles) but the example given on page 23, Figure 3.4 suggests otherwise. It is probably a very good idea to record all responses rather than simply recording the success or failure of the item. Unfortunately the manuals example doesn't follow the rule. Page 39: Figure 4.8 QUESTION/COMMENT 18: This is another minor comment about the manual not showing what the text describes. The graph of scores in Figure 4.8 is missing vertical lines as described on page 38 ". . .draw a short vertical line at the point corresponding to the index." Page 46: Percentile rank description QUESTION/COMMENT 19: The text used to describe the definition of percentile ranks seems to differ from others. “A child who scored in the 94th percentile on a subtest, for example, performed better than 94 percent of his or her age mates on that subtest; a child who scored at the 7th percentile surpassed only 7 percent of comparably aged children” Should this be “for example, performed equal to or better than 94 percent of his or her age mates on that subtest . . .” Pages 47-55: Interpretation QUESTION/COMMENT 20: The first step in interpreting the KABC-II relates to the examiner interpreting the global score. One wonders if this actually shouldn’t this be one of the last step? Shouldn’t an examiner evaluate all the scales to see if the overall scores are interpretable? What is several scales had significant differences between their subtests? Would it be appropriate to interpret the global score as meaningful? How would one know if they hadn't in fact interpreted all the scales, and then interpret, where relevant, the global score. QUESTION/COMMENT 21: Interpretation Step 3a and 3b: The two delayed subtests combine to create the Delayed Recall standard score. Note that there are no statistical properties (reliability, SEm, significant differences) given for the delayed subtests themselves. It is unclear why are there no comparisons between Atlantis vs. Atlantis Delayed and Rebus vs. Rebus Delayed? The only comparison available for delayed composite is that to Glr. For the two "scale comparisons" (Glr vs. Delayed and Glr vs. Gc) – there are no places on record form to record this information? Additionally, for Step 4, Supplemental subtest comparisons – there is no place on the record form to compute these. QUESTION/COMMENT 22: For the limited example of interpretation given in the manual, there is no explanation for what to do when it was determined that the Gsm composite is uninterpretable. What does the examiner do? The material in the manual instructs examiners not to interpret the scales alone. Can one use Hand Movements (or the appropriate supplemental) as a substitute when the subtests on a scale are discrepant? The example in the manual leaves one to wonder where to go or how to summarize the results? It seems unfortunate that the KABC-II manual has such limited explanations for interpretation. A lot of irritation has been expressed by test consumers about the need to purchase a separate (and delayed) interpretative manual for the SB5, or Alan and Nadeen Kaufman's wonderful books to interpret the WISC-R and WISC-III, or Jerry Sattler's text to interpret the Stanford-Binet 4. Must KABC-II users buy additional materials other than the test kits to obtain this material? Page 65: Rover – Regarding teaching the concept of diagonal moves, the manual notes: “This was accomplished by including sample items in which only a diagonal move is possible.” QUESTION/COMMENT 23: This comment seems a bit confusing or at least unclear. On the Rover subtest, the first sample item (A) shows 3 ways to get to the bone, only the last way, the one with the least number of moves, includes a diagonal move. Sample B also allow moves other than a diagonal? How is this “. . . including sample items in which only a diagonal move is possible.”? Page 71: Riddle is listed as measuring “General Reasoning” under the CHC narrow abilities. QUESTION/COMMENT 24: Examiners may be a bit confused by this notation. What is “General Reasoning” This name is not a name of any of the CHC narrow ability. Is it a mistake, and really meant to be General Information? or General Sequential Reasoning? or something else? There is an extensive list of CHC definitions found at : http://www.iapsych.com/chcdef.htm. Page 89: Standard errors of measurement QUESTION/COMMENT 25: What formula was used to compute the SEms? When one examines the table of reliabilities on page 88 and the corresponding table of Standard errors of measurement on page 89, some do not seem consistent? For example, the reliability of the MPI from ages 9 to 16 are reported as being .95 at each age The SEm for these same ages range from 3.14 to 3.37. It is probable that the SEms are computed using reliability coefficients carried out to X decimal points. There is no clear explanation in the manual. Miscellaneous points Page 104: Factor analysis following standardization. “The cause of this finding was that the Concept Formation subtest had a substantial loading on . . . “ QUESTION/COMMENT 26: the text on page 75 notes that 3 subtests included in tryouts were dropped before standardization – Concept Formation being one of them. Does the manual's text on page 104 really refer to Conceptual Thinking? Correlations with other cognitive tests: page 116 QUESTION/COMMENT 27: In Table 8.22, the results of the correlation between the KABC-II and the WJ III are presented. It is unclear from this table if the cluster comparisons (e.g., KABC-II Gv and WJ III Gv) are really cluster comparisons or subtest comparisons. Since the WJ III global score is reported as the GIA standard, one might assume that this table really represents the relationship between the KABC-II and single subtests of the WJ III. What is listed as WJ III clusters may only be single subtests (e.g., Gc= Verbal Comprehension, Gv= Spatial Relations, Gf= Concept Formation, Gs= Visual Matching). Table D.2 pages 174 – 201 for looking up Index scores QUESTION/COMMENT 28: The tables in the manual for looking up FCI and MPI appear reversed from that of the record form. Care must be given so as not to look up the FCI in the MPI column. Tables D.2 pages 174 – 201 for looking up Index scores: for ages 7-18 the tables are divided into 3 levels – 7-9, 10-12, 13-18)
QUESTION/COMMENT 29: This makes sense since each index is made up of the same subtests Gc - same lookup values only for ages 10-12 and 13-18 QUESTION/COMMENT 30: Why is the lookup values table different at ages 7-9 if the same 2 subtests (Verbal Knowledge & Riddles) make it up the scale at all three ages Gv - different lookup values at each level QUESTION/COMMENT 31: Why isn’t the lookup table for ages 7-9 and 10-12 the same – at those ages the Gv cluster is made up of the same subtests (Rover & Triangles). On record form, only the Gv index is different at 7-12 and 13-18) Record form: Qualitative Indicators (QI): QUESTION/COMMENT 32: Page 2 of the Record form lists only 9 QIs in the chart for recording them. There are many more not included. The manual provides no explanation as to how to use the chart or the QI? Does one get a percentage of the total? Page 65 in Manual – Description of Rover subtest QUESTION/COMMENT 33: The description of QIs is missing two that are listed on the record form? subtest administration: Face Recognition: On an item where there are two faces to be identified, if the child points to only one person, say Remember, you have to point to two people. Give credit if the child responds correctly after prompting. QUESTION/COMMENT 34: Would this rule apply in all cases or only if the first face is correct? If the first is wrong and an examiner give the prompt, what does one do if the child changes the first response? Gestalt Closure: If the child has not responded after about 30 seconds, say Make a guess and go on to the next item. Give credit if the child responds correctly. QUESTION/COMMENT 35: Should the examiner wait for the guess after asking for one? QUESTION/COMMENT 36: Again, as noted above, it would be useful to have a note on the Gestalt Closure page to indicate that if Rebus Delayed is going to be given the examiner must delay administering this subtest. Rover: The counting is to “help the child do the task and to help the examiner monitor the child’s response, but it is not essential and does not need to be accurate. However, for items 4 and 5, counting aloud is critical, because it is the only way to verify that the child understands that a rock counts as two moves.” QUESTION/COMMENT 37: There seems to be some lack of clarity regarding this section of the test. On items 4 and 5 if children do not count correctly, even if they gets the correct route, they are to be given a score of 0. The easel does note that “it is very important for the child to count on sample B and items 4-5. If necessary, encourage the child to count." Can this encouragement be done during the child’s actual response to the sample and to other items? How would one score a response in which a child miscounts but does understand the concept of multiple steps. For example, on item 4, the child miscounts “1, 2, 3, 4” but included the starting point? QUESTION/COMMENT 38: Be careful not to count the squares that are outlined on the easel as the correct path – these outlined squares include the start point which is not counted in the moves. For example, item 1 has as a correct number of moves 3, yet 4 boxes are outlined. Rebus QUESTION/COMMENT 39: Why is the rebus for “the” not scored after item 7? There is no explanation in the manual. Triangles: easel page: “Any arrangement of adjacent triangles of the same color is acceptable if it matches the stimulus design.” QUESTION/COMMENT 40: This direction seems very unclear. What does that mean? Does it mean that a child can totally reverse the colors and get credit? Is it only the shape that counts? Triangles: “Place the exact set of plastic or foam triangle needed for the item in front of the child, in a random array.” QUESTION/COMMENT 41: Do not confuse the easel description labeled “Child set” as the order in which to layout the pieces. Block Counting: “The child may touch the picture.” QUESTION/COMMENT 42: It might be useful to give the child a wooden dowel for pointing. Having children point and touch the pictures will result, over time, in smudges on the pages. Pattern Reasoning: If the child spends a great deal of time on an item and does not seem to be making progress, say Make a guess and go on to the next item. Give credit if the child responds correctly. QUESTION/COMMENT 43: Since this is an untimed task, how long is a “great deal of time”? Should you wait for the guess after asking for one? KTEA-II Manual – page 32 – Measuring growth QUESTION/COMMENT 44: There is no explanation at all for how to use the GSV scores when interpreting or using the KTEA-II. Norms Book -Page 303 –Table N8 QUESTION/COMMENT 45: Since the formula for a stanine is S9 = 5+2(z), it appears that several break points between stanines are possibly incorrect:
Grade versus age norms: QUESTION/COMMENT 46: Should there be a caution in the manual about the effects of using the grade based norms (separated into two categories (Fall and Spring), especially for the younger grades. For certain subtests, at grades K through 3, there are often dramatic differences between the standard scores based upon the Fall norms versus those obtained using the Spring norms. For example, on Letter & Word Recognition, a child in the first grade who is tested on January 31st would obtain a standard score of 100 for earning 16 raw score points. That same child tested one day later, on February 1st, would obtain a standard score of 89 for earning the same 16 raw score points | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||