Comments on LearningRx

3 million visitors plus:

Age equivalents are not a proper way to show meaningful change. There is no data to tell who the people were in the sampling, no breakdown by ages, no explanation of the time span between testing, etc. In order to try to understand the results, I tried to “back calculate” the standard scores.

Using data provided in the Statistical Analyses that can be downloaded from the programs site (http://www.learningrx.com/center2/), I estimated the actual standard score change from pre-test to post-test for an 11.5 year old (mean age reported in sample characteristics) regarding the Combined Programs.

For the combined groups, the claim for age equivalent changes was from 2.58 to 3.99 years. However, the standard scores most close resembling the age equivalent at pre-test ranged from 96 to 103 and at post-test from 102 to 108. Standard score point changes ranged from 3 points (Spatial Relations) to 9 points (Pair Cancellation). Raw score changes were generally small, changing from 1 to 10 absolute value points. If one examines the standard errors or measurement for the average 11 year old, the SEm range from 3.64 to 7.43. These represent the 68% confidence level and so I multiplied each one by 1.96 to give the standard error of measurement needed at the 95% confidence level. Note that the SS change needed to “beat” the error associated with the SS score was only large enough to claim a SS change beyond chance for Concept Formation (the SS change of 8 points is larger then the 7 points of error).

Examining the data regarding those who started out 2 years below “age”, SS changes for Visual Auditory Learning, Concept Formation, and Numbers Reversed “beat” the error associated with the standard score.

Another table later in the Statistical analysis paper shows changes in percentile ranks from pre to post. Again, changing the percentiles to standard scores demonstrates average increases on 3 WJ III tests that do not “beat” the SEm.

These are only estimates of the actual changes associated with the intervention.

	COMBINED PROGRAMS (ALL THINKRX PARTNER, READRX PARTNER, READRX PRO)
	ANALYSES OF PRE-/POST- DIFFERENCES ON COGNITIVE MEASURES

	Average Pre-Test			Average Post-Test			Average change in			68%	95%
WJ III Test	RS	AE	SS	RS	AE	SS	RS	Years	SS	Sem	Sem
Visual Auditory Learning	13	10.24	96	9	13.97	104	-4	3.73	8	5.87	12
Spatial Relations	68	12.73	103	70	16.72	106	2	3.99	3	7.43	15
Concept Formation	27	11.19	100	31	14.66	108	4	3.47	8	3.64	7
Numbers Reversed	12	10.31	98	13	12.89	102	1	2.58	4	5.87	12
Pair Cancellation	59	10.74	96	69	13.47	105	10	2.73	9	6.27	12

	COMBINED PROGRAMS
	(STUDENTS WHO PRE-TESTED TWO OR MORE YEARS BELOW AGE EQUIVALENCY)

	Average Pre-Test			Average Post-Test			Average change in
WJ III Test	RS	AE	SS	RS	AE	SS	RS	Years	SS	Sem
Visual Auditory Learning	23	7.84	82	10	13.04	104	-13	5.20	22	5.87	12
Spatial Relations	60	8.46	91	69	14.68	104	9	6.22	13	7.43	15
Concept Formation	17	7.99	89	28	12.38	102	11	4.39	13	3.64	7
Numbers Reversed	10	8.30	88	13	11.56	102	3	3.26	14	5.87	12
Pair Cancellation	56	10.37	93	69	14.59	105	13	4.22	12	6.27	12

RS= raw score associated with the Age equivalent (AE)

SS= Standard score associated with the raw score

Sem= Standard error of measurement as reported in the WJ III technical manual for age 11