Broken Cappilaries On Upper Cheek

confidence intervals and mean - A comment on the Report of Assessment Results 2009 General Diagnostic

Blog Thanks to Jose Manuel Lacasa which I find out the results report the General Diagnostic Assessment 2009 . The temptation is too great and, although the work overwhelms me (including week and a half contest Assistant Doctor), I rush to review the report.

'll comment a single point, technical and statistical cutoff. On page 62 of the report reads:

among all communities with average scores next (with differences such that confidence intervals partly overlap) nothing can be said about the "order" that some have compared to others.

Translated: when the confidence interval of two means overlap, the difference between the two is not statistically significant.

Too bad that this sentence is false. Interested readers can consult, among others:

• Austin, P. C, & Hux, JE (2002). A brief note on overlapping confidence intervals. Journal of Vascular Surgery, 36 ,194-195.
• Cumming, G, & Finch, S. (2005). Inference by eye: Confidence intervals and how to read pictures of data. American Psychologist, 60 ,170-180.
• Schenker, N, & Gentleman, JF (2001). On judging the Significance of Differences by Examining the Overlap Between Confidence Intervals. The American Statistician, 55 , 182-186.
• Wolfe, R, & Hanley, J. (2002). So Different If we're why do we keep overlapping? When 1 plus 1 Does not Make 2. Canadian Medical Association Journal, 166 , 65-66.

As pointed a pair of researchers in the title of an article Researchers misunderstand confidence intervals and standard error bars .

An example will illustrate how two confidence intervals can overlap and, nevertheless, that the mean difference is statistically significant. The data have here. This is the same variable measured on two different groups, with 25 observations per group.

• The group 0 has an average of 500, with a confidence level of 95% between 415.7 and 584.3.
• Group 1 has an average of 650, with an interval of 95% confainza between 565.7 and 734.3. Group 1 is the result of adding 150 to each observation group 0.
  

a graph here shows the same information and see how the confidence intervals overlap.

When we make a comparison of means of two independent samples, we found that ... p = 0.01241 < 0.05. Luego rechazamos la hipótesis nula y podemos afirmar que la diferencia entre las dos medias es estadísticamente significativa.

Serve this as an example that, once again, the Evaluation Institute analyzed the data with a strictly lower than expected. The Evaluation Institute has statistical gaps psychometric and preventing them from making the best decisions of analysis at all times and that sometimes leads them to questionable conclusions.

Certainly, this is not important for a rather simple reason:

• The report is politically irrelevant. This country has given up an education policy based on evidence. Do you think that will have any effect which the report says, explicitly, that increased use of new technologies does not improve performance? Or that private schools provide equivalent results to the public (after controlling for the effect of socio-economic-cultural students) but with a ratio pupil / teacher much higher, that is, that private schools are more efficient?
• The report does not ask what's important. The report asks for all that we can not control, where he studies people who have studied their parents, where they were born ... But do not go into teaching methods which lead to results. We can not know whether this philosophy of education is better than another, because not within the questionnaire. And it's much easier to teach the best way to teach immigrants to mutate national.

back to my hole.