I know the friend's cousin's Counsel - A comment on the Report of Assessment Results 2009 General Diagnostic According to the analysis of data obtained by TEACHERS of the 50 sites evaluated by the Valencian Institute for Evaluation ... [see PS]
data EGD-2009 are not public. Those interested in educational evaluation have to do an act of faith that have been tested properly and we can not ask additional questions that arise from the Appraisal Institute.
Were properly analyzed? We know there
incorrect analysis. For example, multiple comparisons are made mean, but pairwise comparisons, triggering the risk of errors. Even those pairwise comparisons are wrong. Likewise, we know that the Rasch model is wrong when a group of items sharing a set. In this case, violates the assumption of independence is necessary to model local and the correlation between the specificities of the items with an additional parameter
. In this case, the Cronbach alpha coefficient is an inadequate indicator of the reliability of the scale.
Elsewhere, we have questions. For an important part of the results presented, we ignore the statistical techniques employed. The field of measurement in education is not static. Has not reached the point where there are a few methods known and recognized as the best and always be applied. There are several magazines:
Applied Measurement in Education, Applied Psychological Measurement
, Educational and Psychological Measurement
,
Journal of Educational and Behavioral Statistics ,
Journal of Educational Measurement. Only a detailed analysis method to assess whether we have incorporated methods that have shown better performance. One can not be released differential item functioning saying "The procedure used was to compare the difficulty parameter of items within the comparison groups "(p. 36)." Raju method or Thissen? Only took into account also the statistical significance or effect size?
respect to differential item functioning, greatly surprised comments like: "The results of the Basque Country in the competition in knowledge and interaction with the physical world do not appear within the graph by a differential operation items Euskera version" (p. 66). In international reports, such as PISA, is achieved compared to many countries, many with language disparity equal to or greater than what can found between English and Basque. How can the Appraisal Institute has failed in this? We do not know if they fail all items, to what extent they do, if you save some questions that might help to establish a common metric ...
Additional Questions? And here the origin of this annotation. Many / several / some researchers who would like to make a use of data looking for answers to new questions or trying to verify that previous responses are well founded. But only some can.
Now comes the beautiful game "I know a. .." "You will pass the database because I know that your questions will get you the answers I want" or "what you give me in return ...?". This is the idea of \u200b\u200btransparency, promotion of research or evidence-based social policies that have our politicians and managers.
not make public the data seems like a joke. That already have reparse start about people who do that have happened to me like another data joke.
PS (23/06 - 18:40). My apologies to Pablo Rovira, the author of the news journalist of Education. I've had very little touch to title this entry, so that could be understood to accuse Mr. Rovira 'Dirty tricks' or subservience to get the data. It was never my intention.
wanted to criticize that, by not publishing the book, opened a (potential) few clear ways to capture them. I do not criticize those who succeed, but to those who have them and not distributed. Obviously, the work of a journalist involved with their profession is to try to get them and I have no reason to question the way you have come to them Mr Rovira.
