Friday, July 30, 2010

Test Scores: Worse than you Think

This week was actually a perfect time to release the new, revised test scores for a number of reasons. School is out, the mayor is back in, the governor is on his way out, and the Unity Crew of the UFT has already agreed to using test scores to evaluate teachers despite the fact that no one knows what the hell they mean, much less how teachers can "add value" to them.

Perhaps more importantly, test scores just could not be pumped up any further. At some point, they had to be readjusted. Now happened to be a good time because of all the above.

Unfortunately, I believe we are wrongly assuming that because the scores have been readjusted, they are now accurate. Nowhere have I seen the formula the state used to draw the cut lines where they did. Nor did the state disclose whether the scores would have gone down even without the change in cut scores. (If anyone knows how to find this information, please let me know.)

If the state willfully dumbed down the tests and changed the cut scores for years and years, as it now appears they did with gusto, what on earth is there to make us think the current scores are truly accurate?

Think of it this way. If the tests last year were easier than ever, resetting the cut scores to where they were four years ago still does not reflect the reality of student reading skills. And one more thing that disturbs me about the current numbers is that no one is discussing what happened on the writing part of the exams.

I scored three different writing tests in three different grades this year. Anyone who was there can verify that teacher-scorers were always told to give students the benefit of the doubt in grading. I first blogged about it here when I described how a single run on sentence could be graded a 3 (the highest score for mechanics). When grading the actual answers we were told to ignore the mechanics completely and give a 4 out of 5 (five being the highest score) to students who simply addressed the question, and a 5 if there was some evidence of "voice". A number of teachers who were deemed to have graded too leniently were publicly called out by the poobahs running the center and were told give higher scores. This was allegedly done in the name of making sure we were all grading students fairly, i.e., that we were all giving them the same inflated grades for the same substandard writing.

So even if we assume that the multiple choice portion of the ELA tests were re-calibrated correctly, which is a rather large assumption, it remains true that the the writing portions of the tests were inflated beyond all reality.

The math teachers I have spoken with told me the same thing happened to them when they were told to give high scores to students on their constructed responses. Teachers were told to give medium to high scores as long as students showed "some evidence" that they understood how to answer the question, regardless of whether they actually got the problem right.

So while it's a good thing that the scores were reset, it's pretty clear to me that they are still too high. And that is a bad thing, because come the 2011 school year we will all be evaluated on our ability to increase those scores.

No comments: