Are State Test Scores Inflated? Are the Tests “Valid and Reliable” Measurements of Achievement and Progress? A Totally Unscientfic Theory.


On a muggy, unbearably hot June day we marked paper after paper for hours upon hours: that regents marking chore. One group trudged down to an office and waited their turn to feed the bubble sheets into the Scantron. The rest of us marked hundreds of papers, the same essay in each blue booklet. The last group added up the scores and entered the grades into the database.
It’s a nerve wracking time for teachers … how will “your kids” do? After all, in a way you are the coach of the team, you didn’t take the exam, but your “players” did!  All those hours preparing lessons, urging, cajoling, pleading, even occasionally threatening students, and now the culmination of all those efforts.
It’s the same in elementary schools, except the time lag from the test to the release of the scores is many months.
The NY Times raised the question of the accuracy of NYS testing  and re-ignited the embers.
Critics have averred that the tests do not accurately measure student achievement and progress. From scholars: Diane Ravitch, Aaron Pallas and Jennifer Jennings, to journalists: Sol Stern and Andy Wolf, to parent advocates: Leonie Haimson, who sharply criticized the NY Times article, calling out the writers for their falsely “balanced approach,” and ignoring many glaring errors.
The dissenters point to the NYS 2003 testing gains, before the Klein Children First initiative, are “claimed” by Bloomberg/Klein and, remind us of the disparity between the National Assessment of Student Progress (NAEP) scores and the NYS tests.
Gotham Schools finds that calls for an investigation of NYS testing are unanswered.
State Board of Regents Chancellor Meryl Tisch is calling for state exams to be more “defensible,” but a study investigating test score credibility requested a year ago by the state’s testing oversight board has still not received a go-ahead.
The defenders of the faith, actually Howard Wolfson, the leader of the mayor’s re-election campaign, no expert on testing, has raced to the support of the mayor.
I am not a testing expert.
I would urge Chancellor Tisch and Commissioner-elect Steiner to order an investigation.
However, I have another theory re the increase in scores: the ready availability of technology to schools and teachers.
As soon as the state tests scores are announced a school leader can download the scores from the NYSTART, the SED testing site. Downloading the results into an Excel format, disaggregating the items and creating an error matrix. The school leader can look at “errors” by whole school, by grade, by class, by gender, by “group,” share the data with the staff, and urge the staff to create lessons to address the student “weaknesses.”
Is the school leader “gaming the test” or using “data to drive instruction”?
We used to call it “test sophistication,” student familiarity with the types of questions they will face on the state tests.
Athletic teams win due to talent and coaching. Coaches can recruit talented players, for teachers, the “players” are who walks in the door, its the coaching that counts.
We would hope that the tests are “valid” and “reliable,” and accurately measure student achievement and progress, however, we do what we have to do with our kids, giving them the skills to succeed on the tests.



Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s