Cutting Scores: How Will the State Tests Impact Kids and Teachers? Can We Trust the Scores?

In a week or two the State will release the grades 3-8 ELA and Math test scores – and – the Department and the State has been preparing the public for significant lower scores.

Two questions:

How does the State Education Department set “cut scores”?

How will lower student scores impact teacher grades on the APPR (teacher evaluation)?

At today’s Regent meeting the SED data guru Ken Wagner and the Regents Research Fund psychometrician Kristen Huff (see bio here) spent an hour explaining the process for setting cut scores for the current state exams.

Check out detailed power point: http://www.regents.nysed.gov/meetings/2013Meetings/July2013/StandardSetting.pdf

Ninety-five teachers, administrators and college profs spent four days reviewing the test items and deciding whether they are level 1, 2, 3 or 4 difficulty questions, thirty-four of the ninety-five spent an additional day reviewing the original findings and made minor adjustments – the slides reported, “the Commissioner accepted the recommendations of the panel.”

Not wanting to be overly cynical, but, who were the 95 reviewers? Who selected them? Why weren’t the names in the report? Will outside experts review the Pearson technical report? (they only commented on the process).

A leading expert suggests,

… the scale scores, and the cut scores which separate them into proficiency levels, can only be interpreted in the context of the psychometric properties of the test (as distinct from the properties of the standards-setting process.) I’d want outside experts reviewing the Pearson technical report on the properties of the test before scores are released. This is especially important since the 2013 tests are so different from the tests in prior years.

The State made no commitment to follow through on the Pearson technical report. The State, unfortunately, has a history of misusing cut scores to show unjustified progress -they would be ill-advised to be secretive.

The Commissioner and the Chancellor Tisch have reminded us time and time again that the new test is measuring different knowledge and skills and can’t be compared to prior tests.

The test will set a new baseline which will be compared to subsequent tests – it is a harder test. The “unfairness” is that some school districts have been implementing the Common Core for three years, others for two or a single year and some just beginning. Wealthier districts purchased materials and professional development, others have not.

What a surprise – higher wealth district/schools will have better prepared teachers and kids.

The Commissioner came close to acknowledging when he asked local superintendents to “be thoughtful” in using the test score info.

How will lower student scores impact teacher grades on the APPR (teacher evaluation)?

The Commissioner has emphasized over and over again that student test scores are not determinant of teacher scores on the APPR – teachers “compete” against each other based on student growth data while students’ scores are criterion-referenced, based on getting items right or wrong regardless how the kid at the next desk performs.

The Commissioner has already determined that 6% of teachers will be in the “ineffective” band and 7% in the “highly effective” band.

In the 20% student test test score category if one class moves from 4.3 to 4.0 and another class from 2.3 to 2.1 the teacher in the latter category will end up with a higher score. (one class loses .3 and other .2).

The fear that “I teach in a high poverty school,” or, “I teach English Language Learners” does not presage a lower score.

The NYC Department posted a detailed explanation of the teacher observation requirement which counts 60% towards a teacher’s APPR score. Check out here.

Interestingly the Department has yet to post the “state and local measures of student learning,” the remaining 40% – at one of the “Advance” training sessions a principal asked, “If I understand this correctly I can use math test scores to assess my music teacher.” The trainer, hesitantly, agreed, Stay tuned!!

Advertisements

4 responses to “Cutting Scores: How Will the State Tests Impact Kids and Teachers? Can We Trust the Scores?

  1. Test scores like everything else is another tool for teachers to use to evaluate and help their students improve achievement. Test scores like everything else is another tool for principals to use to help evaluate and help their teachers to improve their effectiveness. Think of this scenario; 2 teachers from the same school who have the same program end up with significantly different scores. The variables have been eliminated- same school, same pupil population. What is the difference that makes one teacher highly effective? That teacher teaches and REACHES his/her students on a human dimension that can be observed and identified but not duplicated because teaching/learning is an art form and not a science. Had anyone duplicated Beethovan”s 9th Symphony?

    Like

  2. Pingback: Remainders: Former StudentsFirst flack misstepped in new role | GothamSchools

  3. Pingback: King won’t change cut score advice for new Common Core tests | GothamSchools

  4. Peter:
    Now you know how the members feel when the UFT leaders don’t tell us who are on the various committees and negotiate without member input. Welcome to the club.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s