Can We Measure Teacher Effectiveness By Using Student Test Scores? How Do We Meld Principal Judgment and Test Scores? Is It Too Late for Obama/Duncan to Build Trust Among Teachers?


We live in a world dominated by numbers. Sports teams either win or lose, players are measured by a host of data, baseball has a name for it, sabermetrics. In the social sciences we also collect enormous data sets. Policy makers crunch the data, using tools such as regression analysis, to look at a set of variables and make predictions.
For millennia teachers have been evaluated by their supervisors who sit in the back of the classroom and write a summary of the lesson with a summative evaluation. The New Teacher Project “Widget Effect Report ” tells us that traditional teacher evaluations are a futile exercise,

All teachers are rated good or great. Less than 1 percent of teachers receive unsatisfactory ratings, even in schools where students fail to meet basic academic standards, year after year.

Excellence goes unrecognized. When excellent ratings are the norm, truly exceptional teachers cannot be formally identified. Nor can they be compensated, promoted or retained.

Professional development is inadequate. Almost 3 in 4 teachers did not receive any specific feedback on improving their performance in their last evaluation.

Novice teachers are neglected. Low expectations for beginning teachers translate into benign neglect in the classroom and a toothless tenure process.

Poor performance goes unaddressed. Half of the districts studied have not dismissed a single tenured teacher for poor performance in the past five years. None dismiss more than a few each year.

Traditional teacher observations, if we accept the New Teacher Project report, are mechanical and ineffective, if so, how can we judge teacher effectiveness?

One method is to focus on output rather than input, rather than observe the “teaching” side of the equation focus on the “learning” side, pupil achievement. The analysis of student achievement data, some claim, will allow us to place a score or value on the impact of learning attributable to an individual teacher, called value-added.

The Value-Added Research Center (VARC) at the University of Wisconsin is assisting school districts,

Once the value added by teachers, principals, intervention programs, and school reform efforts can be measured, it is possible to use this datum to make decisions about staff professional development, about educator compensation programs that can increase student success, and about whether or not specific interventions are valuable. All these decisions can be used to initiate and support educators’ efforts at school reform.

The New York Department of Education is beginning the third year of a value-added model, each teacher in grade 3-8 (who teach ELA and Math) receives a “grade” and a percentile ranking among other teachers teaching similar students in similar schools, called Teacher Date Reports ( see here for samples and an explanation).

The Los Angeles Times , with the support of Secty of Education Duncan, will release today the value-added scores for 7,000 elementary school teachers to the public, and have published the views (see here) of a number of scholars as well as a detailed explanation of the value-added data methodology.

Dan Willingham, in a brilliant 4-minute U-Tube  seriously questions the entire value-added model to evaluate/rate teachers.

The Wall Street Journal, no friend of teacher unions, publishes a detailed analysis of a number of value-added studies, with a number of caveats,

Teachers have reason to fear they may be misidentified — roughly one in four would be even after three years of data have been collected, according to a report last month commissioned by the Department of Education. Hanley Chiang, a Mathematica Policy Research researcher who co-authored the report, said, “How tolerable these error rates are, is really a policy judgment.”

“Even with multiple years of data, there are a whole lot of false positives and negatives,” said Barnett Berry, president and chief executive of the Center for Teaching Quality. “If we start using these value-added metrics inappropriately, what could have been a very powerful tool could become the baby that gets thrown out with the bathwater.”

Deborah Stone, in Policy Paradox and Political Reason (1988) maintains that choice-making invariably entails politics. Policymakers  “…start out …with a politically attractive solution … they may ignore research and explanations … “

Reasoned analysis is necessarily political. It always involves choices to include things and exclude others and to view the world in a particular way when other visions are possible.

The American Federation of Teachers has responded with a detailed position paper on teacher evaluation. On the ground AFT locals around the country have negotiated contracts with “multiple measures,” that include some percent of a teacher assessment to be based upon value-added data.

Christine Amanpour, on ABC discussed these issues with Duncan, Weingarten and Rhee, take a look here, Duncan and Weingarten seem to be agreeing a lot more than disagreeing.

A couple of years down the road Diane Ravitch will be writing an update of her The Death and Life of the Great American School System: How Testing and Choice Are Undermining Education,(2010), I can’t wait to read the autopsy, the recounting of the failures in New York City and the missteps of Obama/Duncan politically driven educational policies. Arnie’s wooing of Weingarten may be too late to prevent the upcoming Novmber debacle.

UPDATE: See the Economic Policy Institute criticism of the use of Value-Added Modeling (VAM), aka, student achievement data, to assess/rate teachers here.



One response to “Can We Measure Teacher Effectiveness By Using Student Test Scores? How Do We Meld Principal Judgment and Test Scores? Is It Too Late for Obama/Duncan to Build Trust Among Teachers?

  1. it is way too late for obama

    It is way too late for Obama. I cannot wait to vote against him and his charter garbage and his race to the cash garbage. Since then, I have many many groups and emails of organized people that will do the same. What a mess education is in and what blatant corruption from the hedge funds, from foundations, from billionaires. I have not been proud of my country for a long time under Bush and I did not think it could get worse….but it has. The poor children…it is not about the children at all. It is about all the rich making money. Show me charter school transparency….it does not exist. They pay themselves what they want from public money and no one is checking this? Even the State Ed departments are corrupt and on the take. The unions are also corrupt! My have we taken a nose dive in education. No community voice, no parent voice…just absolute anti-democratic control (mayoral control) and blame it ONLY on the teachers. Hey, ya got to blame someone! While the rest who are causing the problems…sit back and collect. Shame on Obama and our elected officials. America will not heal from this damage.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s