Restoring Trust: Can the Regent/SED, Unions and Superintendents Agree on a “Valid. Reliable and Fair” Teacher Evaluation System?

As part of my union rep duties I served on committees to select teachers for new schools; my favorite question was, “What was the best lesson you taught in the last few weeks and how do you know it?” Teachers had no problem describing lessons, and lots of trouble explaining how they could assess the effectiveness of the lesson. A few would say “exit slips,” others explain that checking the homework assessed the effectiveness of the lesson, or that experience was the best guide. In the real world we plow through the curriculum, an occasional unit exam, differentiating lessons, re-teaching concepts, understanding that god in her wisdom did not make us or our students all equal.

We have been giving statewide exams for decades, before No Children Left Behind, in grades four and eight, and Regents exams are more than a century old. Yes, we did have a triage system, classes were homogeneously grouped and students who failed to make academic progress were placed in classes with similar kids. At the end of the line kids dropped out or received a lesser diploma, and moved into the workforce. Low-skilled union jobs were commonplace, the education system was the “divider” which steered kids to college or the world of work.

For the last thirty years our economy has undergone structural changes, the low-skilled union jobs have fallen victim to automation or moved overseas.

Are schools appropriately preparing students for the rapidly and continuously changing world of work, or, to use the commonly used term, are we preparing “college and career ready” students?

The answer begins in the world of baseball.

I was attending an education conference, and as commonly happens, a book was handed out, not a dense text, not a “how to” book, not a messianic message, the book was Michael Lewis’ Moneyball: The Art of Winning an Unfair Game (2003),

Moneyball is a quest for the secret of success in baseball. Following the low-budget Oakland Athletics, their larger-than-life general manger, Billy Beane, and the strange brotherhood of amateur baseball enthusiasts, Michael Lewis has written not only “the single most influential baseball book ever” (Rob Neyer, Slate) but also what “may be the best book ever written on business” (Weekly Standard). [Lewis’] … intimate and original portraits of big league ballplayers are alone worth the price of admission—but the real jackpot is a cache of numbers—numbers!—collected over the years by a strange brotherhood of amateur baseball enthusiasts: software engineers, statisticians, Wall Street analysts, lawyers and physics professors.

Baseball is no longer ruled by the cigar chomping old-timers, every decision, from salary negotiations, to valuing players, to comparing players, to which pitch to throw to each batter is guided by a mathematical algorithm. As a wonky baseball friend claims, “You could prop Bernie in the corner of the dugout (“Weekend at Bernie’s“) and IBM’s Watson could manage the team.”

We even have a term for baseball data; Sabermetrics, a hobby among a handful of nerds now rules the national pastime.

The widespread use of data has not solved the problems of baseball, the numbers of Afro-American ballplayers and fans has sharply diminished, the fan base is aging out; data may drive decisions, the American pastime is facing a ticking clock.

See Chris Rock on baseball: http://deadline.com/2015/04/chris-rock-baseball-real-sports-with-bryant-gumbel-hbo-video-1201414387/

Whether we like it or not, understand it or not, data drives decisions across a wide spectrum. Ian Ayres, Super Crunchers, Why Thinking by Numbers Is the New Way to be Smart (2007) chronicles how data has embedded itself, from predicting the quality of red wines, to driving the medical profession, to determining which prisoners should be paroled, dense regressive mathematical models rule.

It is not surprising that data influences core decisions in education.

The New Teacher Project 2009 “Widget Effect” report resounded across the education domain,

All teachers are rated good or great. Less than 1 percent of teachers receive unsatisfactory ratings, making it impossible to identify truly exceptional teachers.

• Professional development is inadequate. Almost 3 in 4 teachers did not receive any specific feedback on improving their performance in their last evaluation.

• Novice teachers are neglected. Low expectations for beginning teachers translate into benign neglect in the classroom and a toothless tenure process.

• Poor performance goes unaddressed. Half of the districts studied have not dismissed a single tenured teacher for poor performance in the past five years.
The result has been the movement to assess teacher performance by applying dense mathematical regression models to education, attempting the compare teacher to teacher based upon student achievement data, using a range a variables to level the playing field.

In other words, there is no teacher evaluation system.

The new current systems attempt to address the absence of systems using statistical methods called regression analysis. The students take a common exam, a state test for example, the model allowing for a range of variables, namely, economic status of test takers, students with disabilities, the level of disability, English language learners, student attendance, etc., and the formula differentiates among teachers, within a margin of error.

As with all statistical data sets there are errors of measurement – that “plus or minus” that warns us that the results fall within a range.

“Candidate A leads candidate B 52-48 with an error of measurement of “plus or minus 4%”,” a statistical tie.

As school districts begin to create the models, usually referred to as Value-Added Models or Growth Models, “experts” warn about the problems of VAM. At the NYSED Education Learning Summit three education experts were critical, VAM data was not ready for prime time and a fourth expert argued that VAM was better then what preceded and suggested a combination of VAM student performance data, teacher observations and student surveys.

By mid-June the NYS Board of Regents/SED have to create regulations to implement another new teacher evaluation system in New York State.

The current teacher evaluation law has been on the books for three years (two years in New York City), and has raised more questions than answers.

* Are teachers in Rochester and Syracuse less able or is the algorithm flawed? (A law suit is in progress)
* Are teachers of poorer students (Low SES), English language learner and students with disabilities less able, or, is the algorithm, the formula, flawed?
* Conversely, are teachers higher income (High SES) district more able than other teachers?

On Monday the Regents will begin an in-depth discussion of the new Matrix model (See the Education Learning Summit page here)

The movement to Common Core tests was a disaster, probably too mild a term. Instead of phasing in the Common Core-aligned tests the decision-makers, led by Commissioner John King, used the “push off the end of the diving board” approach. Randi Weingarten asked King to incorporate a “save-harmless” for a year or two, to no avail. The result: over 100,000 parents opting-out, the opt-out movement is spreading across the nation, teachers are highly suspicious of everything and most importantly, electeds are threatened and are introducing legislation to weaken SED initiatives.

The Regents have four new members, three former superintendents and a former Buffalo school board member, who join former superintendents Cashin, Rosa and Young. Regents Cashin and Rosa voted against the original teacher evaluation plan.Hopefully their experience can put the train back on the tracks.

How can the Regents/SED win back the confidence of parent and teachers?

The complex numbers cannot be seen as a tool to punish teachers.

The movement to Common Core-aligned test ignored the impact of drastic drops in test scores, and, when the public expressed discomfort John King blamed parents and “outside agitators;” a classic example of how NOT to roll out a new initiative.

Advice: Accept the recommendations of NYSUT, the state teacher union, the UFT, the NYC teacher union and the NYC Department of Education., in other words, create buy-in, remember Rule # 1 of “change,” participation reduces resistance.

Each year a technical committee that includes the unions can review and modify the model. Shoving new proposals down people’s throats and vigorously defending the position hasn’t worked out too well. Incremental change builds trust.

A simple example: A teacher receives a score of 42 on a model that has an error of measurement of plus or minus 10 %. In other words the teacher’s score could be within the range of 32 to 52. If the cut scores between D (Developing) and E (Effective) is 50 the teacher should be graded E (Effective). We should accept the error of measurement issue and not disadvantage the teacher.

Similarly, on the observation rubrics, we should accept the recommendations of the unions and the NYC Department, set low cut scores and examine the results each year.

If the Regents/SED decides to select cut scores that are “tougher,” the teacher and parent wars will escalate and electeds will jump on the “voter” side, the Opt-Outs, the trash the system side. Charter schools, voucher and tax credit supporters will argue public schools are not fixable.

Just as Chris Rock points out re the underlying problems of baseball, the underlying problems of teacher quality will not be resolved by teacher evaluation regulations; however, the Regents/SED need to be standing on a stage with NYSUT President Karen Magee, UFT President Michael Mulgrew and NYC Chancellor Carmen Farina joining together announcing a teacher evaluation plan that is “valid, reliable and fair.”

Regaining a lost trust is crucial and must precede any further steps, until we can trust each other we cannot move forward, and, with the vandals at the gates, Regents/SED must take the first step.

Advertisements

2 responses to “Restoring Trust: Can the Regent/SED, Unions and Superintendents Agree on a “Valid. Reliable and Fair” Teacher Evaluation System?

  1. Marc Korashan

    Buy-in and participation are key concepts for trust and to foster change, but real buy-in and participation require scrapping the current model which values test scores too highly. The tessts themselves are highly flawed measures of student ability. reading tests tell us only if a student can read (within the limited parameters of the restricted answers on the test) and not the more important question whether s/he does read and what.
    A real teacher effectiveness system will begin with the Danielson or some other rubric that defines good teaching practices, modify and clarify that language, in teacher led discussions about what goes on in classrooms and what should go on. A real teacher evaluation system will build in time for peer-to-peer non-evaluative observations and discussions. It will emphasize lesson study and professional development led by teachers on how to define and meet the needs of the students in that school.
    A real evaluation system will spend time defining the contents of and developing rubrics for evaluating student work and student portfolios, either best work, or I think more useful, growth rubrics. This allows us to see students using the skills they learned in applications they may find meaningful and encourage project-based learning, longer units of study (depth over breadth a la the CCLS).
    I know these are not numbers or easily quantified measures but they are more meaningful, can be more reliable than test based measures, and will involve teachers as professionals in defining good practice and moving the profession forward.

    Like

  2. ken karcinell

    In the final analysis, an objective solution to teacher evaluation is virtually impossible to achieve. The teacher’s ability to succeed is a case of subjective variables. Every public school in America is goverened by a Principal who may or may not be a so called “good principal”. We know that there are many instances of failed teachers from one school, who are found thriving in another. The only objectivity to our profession is the profoundly overstated view that teachers teach and children learn. After saying that ,the question of how , becomes muddled, distorted, and to a lesser extent criminalized, especially when The Mayor of NY declares that suspensions are a thing of the past! Successful teaching cannot be achieved witout classroom management. My credo has always been, if you’re not here to learn, I will put you out. If you’re not here to teach, you have to get out. After that my teacher evaluation was an outgrowth of my PDs and Formal Observation visits.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s