Category Archives: tenure

If VAM is Dead, What Comes Next? How Should Teacher Performance Be Assessed? The Governor/Legislature/Regents Try to Escape a Self-Constructed Abyss

If you toss a rock into a pool of feces you never know whose going get splashed.

Right now the governor and members of the legislature might not smell too good.

Parents are angry and disillusioned; teachers have no confidence in the governor, the education landscape is in disarray.

The Cuomo-appointed Task Force, which includes legislative leaders, is scrambling to find a solution at the edge of Dante’s Inferno. And, perhaps glancing at the inscription,”Lasciate ogne speranza, voi ch’intrate” (most frequently translated as “Abandon all hope, ye who enter here”).

How did our leaders create this yawning abyss? How did they anger a quarter of a million parents?

Foolishly, they allowed themselves to be lured down the (de) former path.

The (de)form side – the “disruptive innovation” agents abjure the normal methods of change, i. e., research, pilot programs, building constituency, electing like-minded candidates. “Disrupters” bring about change by disrupting the existing situation,

disruptive innovation is an innovation that helps create a new market and value network, and eventually goes on to disrupt an existing market and value network.

Abolishing tenure, weakening or eliminating teacher unions, sharply raising standards (Common Core) and imposing a far more difficult examination, disruption theory in practice; by creating chaos the (de)formers hope to skip over the usual processes, to be precise, to blow up and sweep away the current system and build a new education system.

The New Teacher Project (TNTP), a reform-y think tank issued a report, The Widget Effect ,”…our school systems treat all teachers as interchangeable parts, not professionals. Excellence goes unrecognized and poor performance goes unaddressed. This indifference to performance disrespects teachers and gambles with students’ lives.”

The report supports a new evaluation system that will identify and fire “poor performance” as well as recognize excellence, aka, merit pay.

Former Commissioner King and the governor led the charge for the Race to the Top dollars that required a student performance-based teacher evaluation assessment.  Not enough teachers found ineffective, change the plan through legerdemain, using the budget process to change the education law.

To quote poet Robert Byrnes,

The best laid schemes o’ Mice an’ Men

          Gang aft agley,

An’ lea’e us nought but grief an’ pain,

          For promis’d joy!

The “best laid schemes” turned into a nightmare.

As the Task Force scrambles we might ask a fair question: How should teachers be assessed/evaluated/rated?

The current law uses two indicators, a growth model score, referred to as Value-Added Modeling (VAM), a dense mathematical algorithm that compares teachers to teachers across the state who are teaching “similar students,” (Title 1 eligible, ESL, SWD, class size) and, the traditional supervisory lesson observation.

The highly regarded American Education Research Association (AERA) recently issued a policy paper. “AERA Statement on Use of Value-Added Models (VAM) for the Evaluation of Educators and Educator Preparation Programs ” warning schools and school districts about the use of VAM,

There are considerable risks of misclassification and misinterpretation in the use of VAM to inform these evaluations … the education research community emphasizes that the use of VAM in any evaluations must satisfy technical requirements of accuracy, reliability, and validity. This includes attention not only to the construct validity and reliability of student assessments, but also to the reliability of the results of educator and program evaluation models, as well as their consequential validity. In sum, states and districts should apply relevant research and professional standards that relate to testing, personnel, and program evaluation before embarking on the implementation of VAM.

So, if VAM assessments are flawed let’s go back to classroom lesson observations; however, classroom observations are also flawed ,”...teacher performance, based on classroom observation, is significantly influenced by the context in which teachers work. In particular, students’ prior year (i.e., incoming) achievement is positively related to a teacher’s measured performance captured by the [Danielson Frameworks]” 

In other words, teachers with higher achieving kids get higher observation scores. Another study of math teachers found “…math teachers with the highest-achieving students were nearly seven times more likely to get the top observation rating than teachers with the lowest-achieving students.”

A 2014 Brookings Report  explored lesson observations and growth models and reported,

* Under current teacher evaluation systems, it is hard for a teacher who doesn’t have top students to get a top rating. Teachers with students with higher incoming achievement levels receive classroom observation scores that are higher on average than those received by teachers whose incoming students are at lower achievement levels, and districts do not have processes in place to address this bias.

* The reliability of both value-added measures and demographic-adjusted teacher evaluation scores is dependent on sample size, such that these measures will be less reliable and valid when calculated in small districts than in large districts.

* Observations conducted by outside observers are more valid than observations conducted by school administrators

* The inclusion of a school value-added component in teachers’ evaluation scores negatively impacts good teachers in bad schools and positively impacts bad teachers in good schools

We also know that inter-school/inter-district observation reliability is low, in spite of the use of the same observation rubric in a school district; supervisory observers see lessons differently.

Value-Added Modeling (VAM) has been the subject of widespread criticism, although the scoring is free of bias, supervisory observations are heavily dependent on the academic level of the students and are subject to observer bias. Ironically the large errors of measurement in VAM, plus or minus ten or fifteen percent, benefits the teacher.  An example: the cut score between effective and ineffective is a VAM score of 50 and the teacher receives a score of 45, with an error of measurement of plus or minus 10%, the error of measurement impacted teacher score falls between 35 and 55 – although the teacher score fell in the ineffective range the teacher receives an effective score. The statistical unreliability benefits the teacher.

Maybe it’s time to borrow from the (de) former playbook and “disrupt,” in other ways, press the restart button.

How about a moratorium, let’s put the emphasis on teacher evaluation on hold for a few years and take a look at teacher evaluation across the country and in the OECD nations . How do the highest achieving nations assess teacher performance? The OECD website includes a lengthy section on teacher appraisal – check out a study from the Netherlands .

Think the nations with the highest student scores on international tests (PISA) know something about assessing teacher performance?

We want a system that is fair, transparent and supported by teachers and parents, we want a system that not only assesses but also builds teacher competency; and, while we’re taking a deep dive into teacher assessment let’s also take a look into student assessment.

Mixed Martial Arts in Albany: Cuomo versus Heastie versus Skelos

The mixed martial arts bills are progressing through the state legislature, the actual mixed martial arts, the combat in the octagon, the real blood sport is in full bloom. The featured bout: Cuomo v Heastie v Skelos.

The 150 members of the Assembly and the 63 members of the Senate gather in Albany the first week in January, some are deeply involved in introducing bills, other spend their time on constituent services and some work on their outside employment. Over the next term, the 2015 and 2016 sessions, over 15,000 bills will be introduced into the Assembly, about 500 will become laws, less than five percent of the bills introduced.

Members from Manhattan may file hundreds upon hundreds of bills, members from the inner city fifty bills, and chairs of major committees may file hardly any bills. Introducing a bill is a long way from passage, bills require democratic sponsorship in the Assembly, republican sponsorship in the Senate and gubernatorial support for final passage, a long, long road.

From January until the end of March the legislative leadership is consumed with the budget and the leadership has to gauge the temperature of their caucus, called the “conference.” Before or after a floor session members will meet in conference,’ a closed meeting, members and top staff only, no votes are taken, no minutes, the members can speak freely. How “tough” are the members? Do they want to risk going beyond April 1 without a budget? Do they want to “take on” the governor directly? Do they want to risk antagonizing core constituents? Who are the members more afraid of: the governor, their constituents? Or, the speaker?

Sheldon Silver ruled with an iron fist, he probably kept a copy of The Prince at his bedside, and one of his favorite quotes might have been,

“It is much safer to be feared than loved because …love is preserved by the link of obligation which, owing to the baseness of men, is broken at every opportunity for their advantage; but fear preserves you by a dread of punishment which never fails.”

Why am I quoting Machiavelli? Was he a totalitarian or a human rights respecting republican (Read Phillip Bobbitt, The Garments of the Court and Palace: Machiavelli and the World That He Made (2013)).

Heastie, the speaker, needs the total support of his members, the 105 democratic members of the Assembly, the conference. He has to create a team, a united group who supports the speaker without reservations, a team who knows they cannot back away, that unless they stand up to the governor he will roll over them. You gain loyalty by acts, by making decisions that support your members.

Those of you who have played sports or played in an orchestra or danced in a company understand leadership, under the synergy created by teamwork, the sum is greater than the parts in a synergistic organism.

Five Regents are seeking re-appointment and there are two vacancies. In the Westchester-Rockland judicial district the speaker clearly approved of the Assembly democrats making the selection. Open interviews were conducted in Westchester, about a dozen applicants. Apparently the legislature will select Judith Johnson, a retired superintendent who is highly regarded across the counties, who was the choice a majority of the legislators.

Robert Bennett was the Regents from the Buffalo area, Bennett served as the chancellor prior to Merryl Tisch, served on the Board for twenty years, and his bio on the SED website recounts a long and illustrious career. Recently Bennett has begun to antagonize more and more sectors within the community, supporting charter schools, supporting the Common Core, supporting testing and, mostly, unconditionally supporting former Commissioner King. At the Albany interviews Assembly member Ryan skewered Bennett. Bennett proudly announced he was heading a task force to review special ed regulations, Ryan asked Bennett to what extent he was responsible for the failures of the last decade, and Bennett stumbled.

The Sunday Buffalo News reports that Regent Bennett has withdrawn and will support Catherine Fisher Collins,

Dr. Collins is a Pediatric Nurse Practitioner and the first African American nurse practitioner to graduate from the University of Buffalo’s School of Nursing Nurse Practitioner program. In addition she holds three certifications in health education.

The new speaker understands power, he defers to his members to fill a vacancy and is willing to dump a twenty year incumbent to acknowledge bubbling anger among voters.

Heastie is building a team, a team willing to follow their leader wherever he chooses to go, to the edge of the cliff, and, if necessary, over the edge.

Cuomo will bully, threaten, and try to undercut the speaker; veiled threats, not so veiled threats, waiting for the speaker to take whatever is on the table at the eleventh hour.

Senate majority leader Skelos has his own list, how much does he cede to Cuomo, and, can he partner with Heastie against Cuomo?

Speaker Heastie is the most powerful Black elected official in New York State, and, in time, potentially, one of the most powerful in the nation. Standing up to an incumbent governor only increases creds, and standing up to an incumbent governor and losing reduces his image.

Cuomo wants to be standing on the podium early on the morning of April 1st announcing the fifth straight on time budget, how can he reach an agreement without appearing to lose face? Can he “win the battle and lose the war,” by defeating Heastie and alienate Black and liberal voters?

In the Cuomo camp some advisors are probably telling him to follow the Scott Walker path, attack public employee unions unrelentingly, after all, it may be a path to the presidency. Other advisors will remind Cuomo, he’s running as a democrat, not a tea party republican.

I don’t know how Cuomo, Heastie and Skelos get to that April 1st stage, I don’t know the deals, the trade-offs, I don’t know how the questions of teacher evaluation, testing, tenure and “receiverships” will be resolved, for the three men in a room, the endgame, how the public views the “winners” and “losers” will drive the decisions.

“All courses of action are risky, so prudence is not in avoiding danger, but calculating risk and acting decisively. Make mistakes of ambition and not mistakes of sloth. Develop the strength to do bold things, not the strength to suffer.”
― Niccolò Machiavelli