Tag Archives: APPR

“All Politics is Local,” The Saga of the Annual Professional Performance Review (APPR), Assessing Teacher Performance and the Underside of Law-Making in Albany

Mike Schmoker, the author of Focus, wrote a prescient article in Education Week, “Why I’m Against Innovation in Education” at the same time that Bill Gates and Mark Zuckerberg announced yet another richly funded innovation,  “The State of the Art Ideas for Schools,” Schmoker writes,

I’m against innovation in education—as currently conceived and conducted. I’m not against small-scale educational experimentation, where new methods are tested, refined, and proved before they are widely implemented. But I’m against our inordinate obsession with what’s new at the expense of what works—with exceedingly superior (if much older) evidence-based practices

 “Inordinate obsession” is the appropriate term; education policy has been driven by billionaires, economists, statisticians and psychometricians, experts on one field setting policy in another field. A prime example is the work of Raj Chetty, who uses “big data,” statistical tools to analyze huge datasets. Chetty and others, in “The Long-Term Impacts of Teachers: Teacher Value-Added and Student Outcomes in Adulthood” concludes that high Value Added teachers have a substantial positive impact on students into adulthood; however, warns that VA should not be used for teacher evaluation

… our study shows that great teachers create great value and that test score impacts are helpful in identifying such teachers. However, more work is needed to determine the best way to use VA for policy. For example, using VA in teacher evaluations could induce counterproductive responses that make VA a poorer measure of teacher quality, such as teaching to the test or cheating. There will be much to learn about these issues from school districts that start using VA to evaluate teachers.

 In spite of Chetty’s caveat states across the nation hopped on the “assess teachers through student test results” bandwagon. The NYS Board of Regents held a summit, experts from across the nation, expressing opinions on the use of student data as assessment tools. The experts warned that the use of Value-Added was ill-advised; teachers teach different kids each year, as well as different grades and different subjects, the errors of measurement, plus/minus ten, twenty, thirty percent makes the data useless. Another vampire idea, a refuted idea that refuses to die; popping up again and again.

The New York State Race to the Top application, in exchange for $700 million, included a multiple measures plan, teachers rated by a combination of supervisory observations and student test scores. Without going too far into the weeds, the current system, called a matrix, combines supervisory observations and student learning objectives (SLO) also referred to as measurements of student learning (MOSL).

Three years ago the governor agreed to a four year moratorium on the use of student test scores and this year the commissioner was in the early phases of constructing an alternative plan. The commissioner has used a consultative process, task forces or work groups, the names are interchangeable, to propose changes in state policies.

Apparently the state teacher union (NYSUT) had been working with the Assembly leadership to craft a plan, a bill was introduced the day before the state teacher union convention and passed, with only one negative vote a few days later. The commissioner was clearly stunned, and not happy. While changing the law is the responsibility of the legislature, the commissioner is the leader, the CEO of the state education establishment amd would expect to be part of the bill drafting process.

A summary of the changes and the opinions of the stakeholders, read here .

The bill passed in the Assembly was introduced as the Senate, a “same as” bill, and, sponsored by the Senate education chair; however, not so fast. Chalkbeat, the online education website muses over the future of the bill.

Senate Majority leader John Flanagan has a dilemma, the bill is popular with parents as well as teachers, the Long Island Opt Out Facebook page proudly boasts 25,000 members, a number of them in Republican senatorial districts. Flanagan needs cover for his Republican colleagues, and his presser speculates over whether the bill will increase the number of tests, clearly an appeal to Opt Out parents.

Since it was first introduced, the State Education Department, the New York State School Boards Association, and the New York Council of School Superintendents have raised concerns that the legislation as written could inadvertently open the door to even more testing than we have now.  Nobody – not students, not parents, not teachers, nor myself or my legislative colleagues – wants that outcome.  With this in mind, we are performing an extensive review of this legislation to determine the best path forward. 

The School Boards Association also questions the bill, a bill that requires negotiations with the collective bargaining agents

We are concerned that if enacted, proposed APPR legislation that has passed the Assembly would result in additional student testing. 

Unless the state wants to forfeit federal ESSA funds, it still must administer grades 3-8 ELA and math state assessments. Under the proposed APPR legislation, students could have to take both the state tests as well as alternative assessments that would be used for teacher and principal evaluation purposes.

In addition, we have serious concerns about the requirement in the legislation for school districts to negotiate the selection of alternative assessments through collective bargaining. This represents a step backward, as school districts presently have the authority to determine assessments used in teacher evaluations.

School boards would rather see unions disappear than work with them in a collaborative manner.

The leader of Long Island Opt Out sees the proposed law as a “small step,” and is agnostic.

Jeanette Brunelle Deutermann

Admin · May 2 at 6:31pm

I want to be clear on what went down/is going down with the new APPR legislation. First and foremost, this legislation does absolutely nothing for children. Not that all legislation has to be centered around children, but I just want to ensure that if you hear ANY INDIVIDUAL or ORGANIZATION proclaim that it is, they are lying. All this does is take away the REQUIREMENT for districts to use 3-8 scores in evaluations, the way it used to be before the moratorium. After this law is passed, districts will continue to be forced to use test scores as 50% of their evaluation, but now in addition to local computer assessments, the science test, or regents tests being a choice, now 3-8 assessments are back on the list. A minuscule positive detail – they don’t HAVE to use 3-8 tests (as they would have had to use as the moratorium comes to an end). Those involved in creating this bill are celebrating this as a huge win. A more appropriate response would be “very sad that this is all our elected officials could muster.” Some have said “but it’s a step.” I guess that all depends on what shoes you’re wearing.

 The legislature will plod along, adjournment around the end of the third week in June and won’t return, except for an unusual special session, until January, 2019; the governor has until the end of the year to sign bills that pass both houses.

Will the Republican leader bring the bill to the floor for a vote? Will the governor sign the bill?

Flanagan has a conundrum,

  •  Should he try and extract a quid prop quo from the Democrats in the Assembly – signing the bill in exchange for, let’s say, raising the charter school cap in New York City, or, approving pre-K classes in the Success Academy charter schools? Actions probably resulting in dollars from the charter school political action committees.
  • Should Flanagan, sub rosa, try and get NYSUT, the teacher union, not to vigorously oppose fellow Republicans in the November general election? Unlikely
  • Should he simply “say no,” it’s a bad bill, without changes palatable to the commissioner and the school board association? In other words use the commissioner and the school board association as cover.

Or, oppose the bill and try and trash the teacher union as being self-serving.

And, will the governor sign the bill without an endorsement from NYSUT?  The small number of union locals that endorsed Zephyr Teachout four years ago might decide to endorse Cynthia Nixon this time, clearly engendering the animosity of the governor.

The end of the legislative session is called “the big ugly” for a reason.

Oddly, this is not an issue in New York City. The City and the Union agreed upon a system of using SLOs and MOSLs and a sophisticated set of alghorisms that satisfies their needs. Under the last year of Bloomberg 2.7% of teachers received unsatisfactory ratings, under the current matrix, less than 1% of teachers received ineffective ratings. The question of the number of observations will be part of the upcoming collective bargaining negotiations.

Maybe a sentence that should be posted above the Albany legislative chambers:

“No mans life, liberty or property are safe while the Legislature is in session.”

Gideon J. Tucker, NYS Surrogate, 1866

Getting It Right: Building a Research-Based Teacher Assessment System

A couple of years ago I was participating in a Danielson Training Workshop, two Saturdays in a room filled with principals and network support folk. We watched a video of part of a lesson – we were told we were watching a first year teacher in November in a high school classroom.

Under the former Satisfactory/Unsatisfactory rating system the lesson was clearly satisfactory. The Danielson Frameworks (Read the 115-page NYSED document here) requires that teachers are rated on a four-point scale (Distinguished, Proficient, Basic and Unsatisfactory) while New York State also requires a four point scale (Highly Effective, Effective, Developing and Ineffective). The Frameworks divides the teaching process into four domains, 22 components and 76 elements.

The instructor asked us to rate the lesson: at my table we were all over the place. For a teacher in the third month of her first year of teaching the lesson was excellent – clearly “proficient.”  Others argued the time in teaching was irrelevant, you had to rate her against all other teachers regardless of experience – at best, she was “developing.” Inter-rater reliability was absent.

Decades ago the union sent me to an Educational Testing Service conference on teacher assessment; about thirty experienced superintendents from all over the Northeast, and me, one union guy. We began by watching three 15-minutes videos of lessons: one an “old-fashioned” classroom, the kids sitting in rows, the kids answered teacher questions, the kids stood when they answered; the questions were at a high level although a small number of kids dominated the discussion. In the other video kids were sitting at tables, the teacher asked a question, gave the kids a few minutes to “huddle,” and one of the kids answered for the group and the teacher followed up with a few clarifying questions, in the third classroom the kids were at stations around the room, it was noisy, the noise was the kids discussing the assignment, the teacher flitted around the room, answering, clarifying and asking questions.

We were asked to rate the lesson on a provided checklist.

The result: the superintendent ratings were all over the place.

I was serving as the teacher union rep on a Schools Under Registration Review (SURR) team – we were visiting a low performing school. We were told to wait, the principal was busy, four of the 50 teachers were absent and there were three vacancies, the principal was assigning classroom coverages.

At the initial get acquainted session a team member, considering the staffing issues asked, “What are the primary qualities you look for in assessing teacher quality?” The principal blurted, “They come every day and blood doesn’t run out from under the door.”

A colleague was touring a school with very high test scores.  As he walked the building with the principal, he saw uniformly “mediocre” instruction – teacher-dominated, no student engagement. He mentioned the low quality of instruction to the principal, who shrugged, “Why mess with success?”

Once again, there is no inter-rater reliability.

In a number of school districts across the state almost all teachers received maximum observation ratings.

The State Ed folk simply accept the observation ratings of principals and school districts.

Charlotte Danielson, in her other book, Talk About Teaching  (September, 2015), discusses the complex role of the principal as rater as well as staff developer: how can a principal, who is the summative evaluater honestly engage with teachers who they rate?

In an excellent article from the Center for Educator Compensation Reform, Measuring and Promoting Inter-Rater Agreement of Teacher and Principal Performance Ratings (February, 2012), the authors parse the reliability of teacher observation ratings. There are a number of statistical tools to assess reliability – the state uses none of them.

In New York State 60% of a teacher rating is made up of the teacher observation score, and, we have no idea of the accuracy of the rating.

In the pre-Race to the Top days, the Satisfactory/Unsatisfactory rating days, the entire rating was dependent on the observation – in the last year of Bloomberg term 2.7% of teachers in New York City received Unsatisfactory ratings, under the current far more complex system that incorporates student tests scores and other measures of student growth only 1% of teachers were rated ineffective (Read a description of the plan:  APPR 3012-c).

Under the newest  system the other 40% is a combination of Measures of Student Learning and Student Learning Objectives, the use of state test scores is suspended until the 2019-20 school year.

Read a detailed description of the current APPR 3012-d teacher evaluation law here and a lengthy Power Point here.

In May, 2015 the Regents convened a Learning Summit and asked a number of experts to discuss the use of student growth scores (VAM): Watch the lengthy, sometime contentious discussion  here.

With one exception the experts criticized the use of student growth scores (VAM), the VAM scores did not meet the tests of “validity,” “reliability” and “stability.”

There have been glaring errors in the system. In the Sheri Lederman law suit  a teacher had very high observation scores and due to the composition of her class, very low student growth scores. The judge ruled the use of the growth scores, in the individual case, was “arbitrary and capricious.”

The APPR plan negotiated in New York City, on the other hand, allows for appeals by a neutral third party, and, the “neutral” has overturned appeals in which there was a wide disparity between the observation and VAM scores.

The current plan, created by the governor and approved by the legislature has been rejected by teachers and parents. Teachers are convinced that their score is dependent on the ability of the students they teach, not their competence. Parents feel schools are forced to “teach to the test” due to the consequences facing principals and teachers.

Angry parents, angry teachers and principals and a governor and a legislature looking for a way out of the box they created.

And, a cynicism from elements among the public – if two-thirds of kids are “failing” state tests how is it possible that only one percent of principals and teachers are rated “ineffective?”

The Board of Regents has been tasked with finding the “right” plan.

There has been surprisingly little research and public discussion of teacher attrition – in high poverty schools staggering percentages of teachers, 30%, 40%, 50% or more leave within their first few years.

The December, 2015, Cuomo Commission Task Force, in a scathing report, tasked the Regents with “correcting” what has been a disastrous path. Partially the governor creating an incredibly complex teacher evaluation matrix and partially the Commissioner King rushing to adopt the common core, common core testing and teacher evaluation simultaneously.

Can the Regents separate political decisions from research-based and guided decisions? Can the Regents move from the John King path, an emotion-guided political path to actually following “what the research says”?

On Tuesday the new Research Work Group, chaired by Regent Johnson will convene for the first time.

The roadmap for the State Ed Department and the Board of Regents are the twenty-one recommendations of the Cuomo Common Core Task Force. A number of the recommendations: untimed testing, an in-depth review from the field of the standards, greater transparency of the test items, alternatives to the use of examinations for students with disabilities, and, the beginning of an review of teacher evaluation are already in progress.

The Commissioner and the Regents have to regain a lost credibility: from policy emanating from the Gates Foundation and the so-called reformers to policies guided by scholarship and supported by parents and educators.

Killing the Zombies: Why the “Bad Teacher” Canard Refuses to Die

Who is Clay Christensen and what is disruptive innovation in education?

Christensen is a professor in the Harvard Business School and the intellectual force behind the current education reform movement. The professor proffers that education has been basically unchanged for decades, a traditional classroom model, very little has changed including little improvement in achievement. Christensen acolytes argue that the traditional model must be “disrupted.”  A wide range of examples: placing schools in competition; public, private, charter, parochial and home-schooling through a voucher system. Traditional instruction must be replaced by an iteration of personalized learning in the form of computer-based learning and, impediments to removing “bad” teacher removed.

The “disruptors’ include the political leadership, from the White House to state capitals.  The $4.4 billion in competitive state grants, the Race to the Top (RttT), is a prime example. The lure of federal dollars to disrupt the traditional systems; RttT required the creation and expansion of charter schools as well as creating a student test score-based teacher evaluation system.

The New Teacher Project (TNTP), an advocacy organization, a “disrupter” organization, conducted a survey of school districts and a report – The Widget Effect. The findings:

Effective teachers are the key to student success, yet our school systems treat all teachers as interchangeable parts, not professionals. Excellence goes unrecognized and poor performance goes unaddressed. This indifference to performance disrespects teachers and gambles with students’ lives.

The 2009 report, surveyed fifteen schools districts across four states points to the absence formalized evaluation systems resulting in virtually all teachers rated satisfactory with few classroom observations.  For the TNTP there was no sorting of teachers by ability, no one is fired and no one is identified as being an exemplary teacher.

In districts that use binary evaluation ratings (generally “satisfactory” or “unsatisfactory”), more than 99% of teachers receive the satisfactory rating, Districts that use a broader range of rating options do little better; in their districts, 94% of teachers receive one of the top two ratings and less than 1% are rated unsatisfactory.

Since the release of the report the reformers, the “disruptors,’ have been successful, enormously successful, in convincing, coercing, luring states into highly structured teacher evaluation systems; to identify the high performers (merit pay) and prune away the low performers.

As part of their winning Race to the Top proposal New York State designed a multiple measures teacher evaluation system: 60% of a teacher score would be supervisory observations based on a rubric selected by the school district (Danielson, Marzano, Marshall and others), 20% based on a student growth data (VAM) on state grades 3-8 test scores and 20% on a locally negotiated metric – which could be test scores or other measures of student learning (MOSL). The data is pumped into a dense, extremely dense mathematical algorithm and all teachers receive a score that translates in a letter grade on the HEDI spectrum: Highly Effective, Effective, Developing and Ineffective. The teacher evaluation plan, called the Annual Professional Performance Review (APPR) has been amended a number of times – the current plan prohibits the use of student test scores for four years, is called the “matrix.”

The inclusion of a value-added measurement, the student test score algorithm has been sharply criticized by a range of scholars as well as teacher organizations, and, a state court, in a non-precedent setting decision, found the use “arbitrary and capricious.”

Millions of dollars to create a multiple measures teacher evaluation plan and the result: 1% of teachers are ineffective – the same as the Widget Effect report.

In 2009 The New Teacher Project bemoaned that only 1% of teachers were rated “unsatisfactory” and seven years later the New York State APPR plan found, you guessed it: once again, only 1%.of teachers rated “ineffective.”

Millions of dollars to create a teacher evaluation system, a host of “experts,” the application of dense mathematical formulations and the percentage of teachers rated ineffective is unchanged.

This couldn’t possibly be right!! … so say the disruptors.

In a recently released report, The Widget Effect Revisited: Teacher Evaluation Reforms and the Distribution of Teacher Effectiveness  (February, 2016), the authors conducted surveys of  newly designed teacher evaluation plans across a number of states and interviewed principals.

On the new plans, “… less than 3% of teachers were rated below Proficient”

The raters, the principals, also reported,

“…evaluators perceive more than three times as many teachers in their school as below Proficient than they rate as such.”

In lengthy interviews the principals expressed the reasons for not rating more teachers as below Proficient.

* Time constraints (“It takes too much time away from running a school”)

* Teacher potential and motivation (“Fear of the discouraging of teachers”)

* Personal discomfort (“I have a difficult time telling teachers they’re failing”)

* Racial tensions (“Very difficult for a White principal to rate Black teachers poorly”)

* Quality of replacements (“I can’t find adequate replacements”)

* Voluntary departures (“I rate them Proficient and they leave – a deal is made”

* Burdensome dismissal processes (“The process is too complex and time consuming”)

Is the “problem” too many “below Proficient” teachers or “below Proficient” principals?

What the authors failed to investigate, admittedly not the purpose of the study,

* Inter-rater reliability: do the raters from school to school use the same rubrics, and, are they competent to assess teacher performance?

* The bell-curve conundrum: is the lowest rated teacher in the school “below Proficient” when compared with all other teachers in the district?  In other words, is it “arbitrary and capacious” to establish a system that guarantees that the lowest performer in a school must be below Proficient? After all, they may be more proficient than teachers in other schools in the district.

A second baseman on a major league team may be the “least proficient” among major leaguers and in the top 1% of all second baseman across colleges and minor leagues.

I know the idea is disquieting – perhaps only 1% of teachers actually are ineffective.

Prospective teachers must be accepted by a college and meet standards set by the Council on Accreditation of Teacher Preparation (CAEP), teachers must pass a number of nationally recognized pre-service exams, pass interviews by principals/hiring committees, teach a demonstration lesson and serve a probationary period as an at-will employee.  It should not be surprising that very few teachers who survive the rigorous pre-screening end up as “below Proficient.”

Nobel Laureate Paul Krugman coined the term zombie idea: a zombie idea is “a proposition that has been thoroughly refuted by analysis and evidence, and should be dead — but won’t stay dead because it serves a political purpose, appeals to prejudices, or both.”

The disruptor “bad teacher” solution to increasing student achievement is an example of a zombie idea – in spite of reams of evidence the idea refuses to die.

What is so depressing is when compared to teacher attrition the “bad” teacher argument pales – in the lowest achieving, highest poverty schools about half of all teachers leave within five years, and, we have a pretty good idea of why they leave: the way they’re treated.

Susan Moore Johnson, at the Next Generation of Teachers Project at Harvard examines the issue of teacher attrition in the highest poverty schools in detail.  Yes, teachers commonly leave to wealthier, whiter schools; however, they are not fleeing the students, they are fleeing the working conditions.

If we know how to make significant differences (“Improve working conditions in the poorest school”), if we’ve identified the core problem (“Teacher morale and treatment”), why don’t we address the solution?

Those zombies are tough to kill off.

The Regents/Commissioner Agenda: Grappling with the Future of Education in New York State

The Board of Regents meets monthly beginning on a Monday morning with a webcast full board meeting in the historic Regents Room, lined with portraits of former chancellors dating into the 19th century.  The February 24th meeting began with detailed description of the new federal law, the reauthorization of the Elementary and Secondary Education Act now called the Every Student Succeeds Act (ESSA).  (Read the federal description here and the State Ed Power Point explanation here  and watch the archived webcast here which included many questions and many unresolved issues).

The Board moves to a series of committee meetings with presentations by the commissioner and her staff, education leaders from around the state and other invited guests. (The committee meetings are not webcast)The presentations sometimes relate to a topic for discussion that will evolve into policy and at other times an informational topic. Policy issues move to votes, a period for public comment, back to the committee and, if approved by the committee on to the full board for a vote.

On Monday the P-12 Committee spent the morning listening to a lengthy presentation, “Revision and Implementation of the New ELA and Mathematics Standards,” (Watch Power Point here) by the commissioner. The survey posted by State Ed garnered thousands of comments about the Common Core and pointed to a number of specific areas that practitioners thought required revisions. The state will convene stakeholder meetings, just about every constituency imaginable (including Content Advisory Panels and Standards Review Committees), and move towards a revision of the Common Core. At the end of the process, about three years, the next generation of state tests will reflect the revisions; a caveat, someone must provide the dollars to fund the process: BTW, the Common Core will be renamed Aim High NY.

Make sure you click on and watch the Power Point above; the areas for suggested revision are specific.

The P-12 Committee moved into a discussion of Academic Intervention Services/AIS. School districts are required to identify and provide targeted services to a cohort of students; the state determines the targeted students.  Should the targeted group be defined as all students who score below proficient on state tests (below 3.0)?  Or, all students who score below 2.5?  Should school districts be permitted to use multiple measures to identify AIS students?  Two educators, one from upstate and one from New York City described the local processes – a lively discussion ensued, especially among the Regents who were former superintendents, (Watch Power Point here). The commissioner will return to the board with a specific recommendation that will move through the comment period to adoption of recommendations binding on school districts. The sharp decline in scores under the Common Core tests doubled the pool of AIS eligible students potentially sharply increasing the cost to the district and/or reducing the student per capita spending.

In the next session the Regents received an update on creating formal policy regarding increasing the numbers of students in “least restrictive environments” that will be forthcoming in the fall.  Some school district prefer to place students with disabilities in self-contained classrooms, others prefer to place students in integrative, co-teaching classes. The discussions, as they emerge later in the year, should be lively.

The approval of a number of charter schools, usually pro-forma resulted in animated discussion around the New York Times video of a “model teacher” berating a third grade child who could not answer a question. Success Academy charter schools are authorized by SUNY, not the Board of Regents; do the Regents have any authority to investigate the pedagogy in schools that they do not authorize?  Clearly, the Regents were quite upset and four Regents refused to vote on the approval of the charter schools on the agenda.

The NYS Permanent Judicial Commission on Justice for Children presented a policy paper (Read here) and a Power Point, “Promoting School Justice Partnerships to Keep Youth in School and Out of Court” which began with the statements:

Myth: There is really no evidence that the “school to prison pipeline” really exist.

Bottom Line: Suspension often is the first step in a chain of events leading to short and long term consequences, i. e., incarceration.

Are the Regents and the Commissioner moving in the right direction? 

Correlation does not equal causation.

The discussion around AIS service and suspension are typical of “parachute” answers – solutions dropped in from the heavens.

Schools typically purchase an AIS program, we’re talking remediation, for example Read 180.  Schools monitor the implementation of the program, not the basic level of the instruction and the emotional needs of the student. Are the dollars and time spent on the AIS program more effective than the classroom instruction? Would the use of the funding and time be better spent in counseling and attending to the basic physical/emotional needs of the child?  And, the overall question: do AIS services improve student outcomes?

Are suspensions the first step leading to incarceration or are the basic behaviors of the child preceding the act that led to the suspension event the first step to criminal acts?

The core of education is the teacher and the curriculum.

There are no magic bullets; we aren’t hiding large numbers of wonderful teachers in some cave or some secret sauce that improves math skills.

We don’t do a good job of recruiting prospective teachers, we could prepare teachers a lot better and we can certainly support new teacher much better. Teaching is the only profession with such high attrition rates.

We know that trauma has adverse impacts on children, the research is overwhelming (Read some of the research findings here), yet the powers above pour dollars into remediation rather than the health and social services that address the underlying reasons for difficulties in school.

Yes, the teacher is the core, the building block of our entire education system. The fatally flawed teacher evaluation system (APPR) neither measured teacher effectiveness nor discriminated among teachers.

Getting better as a teacher is a career-long trek.

For example, a recent research is troubling, and hopefully will result in teacher introspection.

New research shows that black and white teachers give very different evaluations of behavior of black students. When a black student has a black teacher that teacher is much, much less likely to see behavioral problems than when the same black student has a white teacher.

New research by Adam Wright, “Teachers’ Perceptions of Students’ Disruptive Behavior: The Effect of Racial Congruence and Consequences for School Suspension,” documents that black teachers have much less negative views of black student behavior than do white teachers.

Are white teachers less able to “relate” to students of color?  Can teaching be described by the Danielson Frameworks or is culturally responsive pedagogy essential to be an effective teacher?

Looking for that magical fairy dust that can be sprinkled over the students who are not progressing in literacy and numeracy is a chimera. The better question is why that school a few blocks away with the same kids is doing so much better?  There are high and low suspension schools; once again, why?  I don’t object to restorative justice practices, they are time consuming and can be expensive. Collaborative and demanding school leaders, a team approach, schools in which teachers, together, strategize about kids, are more likely to achieve better academic outcomes and fewer suspensions.

The answers are blowing in the wind, we have to catch them.

The Board of Regents Convene With a Contentious Agenda and the Ominous Shadow of the Governor

Wednesday morning the seventeen members of the Board of Regents and the newly selected commissioner will convene in the ornate Regents Room to begin the 15-16 school year. Oddly the agenda, to a large extent, has been set “across the street,” on the second floor of the Capital building, the executive offices of the governor.

Education policy for two centuries was set by the members of the regents with significant input from the commissioner. Commissioners worked their way up the ladder, from teacher to principal to superintendent to commissioner; all that changed in the last few years. David Steiner came from the university and John King had no public school experience, in fact, only limited experience in the world of charter schools. The newly selected commissioner returns us to the world of experienced educators.

In the current convoluted landscape of education the governor has effectively replaced the regents: adoption of the Common Core State Standards, a massive labyrinthine principal/teacher evaluation system, the receivership of struggling schools have been set in legislation by the governor with the regents being asked to set regulations in place.

The unpaid, un-staffed members of the regents are “elected” by a joint meeting of the NYS legislature. In reality the democrats select the members; there are far more democrats than republicans in the combined houses. In the last session the legislature dumped two of the most senior members of the regents and selected four new members (three incumbents were re-elected, there were two vacancies and two regents replaced); three former school superintendents and one nurse educator (the State Education Department is in charge of all schools, pre-k through college, all museums and libraries and the professions).

The four new members and two second-term regents members have formed a caucus to oppose the approval of the governor’s new matrix principal/teacher evaluation plan (3012-d); the debate will be lively.

The regents will approve regulations for the completely untried receivership law; if low performing schools fail to make progress, as defined in the regulations, the school may be removed from the district and placed under the supervision of a receiver who has sweeping power. (See Regents agenda here).

Not only has the governor seized control of the education agenda the feds have been the agenda-setter for all of the states. The feds require that after being in the country for one year all English Language Learners in Grades 3-8 must be tested regardless of their English language skills. The feds denied the NYS waiver request and the regents and the commissioner are asking the feds to reconsider.

The regents are forming a working group to discuss the pass/fail rates on the new Common Core Regents exams; we are currently in year three of the eight year phase-in of Common Core Regents; the grades are currently scaled to keep pass-fail rates at the same level as before the Common Core: are students making adequate progress in passing the new Regents, and, if not, how should the regents members respond?

Regent Cashin is highlighting the new testing regimen for prospective teachers who are required to pass four exams at a cost of about $1,000; the exams are timed and computer-based: are the exams accurate predictors of success? Are the high failure rates the result of selecting the wrong candidates, faulty college curriculum or simply poorly crafted exams? In an era of sharply declining enrollments in college teacher education programs the poorly designed Pearson-created exams should not be an unnecessary impediment.

While the funding of schools is the responsibility of the governor and the legislature the 2% property tax cap is resulting in drastic cuts in services in low wealth districts, of which there are several hundred located in rural districts with declining revenues. The regents can highlight and recommend changes to the “other side of the street.”

How will the regents address the large numbers of Students with Disabilities who are unable to “pass” grades 3-8 tests and unable to achieve the safety net requirements on the Regents exams? Should the regents create alternative pathways to graduation? Portfolios?

In some schools English Language learners are making progress similar to all other students while in others the majority of students are graduating at extremely low rates: Why? Higher or lower levels of instruction? Better professional development? Better designed instructional models?

Educational decisions, as the state constitution intended, should be made by the Board of Regents. Hopefully the governor will move away from his senseless policies that have antagonized parents and teachers across the state.

Far reaching education policies crafted behind closed doors by invisible staffers is not a fruitful path to better education. The two hundred thousand op-outers will grow and grow; the angry electorate will continue to grow.

Hopefully the governor will rethink his ideas and the legislature will continue to select regent members willing to challenge the governor as well as collaboratively develop approaches to address the core issues confronting children and families across the state.

Who Sets Education Policy in New York State: The Governor or the Board of Regents? Should Education Policy Reflect the Views of the Governor or Parents, Teachers and Principals?

“We, the undersigned, have been empowered by the Constitution of the State of New York and appointed by the New York State Legislature to serve as the policy makers and guardians of educational goals for the residents of New York State. As Regents, we are obligated to determine the best contemporary approaches to meeting the educational needs of the state’s three million P-12 students as well as all students enrolled in our post-secondary schools and the entire community of participants who use and value our cultural institutions” (Comments by seven members of the Board of Regents)

The New York State Board of Regents is the oldest state education policy board in the nation, established in 1794 as part of the state constitution. The current 17-member board is “elected’ by a joint meeting of the State legislature. The selection as Regents was an honorific, a reward for long service in the state. Members served term after term in anonymity, met monthly from September to August, selected a commissioner and set policy, generally the policy originated with the commissioner. Typically the commissioner was a state superintendent with long and distinguished service.

All that changed with the election of Merryl Tisch to replace Bob Bennett as chancellor. The quiescent board became an activist board under the leadership of Tisch. An outsider, David Steiner was selected as commissioner, a report was commissioned that exposed the state testing program, and, with the resignation of Steiner and the appointment of John King the board pursued aggressive polices championed by the new commissioner. In line with the Arne Duncan game plan one major initiative after another was approved by the Regents, frequently over the objections of Regents Cashin and Rosa.

In order to qualify for the Race to the Top grant the state entered into a long negotiations with the state teacher and principal unions and an agreement was reached – called the Annual Personnel Performance Review, aka, APPR, teachers and principals would be assessed annually using a combination of student progress on standardized tests, locally negotiated measures of student achievement and supervisory observations.

In spite of trepidation the APPR plan was benign, with exceptions in a few districts, the overwhelming percentage of teachers received “highly effective” and “effective” scores and only 1% received ” an ineffective score.

The governor was displeased. Last spring the governor accepted millions in charter school campaign contributions and forced New York City to either co-locate charters in public schools or pay the rent for non-school space. Teacher anger grew, the teacher union made no endorsement for governor and it was clear that many teachers in the Democratic primary voted for Zephyr Teachout, an unknown law professor from Fordham. While Cuomo won handily it was not the landslide he anticipated, and, again, in the November general election Cuomo won, not by the landslide he had clearly expected.

On December 18th, barely six weeks after the election Jim Malatras, the governor’s chief of operations penned an accusatory letter to chancellor Tisch and commissioner King. The letter listed a series of condemning questions, demanding responses, and threatened to take over the policy-making role of the board.

As you know, Governor Cuomo has little power over education, which is governed by the Board of Regents, The Governor’s power is through the budget process and he intends the reforms during that process.

Read the full text of the Malatras letter here.

A few weeks later chancellor Tisch and acting commissioner Berlyn responded meekly; agreeing with the accusation of Cuomo through Malatras.

Differentiation is a necessary component of any evaluation system intended to
support professional development and growth. However, as the Governor has previously indicated, changes in State law are necessary in order to achieve better differentiation and to fulfill the goal of a Statewide evaluation system that identifies those who are excelling so that they can be mentors for their colleagues, identifies those who are struggling so they can get support to improve, and informs high‐quality professional development for all educators.

Read the full text of the Tisch-Berlyn response here.

During the last hours of the budget process the Governor did exactly what he threatened; he forced through a totally new teacher evaluation system based on a Massachusetts model known as the “Matrix.”

The board convened an “Education Learning Summit,” and an all-day series of speakers, experts and stakeholders, to express opinions. Three of the four experts were critical of the use of student growth data for high stakes teacher assessment.

At the May Regents meeting acting commissioner Ken Wagner outlined a 56-page proposal to implement the new teacher evaluation law, although, the reconstituted board was much less sanguine; the four newly appointed Regents and the original critics, expressed displeasure with the new proposals.

Seven members of the board, let’s use Diane Ravitch’s term and call them the dissidents, objected to the role of the governor and outlined an alternative plan (Read Ravitch’s blog post here).

Read the entire position paper at the end of the post.

On Monday the board convenes in Albany, under the current law the board must turn the law into regulations. The guidelines introduced by the “dissident” seven directly challenges the December 18th Malatras letter.

The law clearly gives the board substantial leeway to establish regulations; will the remainder of the board support the dissident seven, reject their assertions, or find another path?

The Board of Regents, for the first time in anyone’s memory is asserting itself and reminding the governor that the state constitution grants to the Regents, not the Governor, the power to determine educational policy.

Unfortunately the P-12 Committee, the meeting at which the action will take place is not webcast. I will be tweeting (@edintheapple) – as fast as my fingers can “tweet.”

Maybe, just maybe, a brave group of board members will listen to their constituents and return education to parents and educators.

Position Paper Amendments
to Current APPR Proposed Regulations
BY SIGNATORIES BELOW JUNE 2, 2015
We, the undersigned, have been empowered by the Constitution of the State of New York and appointed by the New York State Legislature to serve as the policy makers and guardians of educational goals for the residents of New York State. As Regents, we are obligated to determine the best contemporary approaches to meeting the educational needs of the state’s three million P-12 students as well as all students enrolled in our post-secondary schools and the entire community of participants who use and value our cultural institutions.
We hold ourselves accountable to the public for the trust they have in our ability to represent and educate them about the outcomes of our actions which requires that we engage in ongoing evaluations of our efforts. The results of our efforts must be transparent and invite public comment.
We recognize that we must strengthen the accountability systems intended to ensure our students benefit from the most effective teaching practices identified in research.
After extensive deliberation that included a review of research and information gained from listening tours, we have determined that the current proposed amendments to the APPR system are based on an incomplete and inadequate understanding of how to address the task of continuously improving our educational system.
Therefore, we have determined that the following amendments are essential, and thus required, in the proposed emergency regulations to remedy the current malfunctioning APPR system.
What we seek is a well thought out, comprehensive evaluation plan which sets the framework for establishing a sound professional learning community for educators. To that end we offer these carefully considered amendments to the emergency regulations.
I. Delay implementation of district APPR plans based on April 1, 2015 legislative action until September 1, 2016.
A system that has integrity, fidelity and reliability cannot be developed absent time to review research on best practices. We must have in place a process for evaluating the evaluation system. There is insufficient evidence to support using test measures that were never meant to be used to evaluate teacher performance.
We need a large scale study, that collects rigorous evidence for fairness and reliability and the results need to be published annually. The current system should not be simply repeated with a greater emphasis on a single test score. We do not understand and do not support the elimination of the instructional evidence that defines the teaching, learning, achievement process as an element of the observation process.
Revise the submission date. Allow all districts to submit by November 15, 2015 a letter of intent regarding how they will utilize the time to review/revise their current APPR Plan.
II. A. Base the teacher evaluation process on student standardized test scores, consistent with research; the scores will account for a maximum of no more than 20% on the matrix.
B. Base 80% of teacher evaluation on student performance, leaving the following options for local school districts to select from: keeping the current local measures generating new assessments with performance –driven student activities, (performance-assessments, portfolios, scientific experiments, research projects) utilizing options like NYC Measures of Student Learning, and corresponding student growth measures.
C. Base the teacher observation category on NYSUT and UFT’s scoring ranges using their rounding up process rather than the percentage process.
III. Base no more than 10% of the teacher observation score on the work of external/peer evaluators, an option to be decided at the local district level where the decisions as to what training is needed, will also be made.
IV. Develop weighting algorithms that accommodate the developmental stages for English Language Learners (ELL) and special needs (SWD) students. Testing of ELL students who have less than 3 years of English language instruction should be prohibited.
V. Establish a work group that includes respected experts and practitioners who are to be charged with constructing an accountability system that reflects research and identifies the most effective practices. In addition, the committee will be charged with identifying rubrics and a guide for assessing our progress annually against expected outcomes.
Our recommendations should allow flexibility which allows school systems to submit locally developed accountability plans that offer evidence of rigor, validity and a theory of action that defines the system.
VI. Establish a work group to analyze the elements of the Common Core Learning Standards and Assessments to determine levels of validity, reliability, rigor and appropriateness of the developmental aspiration levels embedded in the assessment items.
No one argues against the notion of a rigorous, fair accountability system. We disagree on the implied theory of action that frames its tenet such as firing educators instead of promoting a professional learning community that attracts and retains talented educators committed to ensuring our educational goals include preparing students to be contributing members committed to sustaining and improving the standards that represent a democratic society.
We find it important to note that researchers, who often represent opposing views about the characteristics that define effective teaching, do agree on the dangers of using the VAM student growth model to measure teacher effectiveness. They agree that effectiveness can depend on a number of variables that are not constant from school year to school year. Chetty, a professor at Harvard University, often quoted as the expert in the interpretation of VAM along with co-researchers Friedman & Rockoff, offers the following two cautions: “First, using VAM for high-stakes evaluation could lead to unproductive responses such as teaching to the test or cheating; to date, there is insufficient evidence to assess the importance of this concern. Second, other measures of teacher performance, such as principal evaluations, student ratings, or classroom observations, may ultimately prove to be better predictors of teachers’ long-term impacts on students than VAMs. While we have learned much about VAM through statistical research, further work is needed to understand how VAM estimates should (or should not) be combined with other metrics to identify and retain effective teachers.”i Linda Darling Hammond agrees, in a Phi Delta Kappan March 2012 article and cautions that “none of the assumptions for the use of VAM to measure teacher effectiveness are well supported by evidence.”ii
We recommend that while the system is under review we minimize the disruption to local school districts for the 2015/16 school year and allow for a continuation of approved plans in light of the phasing in of the amended regulations.
Last year, Vicki Phillips, Executive Director for the Gates Foundation, cautioned districts to move slowly in the rollout of an accountability system based on Common Core Systems and advised a two year moratorium before using the system for high stakes outcomes. Her cautions were endorsed by Bill Gates.
We, the undersigned, wish to reach a collaborative solution to the many issues before us, specifically at this moment, the revisions to APPR. However, as we struggle with the limitations of the new law, we also wish to state that we are unwilling to forsake the ethics we value, thus this list of amendments.
Kathleen Cashin
Judith Chin
Catherine Collins
*Josephine Finn
Judith Johnson
Beverly L. Ouderkirk
Betty A. Rosa
Regent Josephine Finn said: *”I support the intent of the position paper”
i Raj Chetty, John Friedman, Jonah Rockoff, “Discussion of the American Statistical Association’s Statement (2014) on Using Value-Added Models for Educational Assessment,” May 2014, retrieved from:
http://obs.rc.fas.harvard.edu/chetty/value_added.html. The American Statistical Association (ASA) concurs with Chetty et al. (2014): “It is unknown how full implementation of an accountability system incorporating test-based indicators, such as those derived from VAMs, will affect the actions and dispositions of teachers, principals and other educators. Perceptions of transparency, fairness and credibility will be crucial in determining the degree of success of the system as a whole in achieving its goals of improving the quality of teaching. Given the unpredictability of such complex interacting forces, it is difficult to anticipate how the education system as a whole will be affected and how the educator labor market will respond. We know from experience with other quality improvement undertakings that changes in evaluation strategy have unintended consequences. A decision to use VAMs for teacher evaluations might change the way the tests are viewed and lead to changes in the school environment. For example, more classroom time might be spent on test preparation and on specific content from the test at the exclusion of content that may lead to better long-term learning gains or motivation for students. Certain schools may be hard to staff if there is a perception that it is harder for teachers to achieve good VAM scores when working in them. Overreliance on VAM scores may foster a competitive environment, discouraging collaboration and efforts to improve the educational system as a whole. David Morganstein & Ron Wasserstein, “ASA Statement on Using Value-Added Models for Educational Assessment,” Published with license by American Statistical Association, April 8 2014, published online November 7, 2014:http://amstat.tandfonline.com/doi/abs/10.1080/2330443X.2014.956906. Bachman-Hicks, Kane and Staiger (2014), likewise admit, “we know very little about how the validity of the value-added estimates may change when they are put to high stakes use. All of the available studies have relied primarily on data drawn from periods when there were no stakes attached to the teacher value-added measures.” Andrew Bacher-Hicks, Thomas J. Kane, Douglas O. Staiger, “Validating Teacher Effect Estimates Using Changes in Teacher Assignments in Los Angeles,” NBER Working Paper No. 20657, Issued in November 2014, 24-5:http://www.nber.org/papers/w20657.
ii Linda Darling-Hammond, “Can Value Added Add Value to Teacher Evaluation?” Educational Researcher, March 2015 44, 132-37:http://edr.sagepub.com/content/44/2/132.full.pdf+html?ijkey=jEZWtoEsiWg92&keytype=ref&siteid=spedr.

Restoring Trust: Can the Regent/SED, Unions and Superintendents Agree on a “Valid. Reliable and Fair” Teacher Evaluation System?

As part of my union rep duties I served on committees to select teachers for new schools; my favorite question was, “What was the best lesson you taught in the last few weeks and how do you know it?” Teachers had no problem describing lessons, and lots of trouble explaining how they could assess the effectiveness of the lesson. A few would say “exit slips,” others explain that checking the homework assessed the effectiveness of the lesson, or that experience was the best guide. In the real world we plow through the curriculum, an occasional unit exam, differentiating lessons, re-teaching concepts, understanding that god in her wisdom did not make us or our students all equal.

We have been giving statewide exams for decades, before No Children Left Behind, in grades four and eight, and Regents exams are more than a century old. Yes, we did have a triage system, classes were homogeneously grouped and students who failed to make academic progress were placed in classes with similar kids. At the end of the line kids dropped out or received a lesser diploma, and moved into the workforce. Low-skilled union jobs were commonplace, the education system was the “divider” which steered kids to college or the world of work.

For the last thirty years our economy has undergone structural changes, the low-skilled union jobs have fallen victim to automation or moved overseas.

Are schools appropriately preparing students for the rapidly and continuously changing world of work, or, to use the commonly used term, are we preparing “college and career ready” students?

The answer begins in the world of baseball.

I was attending an education conference, and as commonly happens, a book was handed out, not a dense text, not a “how to” book, not a messianic message, the book was Michael Lewis’ Moneyball: The Art of Winning an Unfair Game (2003),

Moneyball is a quest for the secret of success in baseball. Following the low-budget Oakland Athletics, their larger-than-life general manger, Billy Beane, and the strange brotherhood of amateur baseball enthusiasts, Michael Lewis has written not only “the single most influential baseball book ever” (Rob Neyer, Slate) but also what “may be the best book ever written on business” (Weekly Standard). [Lewis’] … intimate and original portraits of big league ballplayers are alone worth the price of admission—but the real jackpot is a cache of numbers—numbers!—collected over the years by a strange brotherhood of amateur baseball enthusiasts: software engineers, statisticians, Wall Street analysts, lawyers and physics professors.

Baseball is no longer ruled by the cigar chomping old-timers, every decision, from salary negotiations, to valuing players, to comparing players, to which pitch to throw to each batter is guided by a mathematical algorithm. As a wonky baseball friend claims, “You could prop Bernie in the corner of the dugout (“Weekend at Bernie’s“) and IBM’s Watson could manage the team.”

We even have a term for baseball data; Sabermetrics, a hobby among a handful of nerds now rules the national pastime.

The widespread use of data has not solved the problems of baseball, the numbers of Afro-American ballplayers and fans has sharply diminished, the fan base is aging out; data may drive decisions, the American pastime is facing a ticking clock.

See Chris Rock on baseball: http://deadline.com/2015/04/chris-rock-baseball-real-sports-with-bryant-gumbel-hbo-video-1201414387/

Whether we like it or not, understand it or not, data drives decisions across a wide spectrum. Ian Ayres, Super Crunchers, Why Thinking by Numbers Is the New Way to be Smart (2007) chronicles how data has embedded itself, from predicting the quality of red wines, to driving the medical profession, to determining which prisoners should be paroled, dense regressive mathematical models rule.

It is not surprising that data influences core decisions in education.

The New Teacher Project 2009 “Widget Effect” report resounded across the education domain,

All teachers are rated good or great. Less than 1 percent of teachers receive unsatisfactory ratings, making it impossible to identify truly exceptional teachers.

• Professional development is inadequate. Almost 3 in 4 teachers did not receive any specific feedback on improving their performance in their last evaluation.

• Novice teachers are neglected. Low expectations for beginning teachers translate into benign neglect in the classroom and a toothless tenure process.

• Poor performance goes unaddressed. Half of the districts studied have not dismissed a single tenured teacher for poor performance in the past five years.
The result has been the movement to assess teacher performance by applying dense mathematical regression models to education, attempting the compare teacher to teacher based upon student achievement data, using a range a variables to level the playing field.

In other words, there is no teacher evaluation system.

The new current systems attempt to address the absence of systems using statistical methods called regression analysis. The students take a common exam, a state test for example, the model allowing for a range of variables, namely, economic status of test takers, students with disabilities, the level of disability, English language learners, student attendance, etc., and the formula differentiates among teachers, within a margin of error.

As with all statistical data sets there are errors of measurement – that “plus or minus” that warns us that the results fall within a range.

“Candidate A leads candidate B 52-48 with an error of measurement of “plus or minus 4%”,” a statistical tie.

As school districts begin to create the models, usually referred to as Value-Added Models or Growth Models, “experts” warn about the problems of VAM. At the NYSED Education Learning Summit three education experts were critical, VAM data was not ready for prime time and a fourth expert argued that VAM was better then what preceded and suggested a combination of VAM student performance data, teacher observations and student surveys.

By mid-June the NYS Board of Regents/SED have to create regulations to implement another new teacher evaluation system in New York State.

The current teacher evaluation law has been on the books for three years (two years in New York City), and has raised more questions than answers.

* Are teachers in Rochester and Syracuse less able or is the algorithm flawed? (A law suit is in progress)
* Are teachers of poorer students (Low SES), English language learner and students with disabilities less able, or, is the algorithm, the formula, flawed?
* Conversely, are teachers higher income (High SES) district more able than other teachers?

On Monday the Regents will begin an in-depth discussion of the new Matrix model (See the Education Learning Summit page here)

The movement to Common Core tests was a disaster, probably too mild a term. Instead of phasing in the Common Core-aligned tests the decision-makers, led by Commissioner John King, used the “push off the end of the diving board” approach. Randi Weingarten asked King to incorporate a “save-harmless” for a year or two, to no avail. The result: over 100,000 parents opting-out, the opt-out movement is spreading across the nation, teachers are highly suspicious of everything and most importantly, electeds are threatened and are introducing legislation to weaken SED initiatives.

The Regents have four new members, three former superintendents and a former Buffalo school board member, who join former superintendents Cashin, Rosa and Young. Regents Cashin and Rosa voted against the original teacher evaluation plan.Hopefully their experience can put the train back on the tracks.

How can the Regents/SED win back the confidence of parent and teachers?

The complex numbers cannot be seen as a tool to punish teachers.

The movement to Common Core-aligned test ignored the impact of drastic drops in test scores, and, when the public expressed discomfort John King blamed parents and “outside agitators;” a classic example of how NOT to roll out a new initiative.

Advice: Accept the recommendations of NYSUT, the state teacher union, the UFT, the NYC teacher union and the NYC Department of Education., in other words, create buy-in, remember Rule # 1 of “change,” participation reduces resistance.

Each year a technical committee that includes the unions can review and modify the model. Shoving new proposals down people’s throats and vigorously defending the position hasn’t worked out too well. Incremental change builds trust.

A simple example: A teacher receives a score of 42 on a model that has an error of measurement of plus or minus 10 %. In other words the teacher’s score could be within the range of 32 to 52. If the cut scores between D (Developing) and E (Effective) is 50 the teacher should be graded E (Effective). We should accept the error of measurement issue and not disadvantage the teacher.

Similarly, on the observation rubrics, we should accept the recommendations of the unions and the NYC Department, set low cut scores and examine the results each year.

If the Regents/SED decides to select cut scores that are “tougher,” the teacher and parent wars will escalate and electeds will jump on the “voter” side, the Opt-Outs, the trash the system side. Charter schools, voucher and tax credit supporters will argue public schools are not fixable.

Just as Chris Rock points out re the underlying problems of baseball, the underlying problems of teacher quality will not be resolved by teacher evaluation regulations; however, the Regents/SED need to be standing on a stage with NYSUT President Karen Magee, UFT President Michael Mulgrew and NYC Chancellor Carmen Farina joining together announcing a teacher evaluation plan that is “valid, reliable and fair.”

Regaining a lost trust is crucial and must precede any further steps, until we can trust each other we cannot move forward, and, with the vandals at the gates, Regents/SED must take the first step.