Tag Archives: Board of Regents

“Evening the Playing Field:” Will the ESSA Accountability Plan Acknowledge the Work of Teachers in “Truly Disadvantaged” Schools? Will the plan be “Equitable?”

A teacher: “We love our kids and love to teach in this school, we make a difference in the lives of our kids. Our kids are poor, really poor, some live in shelters, others in foster care, trauma is part of their daily lives. We’re building out our community school, we prepare our kids to learn by feeding them, by searching for contributions of clothes, and, make our classes as rich as possible. I wanted to take my class on a series of trips out of the neighborhood, my principal said wait till after the tests, the future of our school depends on six days of testing, it’s sad that no one cares about the social and emotional needs of our kids, needs that precede the ability to learn. We just want an even playing field.”

In some school districts kids come into school knowing their letter and number facts, in other schools it’s  their first exposure; kids are behind from day 1.

In “The Truly Disadvantaged: The Inner City, the Underclass, and Public Policy (1987)” sociologist William Julius Wilson details how programs across the political spectrum have failed the underclass, he is sharply critical of conservative and liberal policies.

Wilson posits social isolation–a distinction which shifts the problem from the psychological to the socio-economic realm. Instead of blaming poverty and its associated pathologies primarily on the individual, as conservatives do, or on the effects of contemporary racism, as some liberal scholars do, Wilson calls for a “refocused liberal perspective” which emphasizes “the dynamic interplay between ghetto-specific cultural characteristics and social and economic opportunities.”

Thirty years later our society, and especially our schools are even more segregated.

The result is islands of poverty, what Wilson called the “truly disadvantaged,” whose children enter school far behind other children.  The Harvard Education Letter writes,

  • According to a seminal 1995 study by Betty Hart and Todd Risley, 3-year-olds whose parents are professionals have vocabularies that are 50 percent larger than those of children from working-class families, and twice as large as children whose families receive welfare.
  • “We could eliminate at least half, and probably more, of the black-white test score gap at the end of twelfth grade by eliminating the differences that exist before children enter first grade.”
  • In a 2002 study, Valerie E. Lee and David T. Burkam of the University of Michigan found that at kindergarten entry, cognitive scores of children in the highest socioeconomic group were 60 percent higher than those of the lowest group.

From the 1965 Elementary and Secondary Education Act to the 2002 No Child Left Behind Law up to our current iteration, the Every Student Succeeds Act administrations across the political landscape have tried to legislate equity. Previous leadership of education in New York State jumped on Common Core State Standards and testing: the unintended consequence, a massive opt-out movement primarily among middle class white parents.

The current Board of Regents, led by Chancellor Betty Rosa is a newer board. Former superintendents and teachers, a judge, a doctor, an attorney. a parent activist, a range that crosses the spectrum of experience and background.  The board is attempting to create, as required by the ESSA law, a new school accountability system that goes beyond test scores. The new law requires grades 3 – 8 tests in ELA and Math but does not limit the accountability system to test scores.

The process of building a new accountability system has been transparent. Scores of open, facilitated meetings, a think tank made up of stakeholders and hours upon hours of open meetings with national experts Linda-Darling Hammond, Learning Policy Institute and Scott Marion, the Center for Assessment.

Your correspondent has sat through them all.

This past Tuesday a three hour plus session with Darling-Hammond and Marion tele-communicating with the board. It was crunch time, decisions had to be made.

You can check out nine presentations here.

Or, check on the individual power points/reports below:

Status of Development of ESSA Plan

Promoting Diversity – Integration in New York State

Considerations for the New York State Assessment System

Models for State Performance Assessment Systems

Building an Accountability and Assessment system Under ESSA

Putting It All Together – Annual Differentiation Under ESSA

I know parents and teachers ask: why do we need state tests? The answer is simple: the law requires that each state include state tests in any school accountability system and the system must identify the bottom 5% of Title 1 schools.

If you’re going to go beyond test scores what would you add?

You can add other indicators: for example, growth, the increase in scores from year to year, you could add counting subgroups differently, you could add chronic absenteeism, maybe other items. Additionally you can weight the indicators: for example ELA and Math scores could count 30% each, growth can count 20%, other items 5% or 10% resulting in a cumulative score.

Think in terms of a dashboard with the indicators across the top and the levels down the vertical column. Do you use satisfactory, unsatisfactory, levels 1 to 4, ineffective, developing, effective highly effective? Do schools receive an overall numerical score? a letter grade? How do you identify the bottom 5% of Title 1 schools?

One of regents members asked if there was a “scientific” method of making these determinations. Scott Marion explained these were subjective decisions, capturing stakeholder values, for weighting indicators is a subjective decision. Marion explained that subjective is not a negative; Marion discussed “credible  defensibility,” making value-based decisions that reflected the experiences of the members..

After extensive discussion the members agreed on a weighted dashboard. The first draft will be released at the May regents meeting, public comments, meetings around the state, submitted to the governor and submission to the feds in September.

Colorado has completed and submitted a dashboard plan that goes beyond test scores: read a description of the plan here.

Hopefully, maybe, the changing of the metrics will “even the playing field” for the “truly disadvantaged.”

The next steps will be to begin to explore alternative assessments, aka authentic assessments. Vermont and New Hampshire involved in pilots; however, the path is long and complex, and to quote Robert Frost,

The Road Not Taken

Two roads diverged in a yellow wood,

And sorry I could not travel both

And be one traveler, long I stood

And looked down one as far as I could

To where it bent in the undergrowth;

 

Then took the other, as just as fair,

And having perhaps the better claim,

Because it was grassy and wanted wear;

Though as for that the passing there

Had worn them really about the same,

 

And both that morning equally lay

In leaves no step had trodden black.

Oh, I kept the first for another day!

Yet knowing how way leads on to way,

I doubted if I should ever come back.

 

I shall be telling this with a sigh

Somewhere ages and ages hence:

Two roads diverged in a wood, and I—

I took the one less traveled by,

And that has made all the difference.

Advertisements

The New, Feisty Board of Regents Explores Principal Preparation: Why Don’t We Have Better Principals?

[Election Update: Yuh Line Niou won the six-way primary in Shelly Silver’s former district as well as all other Ed in the Apple endorsed candidates with the exception of Robert Jackson; however, the Bloomberg/Charter candidate, Micah Lasher lost to a candidate supported by the Independent Democratic Coalition – the breakaway gang of five that caucuses with the Republicans]

The new Board of Regents is a feisty group!!

The Board is a policy board; they hire the CEO, the commissioner, and set overall policy for the state. The line between what is policy and what are operations is a blurred line: a prime example.

In December the Regents voted to accept the 21 recommendations of the Cuomo Task Force on the Common Core.

Recommendation 15: Undertake a formal review to determine whether to transition to untimed tests for existing and new State standardized test aligned to the standards.

A month later the Department announced a shift to untimed tests;  the “formal review” apparently did not involve the Board.

Initially the Commissioner was ecstatic over the unparalleled one year jump in test scores, until the Chancellor, Betty Rosa tuned down the exuberance.  Without knowing which students took extended time the state has set a new baseline, there can be no valid comparisons – you cannot compare apples to oranges. The Regents members were clearly unhappy – why weren’t they involved in the “formal review?”

Under the leadership of Chancellor Tisch and John King, with a few exceptions, the Board was quiescent.

The current members are activists, in order to create policy they clearly intend to take a deep dive into the issue. A prime example: the four exams required for teacher certification. The co-chairs of the Higher Education Committee have held forums all over the state, hundreds of college staff, and degree seekers, have attended and testified. The Board is leading the steps to reconfigure the teacher preparation process that was imposed by Tisch/King.

No longer does the Chancellor and the Commissioner run the show. Chancellor Rosa epitomizes collaborating with her Regent partners.

The September 12th Regents Meeting began with a detailed exploration of a new grant from the Wallace Foundation:  the Principal Preparation Project. In prior years the project would have landed with the Regents Research Fellows with a nary a word of discussion with the Regents members. The world has changed.

After a Power Point presentation the new Board peppered the Deputy Commissioner with questions;

Regent Johnson mused over the purpose of the project.  We must acknowledge the impact of poverty, issues of race and changing demographics. Why weren’t Civil Rights organizations on the team? Regent Mead was concerned over the three years of teaching as a minimum requirement – New York City has a seven year requirement. Regent Norwood was wondering why social/emotional issues appeared absent from the project as well as working in diverse environments, and, the retention of leaders in low performing schools were absent. Regent Brown was concerned with the absence of diversity concerns in the project, should issues of race, i. e., “white privilege” and “cultural competency,” be included in project curriculum?

The discussion went on and on….

In order to become a principal in New York State the applicant must complete an “approved” program; however, the selection is by the elected lay school board, or, in New York City, by the Chancellor; all the state does is create an applicant pool.

A little history:

The first wave of reform swept the nation after the Civil War and culminated in the passage of the Pendleton Act in 1883 – establishing a federal civil service system. The reform movement moved to the states, and, after the creation of New York City (“The Great Consolidation”), the merging of the five boroughs, the legislature moved to reform a political hiring system, by creating a Board of Examiners.

Read a history of principal selection here: https://mets2006.wordpress.com/2008/08/07/the-quest-for-the-leadership-gene-how-do-we-findselect-the-best-school-leaders/

From rigorous examinations to a handful of credits and selection by elected Community School Boards to the Leadership Academy, we haven’t found any magic bullets.

Half-jokingly, I mused that maybe there was a leadership gene. Maybe I’m right!

… a quarter of the observed variation in leadership behaviour between individuals can be explained by genes passed down from their parents. – See more at: https://www.ucl.ac.uk/news/news-articles/0113/15012013-leadership-genetics#sthash.Nmnip8lR.dpuf

If you ask teachers about supervisor competence you will find a wide variability, some praise school leaders, many more are critical.  An NYU Study a few years ago, using student scores on state exams as a measurement: insignificant differences between Leadership Academy and non-Leadership Academy principals.

I have a few questions:

* What percentages of applicants are accepted into leadership programs? Is the quality of the applicant’s teaching part of the applicant selection process, and, if so, how do you measure the quality? (I fear programs accept the vast percentage of applicants)

* Are online or blended learning courses acceptable? Are these courses of the same quality as face-to-face courses?

* How often does the supervising teacher visit the candidate? Four times a year? Weekly? What is the quality of the internship? How is it measured?

* What percentage of candidates find jobs within five years? How successful are the candidates as supervisors and how do we measure success?

The finest leadership I have seen is the leadership provided by coaches, whether athletic, music or dance.

The ultimate question: is this project worthwhile?  Since the state does not hire or supervise principals can changing the requirements actually change who gets hired?  Do we have to change the “hirers” before we can change the “hirees”?

Looking ahead: every state must comply with the new Every Student Succeeds Act (ESSA) and design a state plan -more about the process in my next post.

The Politicization of State Tests: Creating Tests in Which “All Students in New York State Are Above Average”

When the dust cleared the greatest ally to the anti-testing clique was (roll of drums!!!)  MaryEllen Elia, the New York State Commissioner of Education.

The deeply flawed state tests (“All children are above average”) reignited the argument – why do we have state test at all (aside from the federal requirements)?

Statewide ELA test scores jumped by around 7% – although the racial achievement gap remained the same.

A magic potion, incompetence or simply political legerdemain?

A little review: in September, 2015 Governor Cuomo reconvened a blue ribbon panel, actually a process to repair the Governor’s foolhardy attacks on teachers and parents. In 2014 it appeared that Cuomo had a clear path the Democratic nomination for his second term and deep pockets for the November general election. Seemingly out of nowhere Zephyr Teachout, a law professor at Fordham challenged Cuomo for the Working Families Party spot on the ballot and challenged Cuomo in the Democratic primary. While the teacher union made no endorsement some members and locals were on the Teachout side. After defeating Teachout and Rob Astorino, his Republican challenger Cuomo decided to punish teachers. He cozied up to the charter school folks, used the budgeting process to tack on legislation to extend teacher probation, and, was nastier than usual.  NYSUT, the statewide teacher union responded with a series of aggressive TV ads and the opt-out movement was created, 20% of kids opted-out of the 2015 state tests.

Cuomo’s popularity rating tumbled.

I suspect clearer heads prevailed.

The purpose of the Task Force was to guide education policy from afar and place the Board of Regents and the commissioner in the foreground. The recommendations were more than recommendations; they were a pathway for state education policy. (Cuomo: This is the endgame – you figure us out how to get us there)

The Task Force Report (Read here), which was released in December, contained twenty-one recommendations, the last recommendation was a moratorium on the use of state tests to evaluate principals and teachers for four years, applauded by the teacher union.  The recommendations called for a thorough review of the Common Core Standards and teachers would be included in every step of the process.

Recommendation 15: Undertake a formal review to determine whether to transition to untimed tests for existing and new Standardized tests aligned to the standards; not controversial, garnered little,  if any discussion; perhaps a pilot in a few schools and school districts across the state.

Surprisingly, very surprisingly, without any discussion with the Board of Regents, the Commissioner announced that the 2016 state tests would be untimed.

The January announcement, entitled “Changes for the 2016 Grades 3-8 ELA and Mathematics Tests” begins,

This memo outlines changes made as a result of feedback from the field:

* Greater involvement of educators in the test development process

* Decrease in the number of test questions, and

* A shift to untimed testing

The announcement came from Angela Infante, Deputy Commissioner, Office of Instructional Support and Peter Swerdewski, Assistant Commissioner, Office of State Assessment.

The state document states, “…students will be provided with as much time as they need.” No pilot, no transition, jumping off the diving board into the pool, and, the state made no attempt to identify students who took additional time.

The scores soared, the state commissioner, in the Daily News admits the scores are “not exactly a perfect comparison,”

After widespread opposition to the difficulty of the tests erupted in 2015, state education department officials shortened the exams for 2016 and eliminated time limits.

“Because of the changes in testing, it’s not exactly a perfect comparison,” Elia said. “And even with the increases this year, there remains much work to be done.”

The state spent many millions of dollars purchasing tests, teachers and students months of test prep, to collect data from what turns out to be a non-standardized test. A test that might not even meet federal requirements, although I’m sure the feds will simply ignore the faux jump in scores.

Was the test itself “harder” or “easier;” many months down the road a Technical Advisory Committee (TAC) will release a report, hundreds of pages of dense analysis that few will read and fewer will understand.

The basic questions: are the results of the test useful?  Can they be compared with the previous year? Can schools and school districts be compared? And, at the top of the list: are the schools in New York State making academic progress?

Howard Wainer, a Distinguished Research Scientist, the author of innumerable books and articles, an internationally recognized expert writes,

Because of the changes this year’s scores can’t be compared to last year’s and because of the untimed nature of the test (and there being no record of how long anyone took) you can’t compare scores of students who took it this year with one another. It is, in no uncertain terms, an unstandardized test.

This test is akin to measuring children’s heights but allowing some students, we don’t know who, to stand on a stool, we don’t know how high, and then declaring some taller than others.

Fred Smith, another testing expert, writing in City Limits, had doubts about the validity of the test before the test administration.

Either the state education psychometrician is lacking in competence, or knew by adopting untimed tests scores would likely jump – either is unacceptable.

If the state continues down the same path, retaining the untimed tests, even if it keeps track of students who take extra time, and the amount of extra time, we will be once again be comparing apples to oranges. Kids who take extra time or choose not to take extra time may not be the same kids as this year – we simply can’t know.

Will states across the nation also jump on the untimed tests bandwagon?

In the politicized world of education the charter school folk and their acolytes beamed at higher scores, of course, we have no way of knowing why charter school scores were generally higher than public schools, and, the pro-charter print media crowed. Mayor de Blasio and Chancellor Farina also took a victory lap, and the Mayor immediately claimed the scores were proof that mayoral control be made permanent.

Board of Regents Chancellor Rosa reminded us it’s not time for a victory lap, unfortunately everyone else is milking the results – de Blasio and Farina, the charters and principals and teachers are breathing a sigh of relief.

A perverse kind of victimless crime: except for the kids who were tortured preparing for a non-standardized test.

Although the law has changed, No Child Left Behind has been replaced by the Every Student Succeeds Act; the requirement for annual testing in grades 3-8 remains. The Leadership Conference is an umbrella group representing the major civil rights organizations across the spectrum has strongly supported the accountability requirements, aka, testing and reporting scores by subgroup, and, the law is not changing.

Testing is here to stay.

The US Department of Education has announced they will be selecting six or so states or consortiums of states to play with alternate assessments.

The anti-testing crowd points to the new law and the testing kerfuffle in New York State, why not move to portfolios and performance tasks to current replace testing? This is not a new idea.

Vermont spent a decade working to create an assessment system based on portfolios, and after an external report pointed to fatal flaws, abandoned the effort.

…report by the RAND Corporation … found that the “rater reliability” in scoring the portfolios–the extent to which scorers agreed about the quality of a student’s work–was very low. The researchers urged the state to release the assessment results only at the state level.

Daniel M. Koretz, a senior social scientist at RAND and the report’s author, said the low levels of reliability indicate that the scores are essentially meaningless, since a different set of raters could come up with a completely different set of scores.

Can thousands of teachers be expected to rate portfolios the same?

The portfolio process was expensive, extremely time consuming  and there is no guarantee the portfolio work was not “assisted” by parents or others .

Yes, portfolios and performance tasks are effective classroom tools and in the perfect world might be a way of assessing student progress, in the real world, the world in which we live, it is not reasonable to expect inter-rater reliability.

The anti-testing movement will not disappear and the opt-out movement is alive.

What is absent is leadership – Arne Duncan drove us down a path for seven years that divided education: reformers versus deformers, marketeers versus public schools, unions versus the hedge funders: education is bitterly divided. Will the next president nominate an education leader who can bring together the disparate constituencies?

Education is adrift and the unstandardized testing regimen in New York State is a prime example.

Why Did New York State Test Scores Jump? Better Instruction? Untimed Tests? All the Kids Got Smarter, or, Shenanigans?

If you want to bury a news story you issue the press release on a Friday afternoon, if you want as much mileage as possible you issue the release on a Tuesday morning, followed by a press conference, in person and online, followed by laudatory speeches across the state and try to maximize the time the story garners headlines and clicks.

The State Education Department released the 2016 grades 3-8 ELA and Math scores on Friday afternoon with an odd presser. The test scores up, way up; why is the SED ashamed?

You can take a deep dive into the New York City Scores here: http://schools.nyc.gov/Accountability/data/TestResults/ELAandMathTestResults

The SED analysis of the state scores with many disaggregated charts here and here.

The Commissioner was careful not to publicly laud the increase in the scores,

But rather than celebrate the largest bump since New York adopted new tests tied to the Common Core Learning Standards, education officials reported the increases with caution. They suggested that changes in how the tests were given – not actual improvement by schools and students – may have accounted for the gains.

State Education Commissioner MaryEllen Elia also warned against making comparisons with previous years, which is typically done to evaluate schools and teachers.

“It’s not an apples to apples comparison and should be viewed in that context,” Elia said during a news conference when the results were released Friday.

For the data wonks who want to parse the results check out the files here and here.

The SED states, “…  changes in how the tests were given – not actual improvement by schools and students – may have accounted for the gains;” however, a deeper analysis is necessary.

If the increases are due to fewer questions and untimed tests, we should know, if both teachers and kids have been exposed to the more effective Common Core instruction and better professional development, we should know, or, if the SED, as some suspect, manipulated the process, we should know. All of the kids in New York State getting smarter just doesn’t seem creditable.

Under Commissioner Mills test scores increased year after year, when Chancellor Tisch and new Commissioner Steiner took over they asked a Harvard professor, Daniel Koretz to take a look – sure enough – the SED had been using many of the same questions year after year. Whether incompetence, or, more likely a method of increasing scores, we’ll never know. Scandals in Atlanta and accusations elsewhere have cast doubt on the entire testing regimen. Jumps in test scores are treated with skepticism.

For years Howard T. Everson chaired the Regents Technical Advisory Committee (TAC) and was sharply critical of test score inflation.

But given all the flaws of the test, said Prof. Howard T. Everson of the City University of New York’s Center for Advanced Study in Education, it is hard to tell what those rising scores really meant.

“Teachers began to know what was going to be on the tests,” said Professor Everson, who was a member of a state testing advisory panel and who warned the state in 2008 that it might have a problem with score inflation. “Then you have to wonder, and folks like me wonder, is that real learning or not?”

Each year after the release of the state tests scores the TAC issued a lengthy analysis of the quality of the test. Recently the TAC process has changed, as I understand the current process the TAC report goes to the test creator, Pearson, (now replaced by Questar) who vets the report, over the last few years the report was released a year after the test and was so heavily “massaged” it was meaningless.

The SED/Regents should, in the footsteps of Tisch and Steiner, immediately ask Everson or Koretz or a colleague with equally impeccable credentials to examine the current state test results.

If, in fact, the Commissioner doesn’t know why scores jumped we have to ask: why not?  If untimed tests resulted in higher test scores shouldn’t Regents Exams be untimed?  If the increased exposure to better Common Core instruction resulted in higher scores why are the Algebra 1 and Geometry scores not increasing?

Shrugging and simply saying we’re happy with increased scores but we’re clueless as to why is simply not acceptable. Data should influence policy at all levels, and, we have to be confident that the testing regimen is creditable.

The New York State Legislature Adjourns with a “Whimper,”as Educational Policy-Making Moves to the Board of Regents

This is the way the world ends
This is the way the world ends
This is the way the world ends
Not with a bang but a whimper

T. S. Eliot, The Hollow Men (1925)

The last stanza of Eliot’s poem is an apt description of the end of the 2016 legislative session. The final days, called “the Big Ugly,” is a scramble, an endgame, the Republicans and the Democrats vying for an advantage as the state moves toward the November election. All the seats in the legislature, the 150 in the Assembly and the 63 in the Senate will be on the ballot. While the Assembly is firmly in the Democratic column the Senate is far more complex, and byzantine. The Democrats hold a single seat edge in the Senate (32-31); however five Democrats (Jeff Klein, Diane Savino, Tony Avella, David Valesky, and David Carlucci), the Independent Democrat Conference (IDF), under the leadership of Klein (Bronx) caucuses with the Republicans, giving the Republicans control of the Senate.

Hanging in the balance were mayoral control, campaign finance reform, removal of pensions for convicted legislators, online fantasy sports betting and scores of other bills.

You may ask: why is all this conflict and wheeling and dealing necessary? Why can’t legislators have civil conversations and decide the issues?

James Madison, in Federalist # 51 wrote,

Ambition must be made to counteract ambition. The interest of the man must be connected with the constitutional rights of the place. It may be a reflection on human nature, that such devices should be necessary to control the abuses of government. But what is government itself, but the greatest of all reflections on human nature? If men were angels, no government would be necessary. If angels were to govern men, neither external nor internal controls on government would be necessary

The Constitutional Convention (1787) was not covered in CSPAN; the Constitutional Convention was a secret meeting. The only notes we have are Madison’s personal notes, not made public until after the death of all the delegates, The fifty-three delegates argued, came and went, delivered lengthy speeches, met in private, and made deals.

Slavery was one of the most significant stumbling blocks, the anti-slavery Northerners versus the slave-holding South, The compromise: slavery is not mentioned in the constitution, the question of slavery was left to the states, and, as part of a compromise; slaves were counted as 3/5th of a ”free person,” and referred to in the clause as “all other Persons.”

Representatives and direct taxes shall be apportioned among the several states which may be included within this union, according to their respective numbers, which shall be determined by adding to the whole number of free persons, including those bound to service for a term of years, and excluding Indians not taxed, three fifths of all other Persons.

Deal-making, as reprehensible as it may seem, is at the essence of making government work.

Whether to extend mayoral control in New York City had nothing to do with education. Weakening the mayor might give the Republicans a chance in the 2017 mayoral election. In spite of pleas from Merryl Tisch and others in the upper echelons of power Senate leader John Flanagan offered “unacceptable” plan after plan until in the closing hours an agreement was reached, the NY Times describes the plan as a one year extension plus,,

It would effectively create a parallel system of charter schools within the city, allowing “high-performing charter schools in good standing” to switch to join the State University of New York umbrella or the Board of Regents of the State Education Department.

Probably a meaningless change, currently charters schools authorized by both New York City and Buffalo make reauthorization proposals after five years, the authorizer, SUNY or the Board of Regents can reject the recommendation. The proposal allows the charter school, if it’s  “high performing and in good standing” to move directly to SUNY or the Regents for reauthorization.

The session is most interesting for what it did not do – the houses steered clear of legislation directing the State Education Department to take any actions. A host of education bills simply died. Neither the governor nor the party leaders had any desire to once again get involved in the morass of teacher accountability or testing, any of the issues that birthed the opt outs and/or angered teachers and their unions.

The budget was generous and the political leaders appear to be leaving the educational decisions to the educational leaders.

In December the Cuomo-appointed Task Force released their report with 21-recommendations: a blueprint for the Commissioner and the Board of Regents. The core of the report was a 4-year moratorium on the use of student test scores as part of a metric to assess teacher performance.

In the six months since the release of the report the Commissioner has made tests untimed, a recommendation in the report, established a number of large field-based committees to review elements of the Common Core, and, the Regents created a number of alternative pathways to graduation.

Quietly, very quietly, the Commissioner announced a change in the observation section of the teacher evaluation regulation. The outside observer would be scrapped – what might be a good idea in theory was both overly complex and a financial burden on school districts. There was no high drama – no headlines, simply an announcement undoubtedly based on quiet discussions.

The decisions before the Board of Regents are complex, politically explosive and without explicit answers.

Can you create a teacher evaluation plan that is acceptable to principals and teachers and not trashed by external critics?

Can better tests win back opt out parents?  And, what do you mean by “better tests?”

Will alternatives to testing, perhaps, portfolios or other performance assessments, be acceptable to the feds, and acceptable to the principals and teachers?  Are performance assessments practicable in actual classroom settings?

Will additional alternative pathways to high school graduation make students more or less prepared for college?

The Regents appear to have a window – three or four years – to make decisions based on their expertise as well as respond to external pressures and scrutiny, and, hovering aloft: “disruptive” solutions such as unlimited charter schools or vouchers.

Windows open, and windows close.

Getting It Right: Building a Research-Based Teacher Assessment System

A couple of years ago I was participating in a Danielson Training Workshop, two Saturdays in a room filled with principals and network support folk. We watched a video of part of a lesson – we were told we were watching a first year teacher in November in a high school classroom.

Under the former Satisfactory/Unsatisfactory rating system the lesson was clearly satisfactory. The Danielson Frameworks (Read the 115-page NYSED document here) requires that teachers are rated on a four-point scale (Distinguished, Proficient, Basic and Unsatisfactory) while New York State also requires a four point scale (Highly Effective, Effective, Developing and Ineffective). The Frameworks divides the teaching process into four domains, 22 components and 76 elements.

The instructor asked us to rate the lesson: at my table we were all over the place. For a teacher in the third month of her first year of teaching the lesson was excellent – clearly “proficient.”  Others argued the time in teaching was irrelevant, you had to rate her against all other teachers regardless of experience – at best, she was “developing.” Inter-rater reliability was absent.

Decades ago the union sent me to an Educational Testing Service conference on teacher assessment; about thirty experienced superintendents from all over the Northeast, and me, one union guy. We began by watching three 15-minutes videos of lessons: one an “old-fashioned” classroom, the kids sitting in rows, the kids answered teacher questions, the kids stood when they answered; the questions were at a high level although a small number of kids dominated the discussion. In the other video kids were sitting at tables, the teacher asked a question, gave the kids a few minutes to “huddle,” and one of the kids answered for the group and the teacher followed up with a few clarifying questions, in the third classroom the kids were at stations around the room, it was noisy, the noise was the kids discussing the assignment, the teacher flitted around the room, answering, clarifying and asking questions.

We were asked to rate the lesson on a provided checklist.

The result: the superintendent ratings were all over the place.

I was serving as the teacher union rep on a Schools Under Registration Review (SURR) team – we were visiting a low performing school. We were told to wait, the principal was busy, four of the 50 teachers were absent and there were three vacancies, the principal was assigning classroom coverages.

At the initial get acquainted session a team member, considering the staffing issues asked, “What are the primary qualities you look for in assessing teacher quality?” The principal blurted, “They come every day and blood doesn’t run out from under the door.”

A colleague was touring a school with very high test scores.  As he walked the building with the principal, he saw uniformly “mediocre” instruction – teacher-dominated, no student engagement. He mentioned the low quality of instruction to the principal, who shrugged, “Why mess with success?”

Once again, there is no inter-rater reliability.

In a number of school districts across the state almost all teachers received maximum observation ratings.

The State Ed folk simply accept the observation ratings of principals and school districts.

Charlotte Danielson, in her other book, Talk About Teaching  (September, 2015), discusses the complex role of the principal as rater as well as staff developer: how can a principal, who is the summative evaluater honestly engage with teachers who they rate?

In an excellent article from the Center for Educator Compensation Reform, Measuring and Promoting Inter-Rater Agreement of Teacher and Principal Performance Ratings (February, 2012), the authors parse the reliability of teacher observation ratings. There are a number of statistical tools to assess reliability – the state uses none of them.

In New York State 60% of a teacher rating is made up of the teacher observation score, and, we have no idea of the accuracy of the rating.

In the pre-Race to the Top days, the Satisfactory/Unsatisfactory rating days, the entire rating was dependent on the observation – in the last year of Bloomberg term 2.7% of teachers in New York City received Unsatisfactory ratings, under the current far more complex system that incorporates student tests scores and other measures of student growth only 1% of teachers were rated ineffective (Read a description of the plan:  APPR 3012-c).

Under the newest  system the other 40% is a combination of Measures of Student Learning and Student Learning Objectives, the use of state test scores is suspended until the 2019-20 school year.

Read a detailed description of the current APPR 3012-d teacher evaluation law here and a lengthy Power Point here.

In May, 2015 the Regents convened a Learning Summit and asked a number of experts to discuss the use of student growth scores (VAM): Watch the lengthy, sometime contentious discussion  here.

With one exception the experts criticized the use of student growth scores (VAM), the VAM scores did not meet the tests of “validity,” “reliability” and “stability.”

There have been glaring errors in the system. In the Sheri Lederman law suit  a teacher had very high observation scores and due to the composition of her class, very low student growth scores. The judge ruled the use of the growth scores, in the individual case, was “arbitrary and capricious.”

The APPR plan negotiated in New York City, on the other hand, allows for appeals by a neutral third party, and, the “neutral” has overturned appeals in which there was a wide disparity between the observation and VAM scores.

The current plan, created by the governor and approved by the legislature has been rejected by teachers and parents. Teachers are convinced that their score is dependent on the ability of the students they teach, not their competence. Parents feel schools are forced to “teach to the test” due to the consequences facing principals and teachers.

Angry parents, angry teachers and principals and a governor and a legislature looking for a way out of the box they created.

And, a cynicism from elements among the public – if two-thirds of kids are “failing” state tests how is it possible that only one percent of principals and teachers are rated “ineffective?”

The Board of Regents has been tasked with finding the “right” plan.

There has been surprisingly little research and public discussion of teacher attrition – in high poverty schools staggering percentages of teachers, 30%, 40%, 50% or more leave within their first few years.

The December, 2015, Cuomo Commission Task Force, in a scathing report, tasked the Regents with “correcting” what has been a disastrous path. Partially the governor creating an incredibly complex teacher evaluation matrix and partially the Commissioner King rushing to adopt the common core, common core testing and teacher evaluation simultaneously.

Can the Regents separate political decisions from research-based and guided decisions? Can the Regents move from the John King path, an emotion-guided political path to actually following “what the research says”?

On Tuesday the new Research Work Group, chaired by Regent Johnson will convene for the first time.

The roadmap for the State Ed Department and the Board of Regents are the twenty-one recommendations of the Cuomo Common Core Task Force. A number of the recommendations: untimed testing, an in-depth review from the field of the standards, greater transparency of the test items, alternatives to the use of examinations for students with disabilities, and, the beginning of an review of teacher evaluation are already in progress.

The Commissioner and the Regents have to regain a lost credibility: from policy emanating from the Gates Foundation and the so-called reformers to policies guided by scholarship and supported by parents and educators.

A Conundrum: How Do You Create a Teacher Evaluation Process That Satisfies Teachers, Principals, Parents, the Legislature and the Governor? (Hint: With Difficulty)

No one’s life, liberty or property is safe while the New York State Legislature is in session.” Anonymous, 19th century.

Diane Ravitch convened her third annual Network for Public Education conference in Raleigh with hundreds of teachers, parents and public school advocates.  The attendees do not represent organizations; they dug into their own pockets to meet with like-minded public education devotees from across the nation. I met a band director from Fort Worth, a second-career math teacher from Jacksonville, a literacy coach from North Carolina; we chatted and shared experiences, we all face incredible challenges and legislatures and privateers intent on eroding the public in public education.

We stood and cheered as Reverend Barber, the leader of the North Carolina NAACP, called a modern day Martin Luther King, preached and taught us – both a history lesson and a sermon.  Bob Herbert challenged us to vote, and emphasized that while coming to the polls in 2008 and 2012 elected Barack Obama, staying away from the polls allowed the Tea Party to seize control of the House, the Senate and state legislatures in 2010 and 2014. A subtle message to the Bernie voters – staying away from the polls in November could lead to a Trump presidency.

Fifty workshops allowed us to meet together in smaller groups. One theme was teacher evaluation: in school districts across the nation student test scores play a significant role in the evaluation of teachers; a Report  by the Network for Public Education is summarized,

72% of respondents also reported that the use of standardized test scores in teacher evaluation had a negative impact on sharing instructional strategies.

Over 41% of black and 30% of Latino/a educators reported racial bias in evaluations.

About 84% of respondents report a significant increase in the amount of teacher time spent on evaluations.

84% of respondents said that the new evaluation system in their state had negatively changed the conversations about instruction between their supervisors and themselves.

75% of respondents stated that these new evaluation systems incorrectly label many good teachers as being ineffective.

Nearly 85% of respondents stated that these evaluation systems do not lead to high-quality professional growth for teachers.

Nearly 82% of teachers reported that test scores are a significant component of their evaluation.

Opposition to the use of student test data to rate/measure/assess teachers has united teachers from across the nation.

At the conference one of the sessions pitted Jennifer Berkshire, aka EduShyster against Peter Cunningham, the Executive Director of Education Post. Jennifer and Peter are on opposite ends of the teacher evaluation spectrum – forty-five minutes of thrust, parry and riposte – spellbinding!!

Critics pointed to research that avers teachers only account for 14% of a student’s test score, family and income account for the largest percentage, therefore, rating teachers by test scores is invalid, Cunningham responded that teachers are the crucial factor in student achievement, we cannot change a family or income, we can change teachers and highly effective teachers have significant impacts on children, as other research shows. Meeting with teachers from across the nation was invigorating; listening to the anti-teacher stories from state after state was discouraging.

A week earlier the New York State United Teachers (NYSUT) annual convention took place in Rochester, Teachers from hundreds of school districts across the state met to debate and set policy for NYSUT. The state is incredibly diverse, New York City and the Big Four (Buffalo, Rochester, Syracuse and Yonkers), the high tax, high wealth suburban districts, the hundreds of rural low tax, low wealth districts, facing sharply different problems. The delegates representing the teachers within the state university system, (SUNY) and the teachers in the city university system (CUNY); CUNY teachers have not had a raise in six years.

Speeches from Karen Magee, the leader of NYSUT as well as Randi Weingarten, the AFT president and a rousing in-person speech by candidate Hillary Clinton (Watch and listen to Clinton’s speech here). The most vigorous debate: teacher evaluation. Although the state is in the first year of a four year moratorium on the use of student test scores to assess teacher performance the debate was hot and heavy.

Watch videos of convention speeches: http://www.nysut.org/resources/special-resources-sites/representative-assembly

From the NYSUT website,

“Sending a strong message to Albany that more needs to be done to stop the harmful over-testing of students, some 2,000 delegates approved resolutions calling for a complete overhaul of the state’s grades 3–8 testing program; swift implementation of the Common Core Task Force’s recommendations; and new assessments that are created with true educator input to provide timely and accurate appraisals of student learning.”.

A few days after the NYSUT convention the UFT Delegate Assembly held its monthly meeting; a thousand or so delegates, elected in each school by staffs, meeting to listen to a report by UFT President Mulgrew and debate and set policy.  Mulgrew gives updates on the national scene: retired teachers were ringing doorbells in Pennsylvania supporting Hillary; the California Supreme Court reversed the lower court decision in the Vergara case, supporting tenure and reminded delegates that while the US Supreme Court deadlocked on the anti-union Fredericks decision attacks from the right will not end. Mulgrew criticized the use of test scores to rate teachers; however, he reminded teachers that under the Bloomberg administration principals had the sole voice in teacher assessment. In the last year of Bloomberg 2.7% of teachers received “unsatisfactory” ratings, under the current multiple measures system only 1% of teachers received “ineffective” ratings. Almost all schools in New York City use Measures of Student Learnings (MOSL), dense  algorithms that assess student growth attributed to each teacher – there are hundreds of algorithms  to account for the many different school situations. The system, that includes an appeal process, melds principal observations and MOSLs appears to work well.

At the first meeting of the Board of Regents under the leadership of new Chancellor Betty Rosa a lengthy discussion over teacher evaluation took place. Chancellor Rosa appointed Regent Johnson to chair a Work Group to link research to policy decisions.

The 700 school districts in New York State are currently negotiating teacher evaluation plans under the four year moratorium, the use of grade 3-8 test scores are prohibited.  A few members of the Board suggested asking the legislature to clarify exactly what they wanted the Regents to do in reference to teacher evaluation, others argued that the decision was given to the Regents members and it would be wrong to punt back to the legislature. Clearly, the newly constituted Board has a ways to go to reach consensus.

Even Charlotte Danielson, the doyen of teacher assessment has her doubts about the current policies across the nation,

The idea of tracking teacher accountability started with the best of intentions and a well-accepted understanding about the critical role teachers play in promoting student learning. The focus on teacher accountability has been rooted in the belief that every child deserves no less than good teaching to realize his or her potential.

But as clear, compelling, and noncontroversial as these fundamental ideas were, the assurance of great teaching for every student has proved exceedingly difficult to capture in either policy or practice…

There is also little consensus on how the profession should define “good teaching.” Many state systems require districts to evaluate teachers on the learning gains of their students. These policies have been implemented despite the objections from many in the measurement community regarding the limitations of available tests and the challenge of accurately attributing student learning to individual teachers.

I strongly urge you to read the entire Danielson essay: http://www.edweek.org/ew/articles/2016/04/20/charlotte-danielson-on-rethinking-teacher-evaluation.html

There are a few schools that have created teacher assessment processes that are valuable because they both assess teaching and encourage teachers to grow in their profession. There is insightful research; unfortunately we do not know how to “scale up.”  There is no inter-rater reliability, in some districts every teacher received a “highly effective” score, which also means so did the rater. Teachers in high wealth, high achieving districts receive higher scores than teachers in low wealth lower achieving districts. (Check out research studies  from the Chicago Consortium on School Research here  and here).

Getting teacher evaluation/assessment right is exceedingly complex in a highly emotional climate.

The Regents have a challenging task.