Tag Archives: Board of Regents

Should New York State End Regents Exams? Can Authentic Assessments Replace the Regents? Or, Will We Diminish the Value of a Diploma?

If you meet anyone who went to high school in New York State I’m sure they’ll remember Regents tests; they’ve been around since the 1870’s.  The Regents were intended for college-bound students; most students left high school and moved onto jobs that allowed them to live a middle class life; jobs, good jobs, were plentiful, commonly union jobs with fair pay and benefits.

In the high achieving school in which I taught only a quarter of students bothered to earn a Regents diploma, three-quarters of the kids earned a local diploma, the requirement, the 9th grade level Regents Competency Test, the RCT, and the accompanying diploma referred to as the RCT diploma. Today we would call the system multiple pathways.

By the mid-nineties the world of work had changed, a college degree was essential for a job. After a few years of discussion the Board of Regents moved to a single Regents diploma system, the RCT diploma was phased out. The plan, originally scheduled to take five years took a decade.

John King was appointed state commissioner,  the state won a  $700 million Race to the Top grant, and, adopted the Common Core State Standards.

Failure rates on the Common Core Algebra 1 Regents increased and the state decided to “scale’ the scores; currently students can receive a passing grade with fewer than half correct answers The state plan was to increase the number of correct answers to achieve a passing grade over time; it hasn’t been happening.

Unless student grades on the Algebra 1 exam increase graduation rates may be impacted, See “Rough Calculations: Will the Common Algebra 1 Regents Exam Threaten NYC’s Graduation Rates? (2015).

If you haven’t seen Regents exams recently look at the Global Studies here and the English here.

Click and try the Regents  ….  How’d you do?

The June, 2016 New York State rate graduation rate was 80%, the glass half full, the graduation rates keep edging up, the glass half empty, one in five kids fails to graduate in four years; six percent have dropped out and twelve percent are still registered in school. Although more kids are graduating more kids are not prepared for college and must take remedial courses in college.

The Board of Regents have been creating additional pathways to graduation,  4 + 1, CDOS, the “safety net” for students with disabilities, the re-scoring option, all part of multiples pathways to graduation options .

The members of the board and the commissioner are beginning to ask whether the emphasis on passing examinations is the best measurement of college and career readiness.

At the October Regents Meeting the members began to explore a move away from Regents exams. The commissioner set forth “potential goals,”

  • Prepare students for 21st century post secondary options, for example, Baccalaureate :programs in STEM, Humanities and Arts, Technical degree programs, Career training certificate programs, Adult education programs leading to certifications, Military service, Employment
  • Offer more flexibility in completing credit requirements, relevant pathway choice and student interest
  • Expand external certification assessment options
  • Allow students to demonstrate proficiency in multiple ways.

And the commissioner when on to list questions: called “Key Considerations”

  • How do we ensure that all students including students with disabilities and English language learners are able to access rigorous coursework?
  • Should students have the opportunity to demonstrate proficiency in a specific area of graduation through a district designed Capstone project?

 The commissioner could appoint a “blue ribbon” commission, experts, who could review the literature, ask for public input and submit recommendations, or, appoint a regents work group who would work with state education staff to draft a plan.

New York State is one of only seven states that requires exit exams, on the other hand critics defend regents exams; every school should meet the same standards, the same exams. The NY Post, the Manhattan Institute and others on the conservative side might accuse the commissioner and the chancellor of eroding the quality of a diploma.

On the other hand the opt-out parents would applaud, one in five students in the state opts-out of state tests and on Long Island more than half of families opt-out. Opting out of regents exams is not an option.

Daniel Koretz, a leading expert on testing has soured on the emphasis on test-based accountability.

High-stakes tests. Lots of them. And that has become a major problem. Daniel Koretz, one of the nation’s foremost experts on educational testing, argues in The Testing Charade that the whole idea of test-based accountability has failed—it has increasingly become an end in itself, harming students and corrupting the very ideals of teaching. 

Are alternative methods of measuring accountability, such as a portfolio of student work, a viable alternative?

The state of Vermont tried to move to a portfolio system which it abandoned; rater reliability was poor.

 A report analyzing Vermont’s pioneering assessment system has found severe problems with it and raised serious questions about alternative forms of assessment.

The Vermont system, which is being closely watched by educators around the country, is the first statewide assessment program to measure student achievement in part on the basis of portfolios.

 But the report by the RAND Corporation … found that the “rater reliability” in scoring the portfolios–the extent to which scorers agreed about the quality of a student’s work–was very low …

 … the report’s author, said the low levels of reliability indicate that the scores are essentially meaningless, since a different set of raters could come up with a completely different set of scores.

“If you’re not rating reliably, you’re not rating,” he said. “You can’t measure anything unless you measure it reliably.”

 Can the state move backwards, to a dual testing, dual diploma system aimed at improving graduation rates for students with disabilities and English language learners?

The state ESSA plan does not include this option.

The commissioner did endorse district-based Capstone projects.

Capstone projects are an excellent example of authentic assessment; at the college level a project might require an entire term to prepare.

The following comes from a partial description of the requirements of a college Capstone project

Capstone Expectations:

The capstone marks the culmination of the student’s studies. Accordingly, the topic selected should require application of a broad range of the skills and knowledge … The final paper must reflect thorough research, analysis, critical thinking and clear writing.

Capstone Content:

  • The topic students choose must be one they develop and work on independently.
  • The paper must showcase a deep understanding of an area….
  • The finished capstone must be a minimum of xx pages and include: an abstract; a background statement; a literature review; objectives; an analysis of existing research; an original analysis of the … challenges; opportunities, threats and possible solutions, critical and thoughtful conclusions; along with a bibliography, charts and any necessary illustrations.
  • The paper may contain primary research, ….Alternatively and more commonly, students may write their paper based on an analysis of secondary research. This approach may include a secondary data analysis or other specified metrics plan.
  • All secondary research must be attributed throughout the paper and in the bibliography.

This is a significant project: the commissioner suggests a “district-designed Capstone project,” how can we assure rater reliability in 770 school districts?

The commissioner and the regents are beginning a long journey with no clear outcome. Students pass courses and fail regents exams: should the failure prevent a student from graduating?  Should one three-hour exam determine graduation? On the other hand bar exams determine who becomes a lawyer; civil service exams determine who becomes a police officer or fire fighter.

I look forward to a deep discussion with experts and public participation and, I would recommend that the state hold hearings around the state.

Are we too wedded to Regents tests?

Are we jumping on a reform wave which may diminish a diploma?

Can/should we change the nature of instruction from the current modality to an authentic, project-based educational modality?

What do you think?

Advertisements

“Evening the Playing Field:” Will the ESSA Accountability Plan Acknowledge the Work of Teachers in “Truly Disadvantaged” Schools? Will the plan be “Equitable?”

A teacher: “We love our kids and love to teach in this school, we make a difference in the lives of our kids. Our kids are poor, really poor, some live in shelters, others in foster care, trauma is part of their daily lives. We’re building out our community school, we prepare our kids to learn by feeding them, by searching for contributions of clothes, and, make our classes as rich as possible. I wanted to take my class on a series of trips out of the neighborhood, my principal said wait till after the tests, the future of our school depends on six days of testing, it’s sad that no one cares about the social and emotional needs of our kids, needs that precede the ability to learn. We just want an even playing field.”

In some school districts kids come into school knowing their letter and number facts, in other schools it’s  their first exposure; kids are behind from day 1.

In “The Truly Disadvantaged: The Inner City, the Underclass, and Public Policy (1987)” sociologist William Julius Wilson details how programs across the political spectrum have failed the underclass, he is sharply critical of conservative and liberal policies.

Wilson posits social isolation–a distinction which shifts the problem from the psychological to the socio-economic realm. Instead of blaming poverty and its associated pathologies primarily on the individual, as conservatives do, or on the effects of contemporary racism, as some liberal scholars do, Wilson calls for a “refocused liberal perspective” which emphasizes “the dynamic interplay between ghetto-specific cultural characteristics and social and economic opportunities.”

Thirty years later our society, and especially our schools are even more segregated.

The result is islands of poverty, what Wilson called the “truly disadvantaged,” whose children enter school far behind other children.  The Harvard Education Letter writes,

  • According to a seminal 1995 study by Betty Hart and Todd Risley, 3-year-olds whose parents are professionals have vocabularies that are 50 percent larger than those of children from working-class families, and twice as large as children whose families receive welfare.
  • “We could eliminate at least half, and probably more, of the black-white test score gap at the end of twelfth grade by eliminating the differences that exist before children enter first grade.”
  • In a 2002 study, Valerie E. Lee and David T. Burkam of the University of Michigan found that at kindergarten entry, cognitive scores of children in the highest socioeconomic group were 60 percent higher than those of the lowest group.

From the 1965 Elementary and Secondary Education Act to the 2002 No Child Left Behind Law up to our current iteration, the Every Student Succeeds Act administrations across the political landscape have tried to legislate equity. Previous leadership of education in New York State jumped on Common Core State Standards and testing: the unintended consequence, a massive opt-out movement primarily among middle class white parents.

The current Board of Regents, led by Chancellor Betty Rosa is a newer board. Former superintendents and teachers, a judge, a doctor, an attorney. a parent activist, a range that crosses the spectrum of experience and background.  The board is attempting to create, as required by the ESSA law, a new school accountability system that goes beyond test scores. The new law requires grades 3 – 8 tests in ELA and Math but does not limit the accountability system to test scores.

The process of building a new accountability system has been transparent. Scores of open, facilitated meetings, a think tank made up of stakeholders and hours upon hours of open meetings with national experts Linda-Darling Hammond, Learning Policy Institute and Scott Marion, the Center for Assessment.

Your correspondent has sat through them all.

This past Tuesday a three hour plus session with Darling-Hammond and Marion tele-communicating with the board. It was crunch time, decisions had to be made.

You can check out nine presentations here.

Or, check on the individual power points/reports below:

Status of Development of ESSA Plan

Promoting Diversity – Integration in New York State

Considerations for the New York State Assessment System

Models for State Performance Assessment Systems

Building an Accountability and Assessment system Under ESSA

Putting It All Together – Annual Differentiation Under ESSA

I know parents and teachers ask: why do we need state tests? The answer is simple: the law requires that each state include state tests in any school accountability system and the system must identify the bottom 5% of Title 1 schools.

If you’re going to go beyond test scores what would you add?

You can add other indicators: for example, growth, the increase in scores from year to year, you could add counting subgroups differently, you could add chronic absenteeism, maybe other items. Additionally you can weight the indicators: for example ELA and Math scores could count 30% each, growth can count 20%, other items 5% or 10% resulting in a cumulative score.

Think in terms of a dashboard with the indicators across the top and the levels down the vertical column. Do you use satisfactory, unsatisfactory, levels 1 to 4, ineffective, developing, effective highly effective? Do schools receive an overall numerical score? a letter grade? How do you identify the bottom 5% of Title 1 schools?

One of regents members asked if there was a “scientific” method of making these determinations. Scott Marion explained these were subjective decisions, capturing stakeholder values, for weighting indicators is a subjective decision. Marion explained that subjective is not a negative; Marion discussed “credible  defensibility,” making value-based decisions that reflected the experiences of the members..

After extensive discussion the members agreed on a weighted dashboard. The first draft will be released at the May regents meeting, public comments, meetings around the state, submitted to the governor and submission to the feds in September.

Colorado has completed and submitted a dashboard plan that goes beyond test scores: read a description of the plan here.

Hopefully, maybe, the changing of the metrics will “even the playing field” for the “truly disadvantaged.”

The next steps will be to begin to explore alternative assessments, aka authentic assessments. Vermont and New Hampshire involved in pilots; however, the path is long and complex, and to quote Robert Frost,

The Road Not Taken

Two roads diverged in a yellow wood,

And sorry I could not travel both

And be one traveler, long I stood

And looked down one as far as I could

To where it bent in the undergrowth;

 

Then took the other, as just as fair,

And having perhaps the better claim,

Because it was grassy and wanted wear;

Though as for that the passing there

Had worn them really about the same,

 

And both that morning equally lay

In leaves no step had trodden black.

Oh, I kept the first for another day!

Yet knowing how way leads on to way,

I doubted if I should ever come back.

 

I shall be telling this with a sigh

Somewhere ages and ages hence:

Two roads diverged in a wood, and I—

I took the one less traveled by,

And that has made all the difference.

The New, Feisty Board of Regents Explores Principal Preparation: Why Don’t We Have Better Principals?

[Election Update: Yuh Line Niou won the six-way primary in Shelly Silver’s former district as well as all other Ed in the Apple endorsed candidates with the exception of Robert Jackson; however, the Bloomberg/Charter candidate, Micah Lasher lost to a candidate supported by the Independent Democratic Coalition – the breakaway gang of five that caucuses with the Republicans]

The new Board of Regents is a feisty group!!

The Board is a policy board; they hire the CEO, the commissioner, and set overall policy for the state. The line between what is policy and what are operations is a blurred line: a prime example.

In December the Regents voted to accept the 21 recommendations of the Cuomo Task Force on the Common Core.

Recommendation 15: Undertake a formal review to determine whether to transition to untimed tests for existing and new State standardized test aligned to the standards.

A month later the Department announced a shift to untimed tests;  the “formal review” apparently did not involve the Board.

Initially the Commissioner was ecstatic over the unparalleled one year jump in test scores, until the Chancellor, Betty Rosa tuned down the exuberance.  Without knowing which students took extended time the state has set a new baseline, there can be no valid comparisons – you cannot compare apples to oranges. The Regents members were clearly unhappy – why weren’t they involved in the “formal review?”

Under the leadership of Chancellor Tisch and John King, with a few exceptions, the Board was quiescent.

The current members are activists, in order to create policy they clearly intend to take a deep dive into the issue. A prime example: the four exams required for teacher certification. The co-chairs of the Higher Education Committee have held forums all over the state, hundreds of college staff, and degree seekers, have attended and testified. The Board is leading the steps to reconfigure the teacher preparation process that was imposed by Tisch/King.

No longer does the Chancellor and the Commissioner run the show. Chancellor Rosa epitomizes collaborating with her Regent partners.

The September 12th Regents Meeting began with a detailed exploration of a new grant from the Wallace Foundation:  the Principal Preparation Project. In prior years the project would have landed with the Regents Research Fellows with a nary a word of discussion with the Regents members. The world has changed.

After a Power Point presentation the new Board peppered the Deputy Commissioner with questions;

Regent Johnson mused over the purpose of the project.  We must acknowledge the impact of poverty, issues of race and changing demographics. Why weren’t Civil Rights organizations on the team? Regent Mead was concerned over the three years of teaching as a minimum requirement – New York City has a seven year requirement. Regent Norwood was wondering why social/emotional issues appeared absent from the project as well as working in diverse environments, and, the retention of leaders in low performing schools were absent. Regent Brown was concerned with the absence of diversity concerns in the project, should issues of race, i. e., “white privilege” and “cultural competency,” be included in project curriculum?

The discussion went on and on….

In order to become a principal in New York State the applicant must complete an “approved” program; however, the selection is by the elected lay school board, or, in New York City, by the Chancellor; all the state does is create an applicant pool.

A little history:

The first wave of reform swept the nation after the Civil War and culminated in the passage of the Pendleton Act in 1883 – establishing a federal civil service system. The reform movement moved to the states, and, after the creation of New York City (“The Great Consolidation”), the merging of the five boroughs, the legislature moved to reform a political hiring system, by creating a Board of Examiners.

Read a history of principal selection here: https://mets2006.wordpress.com/2008/08/07/the-quest-for-the-leadership-gene-how-do-we-findselect-the-best-school-leaders/

From rigorous examinations to a handful of credits and selection by elected Community School Boards to the Leadership Academy, we haven’t found any magic bullets.

Half-jokingly, I mused that maybe there was a leadership gene. Maybe I’m right!

… a quarter of the observed variation in leadership behaviour between individuals can be explained by genes passed down from their parents. – See more at: https://www.ucl.ac.uk/news/news-articles/0113/15012013-leadership-genetics#sthash.Nmnip8lR.dpuf

If you ask teachers about supervisor competence you will find a wide variability, some praise school leaders, many more are critical.  An NYU Study a few years ago, using student scores on state exams as a measurement: insignificant differences between Leadership Academy and non-Leadership Academy principals.

I have a few questions:

* What percentages of applicants are accepted into leadership programs? Is the quality of the applicant’s teaching part of the applicant selection process, and, if so, how do you measure the quality? (I fear programs accept the vast percentage of applicants)

* Are online or blended learning courses acceptable? Are these courses of the same quality as face-to-face courses?

* How often does the supervising teacher visit the candidate? Four times a year? Weekly? What is the quality of the internship? How is it measured?

* What percentage of candidates find jobs within five years? How successful are the candidates as supervisors and how do we measure success?

The finest leadership I have seen is the leadership provided by coaches, whether athletic, music or dance.

The ultimate question: is this project worthwhile?  Since the state does not hire or supervise principals can changing the requirements actually change who gets hired?  Do we have to change the “hirers” before we can change the “hirees”?

Looking ahead: every state must comply with the new Every Student Succeeds Act (ESSA) and design a state plan -more about the process in my next post.

The Politicization of State Tests: Creating Tests in Which “All Students in New York State Are Above Average”

When the dust cleared the greatest ally to the anti-testing clique was (roll of drums!!!)  MaryEllen Elia, the New York State Commissioner of Education.

The deeply flawed state tests (“All children are above average”) reignited the argument – why do we have state test at all (aside from the federal requirements)?

Statewide ELA test scores jumped by around 7% – although the racial achievement gap remained the same.

A magic potion, incompetence or simply political legerdemain?

A little review: in September, 2015 Governor Cuomo reconvened a blue ribbon panel, actually a process to repair the Governor’s foolhardy attacks on teachers and parents. In 2014 it appeared that Cuomo had a clear path the Democratic nomination for his second term and deep pockets for the November general election. Seemingly out of nowhere Zephyr Teachout, a law professor at Fordham challenged Cuomo for the Working Families Party spot on the ballot and challenged Cuomo in the Democratic primary. While the teacher union made no endorsement some members and locals were on the Teachout side. After defeating Teachout and Rob Astorino, his Republican challenger Cuomo decided to punish teachers. He cozied up to the charter school folks, used the budgeting process to tack on legislation to extend teacher probation, and, was nastier than usual.  NYSUT, the statewide teacher union responded with a series of aggressive TV ads and the opt-out movement was created, 20% of kids opted-out of the 2015 state tests.

Cuomo’s popularity rating tumbled.

I suspect clearer heads prevailed.

The purpose of the Task Force was to guide education policy from afar and place the Board of Regents and the commissioner in the foreground. The recommendations were more than recommendations; they were a pathway for state education policy. (Cuomo: This is the endgame – you figure us out how to get us there)

The Task Force Report (Read here), which was released in December, contained twenty-one recommendations, the last recommendation was a moratorium on the use of state tests to evaluate principals and teachers for four years, applauded by the teacher union.  The recommendations called for a thorough review of the Common Core Standards and teachers would be included in every step of the process.

Recommendation 15: Undertake a formal review to determine whether to transition to untimed tests for existing and new Standardized tests aligned to the standards; not controversial, garnered little,  if any discussion; perhaps a pilot in a few schools and school districts across the state.

Surprisingly, very surprisingly, without any discussion with the Board of Regents, the Commissioner announced that the 2016 state tests would be untimed.

The January announcement, entitled “Changes for the 2016 Grades 3-8 ELA and Mathematics Tests” begins,

This memo outlines changes made as a result of feedback from the field:

* Greater involvement of educators in the test development process

* Decrease in the number of test questions, and

* A shift to untimed testing

The announcement came from Angela Infante, Deputy Commissioner, Office of Instructional Support and Peter Swerdewski, Assistant Commissioner, Office of State Assessment.

The state document states, “…students will be provided with as much time as they need.” No pilot, no transition, jumping off the diving board into the pool, and, the state made no attempt to identify students who took additional time.

The scores soared, the state commissioner, in the Daily News admits the scores are “not exactly a perfect comparison,”

After widespread opposition to the difficulty of the tests erupted in 2015, state education department officials shortened the exams for 2016 and eliminated time limits.

“Because of the changes in testing, it’s not exactly a perfect comparison,” Elia said. “And even with the increases this year, there remains much work to be done.”

The state spent many millions of dollars purchasing tests, teachers and students months of test prep, to collect data from what turns out to be a non-standardized test. A test that might not even meet federal requirements, although I’m sure the feds will simply ignore the faux jump in scores.

Was the test itself “harder” or “easier;” many months down the road a Technical Advisory Committee (TAC) will release a report, hundreds of pages of dense analysis that few will read and fewer will understand.

The basic questions: are the results of the test useful?  Can they be compared with the previous year? Can schools and school districts be compared? And, at the top of the list: are the schools in New York State making academic progress?

Howard Wainer, a Distinguished Research Scientist, the author of innumerable books and articles, an internationally recognized expert writes,

Because of the changes this year’s scores can’t be compared to last year’s and because of the untimed nature of the test (and there being no record of how long anyone took) you can’t compare scores of students who took it this year with one another. It is, in no uncertain terms, an unstandardized test.

This test is akin to measuring children’s heights but allowing some students, we don’t know who, to stand on a stool, we don’t know how high, and then declaring some taller than others.

Fred Smith, another testing expert, writing in City Limits, had doubts about the validity of the test before the test administration.

Either the state education psychometrician is lacking in competence, or knew by adopting untimed tests scores would likely jump – either is unacceptable.

If the state continues down the same path, retaining the untimed tests, even if it keeps track of students who take extra time, and the amount of extra time, we will be once again be comparing apples to oranges. Kids who take extra time or choose not to take extra time may not be the same kids as this year – we simply can’t know.

Will states across the nation also jump on the untimed tests bandwagon?

In the politicized world of education the charter school folk and their acolytes beamed at higher scores, of course, we have no way of knowing why charter school scores were generally higher than public schools, and, the pro-charter print media crowed. Mayor de Blasio and Chancellor Farina also took a victory lap, and the Mayor immediately claimed the scores were proof that mayoral control be made permanent.

Board of Regents Chancellor Rosa reminded us it’s not time for a victory lap, unfortunately everyone else is milking the results – de Blasio and Farina, the charters and principals and teachers are breathing a sigh of relief.

A perverse kind of victimless crime: except for the kids who were tortured preparing for a non-standardized test.

Although the law has changed, No Child Left Behind has been replaced by the Every Student Succeeds Act; the requirement for annual testing in grades 3-8 remains. The Leadership Conference is an umbrella group representing the major civil rights organizations across the spectrum has strongly supported the accountability requirements, aka, testing and reporting scores by subgroup, and, the law is not changing.

Testing is here to stay.

The US Department of Education has announced they will be selecting six or so states or consortiums of states to play with alternate assessments.

The anti-testing crowd points to the new law and the testing kerfuffle in New York State, why not move to portfolios and performance tasks to current replace testing? This is not a new idea.

Vermont spent a decade working to create an assessment system based on portfolios, and after an external report pointed to fatal flaws, abandoned the effort.

…report by the RAND Corporation … found that the “rater reliability” in scoring the portfolios–the extent to which scorers agreed about the quality of a student’s work–was very low. The researchers urged the state to release the assessment results only at the state level.

Daniel M. Koretz, a senior social scientist at RAND and the report’s author, said the low levels of reliability indicate that the scores are essentially meaningless, since a different set of raters could come up with a completely different set of scores.

Can thousands of teachers be expected to rate portfolios the same?

The portfolio process was expensive, extremely time consuming  and there is no guarantee the portfolio work was not “assisted” by parents or others .

Yes, portfolios and performance tasks are effective classroom tools and in the perfect world might be a way of assessing student progress, in the real world, the world in which we live, it is not reasonable to expect inter-rater reliability.

The anti-testing movement will not disappear and the opt-out movement is alive.

What is absent is leadership – Arne Duncan drove us down a path for seven years that divided education: reformers versus deformers, marketeers versus public schools, unions versus the hedge funders: education is bitterly divided. Will the next president nominate an education leader who can bring together the disparate constituencies?

Education is adrift and the unstandardized testing regimen in New York State is a prime example.

Why Did New York State Test Scores Jump? Better Instruction? Untimed Tests? All the Kids Got Smarter, or, Shenanigans?

If you want to bury a news story you issue the press release on a Friday afternoon, if you want as much mileage as possible you issue the release on a Tuesday morning, followed by a press conference, in person and online, followed by laudatory speeches across the state and try to maximize the time the story garners headlines and clicks.

The State Education Department released the 2016 grades 3-8 ELA and Math scores on Friday afternoon with an odd presser. The test scores up, way up; why is the SED ashamed?

You can take a deep dive into the New York City Scores here: http://schools.nyc.gov/Accountability/data/TestResults/ELAandMathTestResults

The SED analysis of the state scores with many disaggregated charts here and here.

The Commissioner was careful not to publicly laud the increase in the scores,

But rather than celebrate the largest bump since New York adopted new tests tied to the Common Core Learning Standards, education officials reported the increases with caution. They suggested that changes in how the tests were given – not actual improvement by schools and students – may have accounted for the gains.

State Education Commissioner MaryEllen Elia also warned against making comparisons with previous years, which is typically done to evaluate schools and teachers.

“It’s not an apples to apples comparison and should be viewed in that context,” Elia said during a news conference when the results were released Friday.

For the data wonks who want to parse the results check out the files here and here.

The SED states, “…  changes in how the tests were given – not actual improvement by schools and students – may have accounted for the gains;” however, a deeper analysis is necessary.

If the increases are due to fewer questions and untimed tests, we should know, if both teachers and kids have been exposed to the more effective Common Core instruction and better professional development, we should know, or, if the SED, as some suspect, manipulated the process, we should know. All of the kids in New York State getting smarter just doesn’t seem creditable.

Under Commissioner Mills test scores increased year after year, when Chancellor Tisch and new Commissioner Steiner took over they asked a Harvard professor, Daniel Koretz to take a look – sure enough – the SED had been using many of the same questions year after year. Whether incompetence, or, more likely a method of increasing scores, we’ll never know. Scandals in Atlanta and accusations elsewhere have cast doubt on the entire testing regimen. Jumps in test scores are treated with skepticism.

For years Howard T. Everson chaired the Regents Technical Advisory Committee (TAC) and was sharply critical of test score inflation.

But given all the flaws of the test, said Prof. Howard T. Everson of the City University of New York’s Center for Advanced Study in Education, it is hard to tell what those rising scores really meant.

“Teachers began to know what was going to be on the tests,” said Professor Everson, who was a member of a state testing advisory panel and who warned the state in 2008 that it might have a problem with score inflation. “Then you have to wonder, and folks like me wonder, is that real learning or not?”

Each year after the release of the state tests scores the TAC issued a lengthy analysis of the quality of the test. Recently the TAC process has changed, as I understand the current process the TAC report goes to the test creator, Pearson, (now replaced by Questar) who vets the report, over the last few years the report was released a year after the test and was so heavily “massaged” it was meaningless.

The SED/Regents should, in the footsteps of Tisch and Steiner, immediately ask Everson or Koretz or a colleague with equally impeccable credentials to examine the current state test results.

If, in fact, the Commissioner doesn’t know why scores jumped we have to ask: why not?  If untimed tests resulted in higher test scores shouldn’t Regents Exams be untimed?  If the increased exposure to better Common Core instruction resulted in higher scores why are the Algebra 1 and Geometry scores not increasing?

Shrugging and simply saying we’re happy with increased scores but we’re clueless as to why is simply not acceptable. Data should influence policy at all levels, and, we have to be confident that the testing regimen is creditable.

The New York State Legislature Adjourns with a “Whimper,”as Educational Policy-Making Moves to the Board of Regents

This is the way the world ends
This is the way the world ends
This is the way the world ends
Not with a bang but a whimper

T. S. Eliot, The Hollow Men (1925)

The last stanza of Eliot’s poem is an apt description of the end of the 2016 legislative session. The final days, called “the Big Ugly,” is a scramble, an endgame, the Republicans and the Democrats vying for an advantage as the state moves toward the November election. All the seats in the legislature, the 150 in the Assembly and the 63 in the Senate will be on the ballot. While the Assembly is firmly in the Democratic column the Senate is far more complex, and byzantine. The Democrats hold a single seat edge in the Senate (32-31); however five Democrats (Jeff Klein, Diane Savino, Tony Avella, David Valesky, and David Carlucci), the Independent Democrat Conference (IDF), under the leadership of Klein (Bronx) caucuses with the Republicans, giving the Republicans control of the Senate.

Hanging in the balance were mayoral control, campaign finance reform, removal of pensions for convicted legislators, online fantasy sports betting and scores of other bills.

You may ask: why is all this conflict and wheeling and dealing necessary? Why can’t legislators have civil conversations and decide the issues?

James Madison, in Federalist # 51 wrote,

Ambition must be made to counteract ambition. The interest of the man must be connected with the constitutional rights of the place. It may be a reflection on human nature, that such devices should be necessary to control the abuses of government. But what is government itself, but the greatest of all reflections on human nature? If men were angels, no government would be necessary. If angels were to govern men, neither external nor internal controls on government would be necessary

The Constitutional Convention (1787) was not covered in CSPAN; the Constitutional Convention was a secret meeting. The only notes we have are Madison’s personal notes, not made public until after the death of all the delegates, The fifty-three delegates argued, came and went, delivered lengthy speeches, met in private, and made deals.

Slavery was one of the most significant stumbling blocks, the anti-slavery Northerners versus the slave-holding South, The compromise: slavery is not mentioned in the constitution, the question of slavery was left to the states, and, as part of a compromise; slaves were counted as 3/5th of a ”free person,” and referred to in the clause as “all other Persons.”

Representatives and direct taxes shall be apportioned among the several states which may be included within this union, according to their respective numbers, which shall be determined by adding to the whole number of free persons, including those bound to service for a term of years, and excluding Indians not taxed, three fifths of all other Persons.

Deal-making, as reprehensible as it may seem, is at the essence of making government work.

Whether to extend mayoral control in New York City had nothing to do with education. Weakening the mayor might give the Republicans a chance in the 2017 mayoral election. In spite of pleas from Merryl Tisch and others in the upper echelons of power Senate leader John Flanagan offered “unacceptable” plan after plan until in the closing hours an agreement was reached, the NY Times describes the plan as a one year extension plus,,

It would effectively create a parallel system of charter schools within the city, allowing “high-performing charter schools in good standing” to switch to join the State University of New York umbrella or the Board of Regents of the State Education Department.

Probably a meaningless change, currently charters schools authorized by both New York City and Buffalo make reauthorization proposals after five years, the authorizer, SUNY or the Board of Regents can reject the recommendation. The proposal allows the charter school, if it’s  “high performing and in good standing” to move directly to SUNY or the Regents for reauthorization.

The session is most interesting for what it did not do – the houses steered clear of legislation directing the State Education Department to take any actions. A host of education bills simply died. Neither the governor nor the party leaders had any desire to once again get involved in the morass of teacher accountability or testing, any of the issues that birthed the opt outs and/or angered teachers and their unions.

The budget was generous and the political leaders appear to be leaving the educational decisions to the educational leaders.

In December the Cuomo-appointed Task Force released their report with 21-recommendations: a blueprint for the Commissioner and the Board of Regents. The core of the report was a 4-year moratorium on the use of student test scores as part of a metric to assess teacher performance.

In the six months since the release of the report the Commissioner has made tests untimed, a recommendation in the report, established a number of large field-based committees to review elements of the Common Core, and, the Regents created a number of alternative pathways to graduation.

Quietly, very quietly, the Commissioner announced a change in the observation section of the teacher evaluation regulation. The outside observer would be scrapped – what might be a good idea in theory was both overly complex and a financial burden on school districts. There was no high drama – no headlines, simply an announcement undoubtedly based on quiet discussions.

The decisions before the Board of Regents are complex, politically explosive and without explicit answers.

Can you create a teacher evaluation plan that is acceptable to principals and teachers and not trashed by external critics?

Can better tests win back opt out parents?  And, what do you mean by “better tests?”

Will alternatives to testing, perhaps, portfolios or other performance assessments, be acceptable to the feds, and acceptable to the principals and teachers?  Are performance assessments practicable in actual classroom settings?

Will additional alternative pathways to high school graduation make students more or less prepared for college?

The Regents appear to have a window – three or four years – to make decisions based on their expertise as well as respond to external pressures and scrutiny, and, hovering aloft: “disruptive” solutions such as unlimited charter schools or vouchers.

Windows open, and windows close.

Getting It Right: Building a Research-Based Teacher Assessment System

A couple of years ago I was participating in a Danielson Training Workshop, two Saturdays in a room filled with principals and network support folk. We watched a video of part of a lesson – we were told we were watching a first year teacher in November in a high school classroom.

Under the former Satisfactory/Unsatisfactory rating system the lesson was clearly satisfactory. The Danielson Frameworks (Read the 115-page NYSED document here) requires that teachers are rated on a four-point scale (Distinguished, Proficient, Basic and Unsatisfactory) while New York State also requires a four point scale (Highly Effective, Effective, Developing and Ineffective). The Frameworks divides the teaching process into four domains, 22 components and 76 elements.

The instructor asked us to rate the lesson: at my table we were all over the place. For a teacher in the third month of her first year of teaching the lesson was excellent – clearly “proficient.”  Others argued the time in teaching was irrelevant, you had to rate her against all other teachers regardless of experience – at best, she was “developing.” Inter-rater reliability was absent.

Decades ago the union sent me to an Educational Testing Service conference on teacher assessment; about thirty experienced superintendents from all over the Northeast, and me, one union guy. We began by watching three 15-minutes videos of lessons: one an “old-fashioned” classroom, the kids sitting in rows, the kids answered teacher questions, the kids stood when they answered; the questions were at a high level although a small number of kids dominated the discussion. In the other video kids were sitting at tables, the teacher asked a question, gave the kids a few minutes to “huddle,” and one of the kids answered for the group and the teacher followed up with a few clarifying questions, in the third classroom the kids were at stations around the room, it was noisy, the noise was the kids discussing the assignment, the teacher flitted around the room, answering, clarifying and asking questions.

We were asked to rate the lesson on a provided checklist.

The result: the superintendent ratings were all over the place.

I was serving as the teacher union rep on a Schools Under Registration Review (SURR) team – we were visiting a low performing school. We were told to wait, the principal was busy, four of the 50 teachers were absent and there were three vacancies, the principal was assigning classroom coverages.

At the initial get acquainted session a team member, considering the staffing issues asked, “What are the primary qualities you look for in assessing teacher quality?” The principal blurted, “They come every day and blood doesn’t run out from under the door.”

A colleague was touring a school with very high test scores.  As he walked the building with the principal, he saw uniformly “mediocre” instruction – teacher-dominated, no student engagement. He mentioned the low quality of instruction to the principal, who shrugged, “Why mess with success?”

Once again, there is no inter-rater reliability.

In a number of school districts across the state almost all teachers received maximum observation ratings.

The State Ed folk simply accept the observation ratings of principals and school districts.

Charlotte Danielson, in her other book, Talk About Teaching  (September, 2015), discusses the complex role of the principal as rater as well as staff developer: how can a principal, who is the summative evaluater honestly engage with teachers who they rate?

In an excellent article from the Center for Educator Compensation Reform, Measuring and Promoting Inter-Rater Agreement of Teacher and Principal Performance Ratings (February, 2012), the authors parse the reliability of teacher observation ratings. There are a number of statistical tools to assess reliability – the state uses none of them.

In New York State 60% of a teacher rating is made up of the teacher observation score, and, we have no idea of the accuracy of the rating.

In the pre-Race to the Top days, the Satisfactory/Unsatisfactory rating days, the entire rating was dependent on the observation – in the last year of Bloomberg term 2.7% of teachers in New York City received Unsatisfactory ratings, under the current far more complex system that incorporates student tests scores and other measures of student growth only 1% of teachers were rated ineffective (Read a description of the plan:  APPR 3012-c).

Under the newest  system the other 40% is a combination of Measures of Student Learning and Student Learning Objectives, the use of state test scores is suspended until the 2019-20 school year.

Read a detailed description of the current APPR 3012-d teacher evaluation law here and a lengthy Power Point here.

In May, 2015 the Regents convened a Learning Summit and asked a number of experts to discuss the use of student growth scores (VAM): Watch the lengthy, sometime contentious discussion  here.

With one exception the experts criticized the use of student growth scores (VAM), the VAM scores did not meet the tests of “validity,” “reliability” and “stability.”

There have been glaring errors in the system. In the Sheri Lederman law suit  a teacher had very high observation scores and due to the composition of her class, very low student growth scores. The judge ruled the use of the growth scores, in the individual case, was “arbitrary and capricious.”

The APPR plan negotiated in New York City, on the other hand, allows for appeals by a neutral third party, and, the “neutral” has overturned appeals in which there was a wide disparity between the observation and VAM scores.

The current plan, created by the governor and approved by the legislature has been rejected by teachers and parents. Teachers are convinced that their score is dependent on the ability of the students they teach, not their competence. Parents feel schools are forced to “teach to the test” due to the consequences facing principals and teachers.

Angry parents, angry teachers and principals and a governor and a legislature looking for a way out of the box they created.

And, a cynicism from elements among the public – if two-thirds of kids are “failing” state tests how is it possible that only one percent of principals and teachers are rated “ineffective?”

The Board of Regents has been tasked with finding the “right” plan.

There has been surprisingly little research and public discussion of teacher attrition – in high poverty schools staggering percentages of teachers, 30%, 40%, 50% or more leave within their first few years.

The December, 2015, Cuomo Commission Task Force, in a scathing report, tasked the Regents with “correcting” what has been a disastrous path. Partially the governor creating an incredibly complex teacher evaluation matrix and partially the Commissioner King rushing to adopt the common core, common core testing and teacher evaluation simultaneously.

Can the Regents separate political decisions from research-based and guided decisions? Can the Regents move from the John King path, an emotion-guided political path to actually following “what the research says”?

On Tuesday the new Research Work Group, chaired by Regent Johnson will convene for the first time.

The roadmap for the State Ed Department and the Board of Regents are the twenty-one recommendations of the Cuomo Common Core Task Force. A number of the recommendations: untimed testing, an in-depth review from the field of the standards, greater transparency of the test items, alternatives to the use of examinations for students with disabilities, and, the beginning of an review of teacher evaluation are already in progress.

The Commissioner and the Regents have to regain a lost credibility: from policy emanating from the Gates Foundation and the so-called reformers to policies guided by scholarship and supported by parents and educators.