Should Teachers of Immigrant Children Be Assessed Using the NYSESLAT Student Assessment Test?

The Supreme Court, in a landmark decision, Lau v Nichols (1974), school districts across the nation were directed to address the issue of the “inability to speak and understand the English language,” the court wrote,

Where inability to speak and understand the English language excludes national origin-minority group children from effective participation in the educational program offered by a school district, the district must take affirmative steps to rectify the language deficiency in order to open its instructional program to these students.

Any ability grouping or tracking system employed by the school system to deal with the special language skill needs of national origin-minority group children must be designed to meet such language skill needs as soon as possible and must not operate as an educational dead end or permanent track …

Simple justice requires that public funds, to which all taxpayers of all races contribute, not be spent in any fashion which encourages, entrenches, subsidizes, or results in racial discrimination.

After a few years hassling with advocates New York State created Part 154 of state regulations , a dense set of regulations setting forth identification procedures, number of minutes of specialized instruction per week and employing teachers with appropriate certification – Part 154 was a compliance document.

For decades educating English language learners was a New York City-only problem, and, the city closely monitored the schools across the city.

The “issue” was not without political disagreements. Were bilingual classes segregating and dead ends? Were English language learner (ESL) classes more effective than bilingual classes? Were bilingual classes essential to retain student cultural identity, or, were bilingual classes simply job opportunities for bilingual teachers? Politicians, advocates, school boards lined up on one side or another, researchers supported and opposed one approach or another, the arguments, at times, were vociferous, and, disagreements exist today.

Over the last decade new immigrants entered cities across the state, from the “Big Five” to mid-size cities on Long Island (Brentwood, Hempstead) as well as upstate. The education of new immigrants was no longer a New York City only issue.

The regulations, Part 154, were essentially unchanged for thirty years as the “practitioners” disagreed over how the regulations should be changed, finally, in 2014, the state revised the regulations. Among the changes: prohibiting “pull-out” models and requiring more bilingual classes, a model that requires hiring more teachers, as well as, a new Common Core aligned assessment test, a new NYSESLAT test. (NYSELAT = New York State English Language Learner Assessment Test). (Peruse links to the new NYSESLAT test and associated documents here and many resources for teachers from the Engage NY site here ).

The new Part 154 regulations present another political issue: the regs require the creation of bilingual classes and the state funding formula does not provide additional dollars. Last year as the new regs were rolled out a member of the Regents, reflecting the worries of school districts raised the question of the costs, another unfunded mandate, ex-Commissioner King responded, “The districts can cut Advanced Placement Classes …,” not exactly the most politic of replies.

In the fall of 2014 undocumented minors began to arrive, mostly on Long Island, and, some school districts were not assigning students to classes, claiming the students could not document their addresses, or lacked a birth certificate, or lacked a legal parent or guardian. Students were refused registration or assigned to sit in auditorium for weeks at a time. In December the Department issued emergency regulations directing school districts to admit the students,

o Clear and uniform requirements, which comply with federal and State laws and guidance on the enrollment of students, particularly for unaccompanied minors and undocumented youths;

o Enrollment requirements whereby districts must accept additional forms of proof beyond the highly restrictive forms listed in the enrollment instructions/materials of school districts under review to date.

Reluctantly school districts admitted the students, although in some instances the result was sharp increases in class size and a scramble to hire properly licensed bilingual teachers.

English language learners is a compliance word, all new students whose home language is not English must take the LAB-R within ten days of registration and all student identified are required to receive specialized instruction and are tested each spring using the NYSESLAT, students are tested each spring until they “score out” on the NYSESLAT or receive 3.0 or higher on the state ELA exam, or, pass the English Regents – the testing process is time -consuming, teachers are removed from their normal instructional duties to test each and every ELL student.

The new Part 154 also introduced the new Common Core-aligned NYSELSAT exam – the exam covers reading, writing, speaking and listening and is untimed; it can take hours to administer the test to each student. (Read an interesting critique of the exam here) and a superb scholarly article that is highly critical of the test here)

To further complicate the issue the term “English language learner” incorporates a wide range of students:

* Unaccompanied minors who are living with relatives or strangers and may face serious social and emotional issues.

* Students with interrupted education (SIFE), arrivals may have not been in school for years, or, perhaps, have never attended school in their native country.

* Students who are illiterate in their native language.

*Students whose native language is an indigenous language. (Students from Central America who do not speak or understand Spanish or students from West Africa who speak tribal languages)

* Students who suffered from traumatic conditions in their native country.

Unfortunately in most schools and school districts Part 154 simply means compliance, does the student have a properly licensed teacher, is the student receiving the appropriate minutes of specialized instruction per week. The instruction might be in a bilingual class, or, in an integrated classroom with two teachers, one of whom possess a bilingual, ESL license or extension, or in a self-contained ESL classroom. In their second year in the country students must take the state exams, regardless of their lack of English skills. The feds recently denied the state request to delay the exam for two years after admittance to the program.

Graduation data for ELLs is poor, and “college and career readiness” data very poor. Of course the state “judges” ELLs using the same metrics at all other students. After a student “scores out” of an ELL program school districts are required to provide “transitional” supports.

Some ELL students “score out” in a few years, others, called “ever-Ls” never score out. To expect that new immigrants will achieve at grade level after a year is ludicrous, researchers agree that the average student takes 5 -7 years to master English yet we “punish” schools and teachers.

See Graduation rate data by district and by school here.

Too many schools and school districts are crippled by limited budgets, internal bickering on school boards and gubernatorial politics that see charters and receivership as an “answer” to complex problems.

New immigrant students with limited English skills must take state exams; the result, lower scores for the schools and districts. The students are faced with the dual task of learning English and learning content.

The state intends to use the new NYSESLAT exam, which assesses student progress in learning English, to assess the teachers of these students. The decision ignores the wide diversity among students who are learning English. Emergent learners of English learn at different rates. A student who is illiterate in his/her native language will take a substantially longer period of time to learn English than a student who was in school at grade level in his native nation.

As Regent Brown stated at the last Regents Meeting, “We should not use test designed to assess students to assess teachers.”

To use the NYSESLAT to assess teachers, to “punish” teachers who choose to teach immigrant students will only discourage teachers from teaching the neediest students

Shouldn’t An Open Dialogue Have Preceded the Extension of Mayoral Control? Mayoral Control Without Public Input Is Autocratic Control.

In the final days and hours of the legislative session e-mails and faxes clogged the e-mailboxes of every legislator. The phones never stopped ringing and lobbyists prowled the halls of the legislative office building (LOB). For the Democrats in the Assembly extending and improving rent control laws and stopping the education tax credit and charter school expansion efforts. For the Republicans in the Senate extending the property tax cap, which was due to expire next year, an election year; and, for Mayor de Blasio, the extension of mayoral control which was due to expire today.

Geoff Decker at Chalkbeat reported on Senate leader Flangan’s problem with the extension of mayoral control,

As the legislature prepared to vote on a bill that includes a host of education and housing issues Thursday evening, including the mayoral control extension, Senator John Flanagan offered the most expansive explanation yet for why Republicans stood in the way of giving de Blasio more time. He personally backed mayoral control, he said, but the city’s spending on education had not received enough scrutiny.

Perhaps the reason is simpler; there was no voter, no constituent pressures, to include mayoral control beyond the minimum of one year. If mayoral control was well down the list for the Democrats why should the Republicans accede to the Mayor of New York City? If the third “man in the room,” the Governor, also seemed unconcerned, pushing mayoral control into next year was the easiest way to move it off the table.

While newspaper editorials and the elites all saw the extension as crucial absent was any grassroots support.

Even one of the Mayor’s strongest supporters, the leader of the NYC teacher union, only agreed with mayoral control with caveats.

In a rare break with Mayor de Blasio, United Federation of Teachers president Michael Mulgrew said Thursday he doesn’t support mayoral control of the schools in its current form.

Mulgrew, speaking after an event with the mayor in which they both touted career and technical education, said the union doesn’t think the mayor should have primary authority over the Panel for Educational Policy, which he does now.

But he said he likes “the idea of mayoral control.”

The 2003 mayoral control law (Read law here) has been extended twice with only minor tinkering.

The law established a thirteen member board,

Such board of education shall consist of thirteen appointed members: one member to be appointed by each borough president of the city of New York; and eight members to be appointed by the mayor of the city of New York. The chancellor shall serve as an ex-officio non-voting member of the city board. The city board shall elect its own chairperson from among its voting members. All thirteen appointed members shall serve at the pleasure of the appointing authority.

In other words the mayor appoints the majority of the board and can fire them as he sees fit. When two members of the Bloomberg-appointed Board voted against the chancellor early in his term he fired the members. The eight members appointed by the mayor have raised their hands regardless of the opposition in the audience, regardless of the issues; the hottest issues have been charter school co-locations. Ironically, a few weeks before the Albany votes the Board failed to approve the placement of an Eva Moskowitz elementary charter school in Hudde Junior High School; due to the absence of Board members and the organized opposition of Hudde parents. Of course next moth the decision may be reversed, or, another board member replaced.

One of the intended, or, to be kind, unintended consequences of mayoral control has been to exclude parents and communities from the process. The prior board, one member appointed by each borough president and two by the mayor was highly politicized, and, the 32 elected community school boards ranged from grossly inept and corrupt to highly competent and engaged. Regrettably, the poorest school districts ended up with the least capable and most corrupt school boards. The political power structures in communities dominated the school board elections. The law did change in the late nineties and all supervisory appointments were made by the chancellor, with the advice, not consent of school boards.

The school board in the district in which I served as the union leader encouraged school and district-based leadership teams, school-based budgeting and provided continuous, extensive training of school teams.

The current Community Education Councils (CEC) are toothless. Seats on councils are frequently unfilled and at the meetings there are more council members than members of the community present.

The many iterations of organizational structures, from Regions to Empowerment to Networks excluded communities, except if the leader had skills from the “old days,” although that were discouraged. Where did a parent go with a question? Call 311 and you were referred back to the person who originally failed to answer your question. Frequently Asked Questions were web-based in communities in which parents had limited access and limited skills in accessing the sites.

School programs either emanated from Tweed or the Network. It was the rare school in which school-based leadership teams felt invested in school programs.

The current 92 Renewal Schools, in year one, the year just completed, were supposed to develop school plans. The program has stumbled, Patrick Wall, in Chalkbeat chronicles the efforts of Brooklyn Generation High School, a Renewal School on the South Shore High School campus, to deal with the mixed messages from the city and the state.

The system feels adrift and mayoral control lacks a rudder.

How many public forums were held to seek public input on mayoral control?

How much outreach from the offices of the chancellor or the mayor?

How many legislators attended community forums?

I would hope that next year the office of the mayor, legislators, perhaps the local members of the Board of Regents will sponsor a series of community forums to seek input.

Should the appointed members of the Board serve fixed terms?

Should the Board expand to include members appointed by others, the Speaker of the City Council? The Comptroller?

Should all Board meetings be webcast?

Should the Community Education Councils be re-structured? Should the CECs be granted additional authority? Should they play a role in setting actual district policies? An advisory role in the selection of principals? Or the superintendent?

Should superintendents be required to hold open meetings with parent leaders monthly?

Should the Department conduct locally-based in-depth training programs for school leadership teams?

We know the most effective schools have strong school cultures: parents, teachers and school leaders working from the bottom up with the support from the district and the chancellor; building distributive leadership structures; both encouraging school communities to design local initiatives and holding them accountable for the results of their efforts.

Perhaps, create a default structure for schools not yet capable of designing their own strategies; a limited Chancellor’s District.

And, the core issue: we must “measure” schools from where they are, taking into account the obstacles, the “risk load factors” that schools face.

I’ve been in schools in which Regents scores are extremely high or reading and math scores well into the highly proficient range with mediocre instruction: the kids came to school with impressive skills with strong family supports and other schools with dynamite teachers and school leaders that appear to have mediocre results if we only measure by test scores.

Principals tell me over the last few months superintendents have held principal meetings in schools and teams of principals have observed lessons and, in facilitated meetings, critiqued the lessons. The purpose: so that principal grading of lessons will be consistent across the system. While a consistent view of a lesson is important the key is the post-observation conference. Written observation reports do not improve instruction – the on-going school leader-teacher dialogue, hopefully not just at a post observation conferences, improves practice. Do school leaders have the skills to engage teachers in discussions about practice and do the discussions lead to improvements in practice, and, how do we know it?

In too many schools triage management is the norm, the crisis of the moment, is the rule; teacher observation an annoying compliance chore, and both teachers and school leaders think “if we only had better kids.” A hint: parents send you the best kids they have.

Mayoral control should not be a synonym for autocratic leadership.

An acquaintance had scheduled a meeting with a principal at the beginning of the second period; they arrived early and asked the school secretary if it was possible to meet earlier. The secretary said, “No.”

The guest asked, “Is he observing a class?” The secretary, “No.”

The guest, “Is he meeting with teachers or parents?” The secretary, “No.”

The guest, a little brusquely, “Can’t you disturb him?” The secretary, “No.”

The guest, a little rudely, “Why not?” The secretary, “He’s teaching.”

The principal taught a gym class first period to allow teachers to meet and plan at the beginning of the school day.

Instruction was at the core of the school.

My friend Jonathan always reminds us, “The answers are in the room,” staffs may need a little encouragement, a little prodding, a little help from the school or district leadership, you can’t beat teachers or principals into “getting better.”

Some schools/districts should be required to post the same quote that Dante saw on the gate of Hell, “Lasciate ogne speranza, voi ch’intrate”, or “Abandon all hope, ye who enter here, ” I would prefer schools that can post others,

“Give the pupils something to do, not something to learn; and the doing is of such a nature as to demand thinking; learning naturally results.”
― John Dewey

“Teaching is not a lost art, but the regard for it is a lost tradition.”
― Jacques Barzun,

Or, one of my favorites,

“People cited violation of the First Amendment when a New Jersey schoolteacher asserted that evolution and the Big Bang are not scientific and that Noah’s ark carried dinosaurs. This case is not about the need to separate church and state; it’s about the need to separate ignorant, scientifically illiterate people from the ranks of teachers.”
― Neil deGrasse Tyson

Do the New York City Renewal School/Out of Time School Polices Include Encouraging Cheating by Principals?

Unfortunately a core principle of the Department of Education Renewal Schools initiative seems to be: cheat.

The dozen years of Bloomberg educational policy centered on the creation of small schools and the closing of low performing schools: the result – an increase in graduation rates.

The increase in graduation rates during the Bloomberg year depended on questionable practices: credit recovery and “scrubbing” of Regents examination papers, and, sadly, the current Farina administration appears to be following the Bloomberg playbook.

Carl Campanile in the March, 2015 New York Post, reported widespread highly questionable practices at John Dewey High School, a Renewal School in Brooklyn,

Investigators are probing accusations of a massive grade-fixing scheme by educators desperate to boost the graduation rate at Dewey, The Post has learned.

Multiple sources claim Dewey is cutting corners by passing kids with the help of a shady “credit recovery” program that students sarcastically call “Easy Pass.”

In today’s New York Post Susan Edelman reports the changing of improper changing of Regents scores at Automotive High School, an “out-of-time: school.

How do you fix a failing high school? Change the grades.

Under pressure to boost student achievement, the state-designated “out of time” Automotive HS in Greenpoint, Brooklyn, has resorted to rigging Regents exam scores.

“Credit recovery” is a process by which kids who fail courses can pass the class through an expedited process; a decade or so ago a handful of schools created alternative methods of “recovering” credits. If a student attended the course regularly and failed the course the student could exhibit mastery through the creation of detailed projects, the student is granted the “lost” or “recovered” credit. I worked with a school support organization that assisted schools: a student worked with a teacher, spend 24 hours of seat time researching and writing a 12-15 page term paper to exhibit mastery.

Over time credit recovery deteriorated into credit mills. Students who never attended courses spent a handful of hours and were granted a credit. How commonplace? We’ll never know; however, virtually every school created some iteration of credit recovery, and many were educational charades,

On the verge of an investigation by the State Department of Education the city conducted their own audit and adopted more stringent guidelines. (see commissioner’s reg here)

The Department February, 2012 audit report here.

The Department press release here.

The Department High School Academic Policy Reference Guide – “credit recovery” pages 37-38

The Reference Guide requires:

Only students who have attended at least two-thirds of the class time of the original failed course are eligible to earn credit through targeted credit recovery.

Students may earn no more than a total of three core academic credits through targeted credit recovery throughout high school.

Students can only earn credit through targeted credit recovery during the semester or summer immediately ollowing the one in which they failed the original course

Intensive instruction in the applicable subject area under the direction or supervision of a teacher certified in the subject area in which the student is making up credit.

A school-based panel, which must include the principal, a teacher certified in the subject area for which the student needs to make up credit, and a guidance director or other administrator must approve a student’s participation in a make-up credit program.

To receive credit, the student must successfully complete the make-up credit program and demonstrate mastery of the learning outcomes for the subject, including passing the Regents exam, if the Regents exam is required for graduation.

These regulations are routinely ignored, with a “wink and a nod” from the Department.

During the Bloomberg administration the creation of hundreds of small schools led to teachers scoring papers of students that they taught. In the days of large high schools teachers scored many, many hundreds of papers each June. In my school we re-scored papers with grades of 62-64, sometimes we “found” an additional point or two and sometimes not, we always had two teachers re-read the papers and if they disagreed a third teacher would read the papers. In those days schools were not in danger of closing and the re-reading and “scrubbing” of papers just seemed like a reasonable policy.

During the Bloomberg years with the danger of school closings grading papers of your own students in a school closing climate; the temptation to “lean over backwards” is great.

The State Department of Education issued strict regulations – teachers could not mark papers of their own students and no re-grading of papers, no scrubbing.

Principals and other administrative staff in a school or district do not have the authority to set aside the scores arrived at by the teacher scoring committee and rescore student exam papers or to change any cores assigned through the procedures described in this manual and in the scoring materials provided by the Department. Any principal or administrator found to have done so, except in the circumstances described below, will be in violation of Department policy regarding the scoring of State exams. Teachers and administrators who violate Department policy with respect to scoring State exams may be subject to disciplinary action in accordance with Sections 3020 and 3020-a of Education Law or to action against their certification pursuant to Part 83 of the Regulations of the Commissioner of Education.

Two “out-of-time” high schools, Boys and Girls and Automotive are far beyond the “tipping point.” Boys and Girls has spent the year discharging students to other programs or just cleansing their register – the school currently reports a register of 478 with a graduation 2015 class of 95 students – more than 20% of students are absent each day – the September, 2015 entering class is miniscule.

Automotive reports a register of 350 students, a daily absence rate of 20% and only 43 students graduated this year. Again, a miniscule anticipated September, 2015 entering class, as the school data continued to erode the chancellor continued to praise the principal.

The Department is scrambling to address the wide range of issues confronting the Renewal and Out-of-Time schools. In many instances school and district leadership are faulty. In the scores of Schools Under Regents Review (SURR) the state pointed to “lack of effective leadership at the school and/or district levels” as the most distressing flaw. In too many instances I see school leaders without the requisite skills. Sometime “nurturing” school leaders who are not demanding of students and others who are autocratic and not team-builders.

The Department had failed to effectively address the core issue: getting the kids to school.

In 2008, the Center for New York City Affairs published a widely discussed report revealing that one in every five elementary school students in the city—more than 90,000 children—were chronically absent from school, missing the equivalent of a month or more of their school year. The problem was particularly acute in high-poverty neighborhoods, where 40 percent or more of students were chronically absent in some buildings.

“It’s an astounding figure,” (de Blasio] said.
“If we can’t do better on absenteeism, then none of our other educational outcomes make sense.”

The Center for NYC Affairs report identifies “risk load factors” that impede academic progress, the factors are not excuses; they are impediments.

SCHOOL FACTORS:
1. Students eligible for free lunch
2. Students known to be in temporary housing
3. Students eligible for welfare benefits from the Human Resources
Administration
4. Special education students
5. Black or Hispanic students
6. Principal turnover
7. Teacher turnover
8. Student turnover
9. Student suspensions
10. Safety score on the Learning Environment Survey
11. Engagement score on the Learning Environment Survey
NEIGHBORHOOD FACTORS:
12. Involvement with the Administration for Children’s Services
13. Poverty rate
14. Adult education levels
15. Professional employment
16. Male unemployment
17. Presence of public housing in a school’s catchment
18. Presence of a homeless shelter in a school’s catchment

Schools should be assessed taking into account the school and neighborhood risk load factors that schools confront. We differentiate among students, we should differentiate among schools.

“Cheating,” either a ‘wink and a nod” that allows shabby credit recovery or cutting corners to add points to Regents exams is a disservice to schools and students. Pushing kids “over the top” to pad data is not acceptable.

The new State Commissioner should direct the state inspector-general for testing to examine the clear violations condoned by the city.

A year and half into the new Farina administration we can point to one enormous success, the Universal Pre-K program, and, we struggle, along with staffs, to understand the direction of the remainder of the school system.

Staffs are confused, especially the Renewal Schools, inspectors from the city and state visit schools and criticize, parent involvement is absent, “districts” are shadows of the past, schools are adrift.

Turning to cheating to graduate kids is a sign of desperation.

Getting to Yes: New York State End of Session Negotiations Appear to Result in a “Framework.” Who’s Happy? Who’s Not?

ALBANY, N.Y. (CBSNewYork) — State leaders have reached a budget agreement, Gov. Andrew Cuomo announced Tuesday.

The deal includes a framework for a four-year extension of rent regulations. It also provides for a one-year extension of mayoral control of New York City’s public schools. The 421a tax abatement program for real estate developers will also be extended for six months.

Capital Confidential Bog

We have a deal — or at least the framework of a deal.
Gov. Andrew Cuomo and legislative leaders announced at an impromptu press conference Tuesday afternoon that they have struck a framework agreement on an omnibus end-of-session deal that includes a four-year extension of New York City’s rent laws, $250 million for non-public schools and a property tax rebate program worth $1.3 billion.

Minutes ago Governor Cuomo announced a “framework” for a budget agreement, the two news sites above announced different aspects of the settlement.

See Cuomo press conference here.

Over the next few hours and days more details will emerge, at first glance the tax credit for private, charter and parochial schools died and the expansion of the charter school cap “sort of” increased. Other issues were punted; mayoral control will be back next year.

Why was this year so contentious, so difficult and why was the “settlement” so limited?

First, a caveat: a settlement in concept,” or a “framework” has to be turned into legislative language and embedded in a series of bills, and “agreements in concept” have been known to fall part.

This year we saw new, more open, more consensus-building leadership in both parties. Carl Heastie on the Assembly side is completing his first legislative year as Speaker and John Flanagan only months into his job as Senate Majority Leader. Both have proven themselves to their conferences. The days of Shelly Silver and Dean Skelos are gone; the days of leaders “cracking the whip” are now days of listening to conference membership concerns.

The Assembly Democratic conference is heavily tilted toward New York City and the Senate Republican Conference tilted out of New York City, only three Republican Senators come from within the City.

Rent control, mayoral control and 421-a tax abatement are wholly New York City issues, the private, parochial, charter school tax credit a Republican issue due to the large number of dollars poured into their coffers from the charter school hedge funders.

Some laws are passed with expiration dates, called “sunset.” Both rent control and mayoral control expire at the end of the session. For the Democrats, rent control must be extended; it impacts 2.5 million renters in the city, mayoral control of schools is a de Blasio more than a conference issue. The four year rent control extension and the one year extension of mayoral control represent the importance of the issues to the Democrats; one critical, one, not so much.

Just Out – More Details:

The four-year extension of the rent laws would also include a four-year extension of the property tax cap. The two will now sunset together, rather than letting the tax cap lag for an extra year, the governor said. (To be clearer, the rent laws expired this year, while the tax cap, which was coupled with the rent laws in 2011, wouldn’t have expired until next year. The deal would mean they both end at the same time).

While the controversial EITC (tax credit) didn’t make it in, a $250 million fund for non-public schools for mandated services reimbursement would be made available.

The charter school cap would be reconfigured in order to add up to 50 new charter schools downstate and roughly 130 new charters upstate. SUNY or the state Education Department would be in charge of authorizing new charters.

More on the charter school cap from Chalkbeat here.

The state Education Department also would be required to give out more information to parents about things like teacher evaluations. The department would have to open up about test questions, growth patterns and other data, Cuomo said.

All negotiations require both sides to point to an issue that satisfies their constituencies. In this set the key item for out-of-New York City crowd was the extension of the property tax cap. The tax cap was due to expire next year, an election year, and there is little question the city law makers would have held the Republicans hostage in the same manner that the Republicans held the Democrats hostage this year.

The failure of EITC (tax credit) is a major victory for the teachers union and the many legislators that refused to fold in the conference; the $250 million for charter school “mandate relief,” I believe, is already in the budget: EITC’s demise is a major Heastie victory.

The “charter school cap reconfiguration” sounds like moving unused charter school slots upstate to New York City. “Opening up about test questions, growth patterns at other data,” needs more info.

In these types of deals everyone attempts to claim credit:

* The “three men in a room” all explaining to their conferences and the public supporters: their intransigence paid off.

* The governor, aloof during most of the negotiations, sweeps in to announce the framework, and, to the public appears to be the statesman who crafted the settlement

* The teacher union – the failure of the tax credit is a victory

* For de Blasio rent control is a victory, and, mayoral control a slap in the face.

Let me mention again, this is a framework; the thousands of pages of legislative language are the guts of the settlements. A few days from now, maybe a few weeks, someone will point to a paragraph, a sentence or a few words that got into the actual bill and will have an immense impact.

Negotiating contracts or negotiating legislation is an art form and a learned skill, a form of “Getting to Yes,” the award winning book that espouses four principles: 1) separate the people from the problem; 2) focus on interests rather than positions; 3) generate a variety of options before settling on an agreement; and 4) insist that the agreement be based on objective criteria.

In the education arena the power of the Opt-Out parents will not be assuaged by the minor points in the agreement, the movement will continue to gain traction across the state, unless, the new commissioner decides to address the testing issue, put the issue at the top of her agenda,

She arrives officially in two weeks.

The World of the Regents Are Changing – Message from the Legislature: “You’re in Charge – You’re the Experts – Do What’s Best for the Families and Staffs.”

Democracy can be contentious, awkward, frustrating and, at times, dizzying.

The fifty-four members of the Constitutional Convention, slaveholders and abolitionists, Northerners and Southerners, from large and small states, plantation owners and farmers, highly educated and not, argued, debated, reviled, left and returned, and, during the steamy months of the summer of 1787 wrote our constitution.

The Federalist and the Anti-Federalists (the “pros” and the “cons”) campaigned in the fall and winter across the thirteen colonies as voters grappled with the proposed new system of government. Madison, Hamilton and John Jay wrote eighty plus op ed essays that were published in the newspapers of the day urging support of the new constitution (Read Federalist # 10: “The Utility of the Union as a Safeguard Against Domestic Faction and Insurrection,” here). [An aside: can you imagine the NY Post or the NY Daily News publishing an essay such as Federalist # 10? Or, our citizenry reading it? Can it be reduced to 140 characters?].

Albany is in the midst of end-of-session angst, the threatening, rambunctious infighting that may or may not produce legislation on a range of seemingly intractable issues: rent control, school tax credits, mayoral control, teacher evaluation and charter schools.

Getting to the endgame is not pretty.

19th century German Chancellor Bismarck often quoted comment is an apt description: “Laws are like sausages, it is better not to see them being made.”

The Board of Regents, although constitutionally the education policy-making body in the state, has never viewed itself as a “political body.” The Regents Room is a throwback to a former century, or two. In the center of the room is a long table, the seventeen members of the Regents sit at the table, in seniority order, the Chancellor at the head of the table, flanked by the commissioner, alongside the table are three rows of seats for staff, press and guests. Hanging on the walls are paintings of former chancellors, all men, all white, and considering the Board was chartered in 1794, dressed in garb appropriate to their era.

The Board members are “elected” for five year terms by a joint meeting of the two houses of the state legislature, and, considering the current large majority of Democrats in the Assembly, the selection reflects the majority.

To be selected as a regent was an honorific, an honor for service rendered. The members were business men, lawyers, college professors, “pillars of the community,” who trekked to Albany monthly to discuss and approve a recommendation of the commissioner. Commissioners, chosen by the board, had worked their way up through the ranks, commissioners were chosen from among the ranks of state superintendents.

With the exception of Saul Cohen, former President of Queens College, the Regents rubber stamped, approved decisions, most of which has already been made by the commissioner and the chancellor. Cohen, who served a member from 1994-2010, was frequently an outspoken critic of Board actions, a lone voice.

With the election of Betty Rosa, followed by Kathleen Cashin the nature of the discussions began to change. Rosa and Cashin, both New York City superintendents with long and highly successful service began to ask the “hard questions,” and actually publicly voted against the policies of the new commissioner. King, only in his late thirties, possessed impressive degrees; however, his experience was limited to charter schools. Increasingly King, with the support of the Board, pushed the Arne Duncan agenda. While Board members may have felt uncomfortable only Rosa and Cashin actually challenged King at open Regents meetings.

The quiescent Board increasingly became the center of attention.

Newspaper coverage, blog posts, op eds and editorials and a rising uneasiness among parents that rose and rose and bubbled to a boiling cauldron. The Regents were oblivious, after all, they weren’t politicians (said with a sneer); they were “policy-makers.”

Of course when Regent Bennett cudgeled his colleagues to extend the length of charter renewals in Buffalo, over the recommendation of the commissioner, that wasn’t “politics,” in fact, it was the “good, old boy” politics that characterized the Regents.

In March the legislature acted: the two must senior members of the Regents were dumped and four new Regents selected, three former superintendents, a college professor/school nurse, all women, and, three women of color.

Regents Rosa and Cashin now had four allies.

In the waning days of the budget process the governor forced changes in the education law: extending new teacher probation from three to four years, setting in process a “school receivership,” a la Massachusetts, the poorest performing schools or school districts could be handed over to a “receiver” under provisions of the new law, and, yet another change in the teacher-principal evaluation law, the third in four years.

At the May Board meeting the acting commissioner presented a 56-page draft of recommendations for the implementation of the law, with considerable pushback, almost entirely from the six, to use Diane Ravich’s encomium: the “dissidents.”

The “dissident six” circulated a set of Guidelines to their colleagues, asked them to sign on, and, at the June Meeting offered the Guidelines as a substitute for the revised draft from the acting commissioner.

A three-hour, at times heated, at times uncomfortable discussion ensued, many of the Guidelines had been included in the new draft; however, a key portion was not changed, the percentage of student test scores to be used in computing the new Matrix.

The “six” forced their colleagues to vote publicly on all the items and, at the “official” Board meeting at the end of the session, once again, a recorded formal vote.

The Regents moved from policy-makers, above the fray, a comfortable group of seventeen “wise men” (and women) who succumbed to the policy directives of the commissioner and the chancellor to seventeen representatives of the legislature and the public.

A legislator told me: “Education policy should be left to the experts, the Regents, that’s why we select them.”

Regent Tisch: “When education policy becomes part of the budget process we have neither education nor policy.”

Regent Brown: “Tests created to assess student growth should not be used to assess teacher quality.”

A new commissioner will arrive in a few weeks, a new commissioner and a new set of rules. No longer will the commissioner solely drive policy; no longer will the Board debate and meekly approve initiatives of the commissioner; the Regents and the commissioner, of necessary, will become active partners. The Regents are no longer immune from the world of politics, immune from the voices in communities around the state.

The governor has clearly been spanked by the public; his approval ratings diving, no doubt, to an abiding anger over his meddling in education. Teacher-principal evaluation was not broken. “Receiverships” will be an albatross for the governor; no one thinks the new law is workable.

The legislature sent a clear message: education is the domain of the Regents and the commissioner, we expect them to listen to parents and teachers, we expect them to be transparent, we understand the enormous complexities of the world of education, and we will select Regents in whom we have confidence.

Regents meetings should be an exchange of ideas, differences in opinions, at times conflicts, and the process may be uncomfortable, democracy is uncomfortable. I am hopeful that a new commissioner and a far more activist Board, supported by the legislature, will begin to address the core problems confronting students and their families, teachers and the state.

Cuomo for President: Is There a “Scott Walker” Democratic Path to the White House?

Are you thinking: Is he crazed? How can he ever think that Andrew Cuomo could be a viable candidate for the Democratic nomination for the presidency?

Read on…

Scott Walker was a rather anonymous Republican governor of a blue state, Wisconsin. Walker took on the public employee unions; he supported legislation to remove collective bargaining rights for almost all public employees (except the police); demonstrations, sit-ins, civil disobedience, support from unions across the nation, all to no avail. The unions began a recall campaign, recall the legislators and the governor, once again a vicious battle, and once again a campaign that failed. Not only did the recall campaign not succeed Walker was re-elected. Walter next began an assault on private unions who had not been overly involved in the fight stripping public employees of collective bargaining. After another tendentious battle the legislature voted to make Wisconsin a right-to-work state. Walker is beginning a campaign to eliminate tenure in the state universities, and, Walker is at the top of the pile of Republicans running to become the Republican candidate for president. To the public, Walker is viewed as “tough and no nonsense.”(Walker, Rubio and Bush are currently considered finalists – Reading polling data here)

The NY Times magazine chronicles the involvement of member of the Iron Workers Union.

The anonymous governor of Wisconsin has become a real live competitor for the White House. He was underestimated until it was too late.

Martin Niemöller, a Protestant pastor, was an outspoken public foe of Adolf Hitler and spent the last seven years of Nazi rule in concentration camps.

Niemöller’s famous quote is especially relevant:

First they came for the Socialists, and I did not speak out—
Because I was not a Socialist.
Then they came for the Trade Unionists, and I did not speak out—
Because I was not a Trade Unionist.
Then they came for the Jews, and I did not speak out—
Because I was not a Jew.
Then they came for me—and there was no one left to speak for me.

Andrew Cuomo, once a dyed-in-the-wool liberal Democrat from New York is adjusting the Scott Walker playbook to game plan a presidential run from the Democratic side of the aisle.

Yes, there are obstacles that may be insurmountable, barring surprises Hillary will garner the 2016 Democratic nomination; however, surprises happen and the other Democratic contenders, Saunders, O’Malley and Chafee are polling in single digits. Than again, while Hillary appears to have an easy path to the nomination the actual November 2016 election is, at best, a toss-up.

Hillary losses would set the stage for Cuomo, either next year, or, four years later.

Obama’s path to the White House was unusual, and probably not replicable. An enormous number of new and young voters combined with traditional liberal voters, women, unions, minorities and blue-state voters.

The path to the Republican nomination is a battle on the far right: a combination of anti-abortion, Christian fundamentalists, anti-Obama, anti-immigrant, both militant foreign policy and isolationist, cut taxes, smaller government, etc., a combination which will select a Republican and may place the eventual Republican candidate outside of the mainstream of American voters.

Cuomo has carefully crafted a persona.

Socially liberal: Leading the marriage equality movement, supporting a women’s equity agenda, passing the anti-gun SAFE law, supporting raising the minimum wage; padding his credentials with the initiatives at the heart of the socially liberal Democratic agenda.

An Education Reformer: Attacking teacher unions as standing in the way of education “progress,” supporting charter schools, pushing hard for tax credits for private, parochial and charter schools, eroding tenure, ridding schools of “bad teachers,” etc.

An Economic Conservative: Year after year of balanced timely budgets, reducing taxes for the business side, a highly popular property tax cap, tax-free zones around the state, supporting both affordable housing and tax breaks for developers.

Controlling the Message: As the calendar ticks down to the June 17th legislature adjournment,the air waves are filled with pro-Cuomo TV ads supporting his policies on the Albany legislative agenda. Cuomo is the unique politician, he does not consistently place himself in the public spotlight, he carefully controls media access, and he rarely allows any interviews. Every media or public access is fully scripted and carefully controlled, no off- the- cuff comments, no interviews with reporters, no press conferences; the message is totally and completely scripted – word by word.

The Cuomo game-planners are building a candidate who straddles the center, attracting liberal Republicans and conservative Democrats, appealing to both business interests and workers (although not their unions). With Republican contenders on the far right and Democratic progressives on the left Cuomo is betting that a middle of the road candidate can emerge. A “tough” Walker is the model for a “tough” Cuomo.

Bernie Saunders, the independent Senator from Vermont was on Face the Nation this morning, clearly popular on the Democratic left. The former Brooklynite, a James Madison High School graduate is extremely popular on the progressive left, yet, polls poorly as a presidential candidate.

I love Bernie, I knew many of his teachers; when I came to Madison as a teacher, the old-timers, refugees from the Great Depression and World War 2 veterans were an extraordinary group. Both brilliant, challenging, former, or not so former Socialists; Madison was a hotbed of the old far left Teachers Union (”
Reds at the Blackboard: Communism, Civil Rights, and the New York City Teachers Union“). and, maybe helped craft his thinking.

Could you ever see yourself supporting Cuomo for president?

The Republican right appears to support repealing the Thirteenth Amendment, legislation to re-institute peonage, denigrate the role of women, abolish unions, expel undocumented immigrants and on and on, an agenda of a former century. Maybe harsh, maybe not.

Yes, I would vote tomorrow to recall Cuomo as governor of New York (the NYS Constitution does not have a recall provision); on the other hand could I support Cuomo against a far right Republican?

Politics makes for strange bedfellows.

Who Sets Education Policy in New York State: The Governor or the Board of Regents? Should Education Policy Reflect the Views of the Governor or Parents, Teachers and Principals?

“We, the undersigned, have been empowered by the Constitution of the State of New York and appointed by the New York State Legislature to serve as the policy makers and guardians of educational goals for the residents of New York State. As Regents, we are obligated to determine the best contemporary approaches to meeting the educational needs of the state’s three million P-12 students as well as all students enrolled in our post-secondary schools and the entire community of participants who use and value our cultural institutions” (Comments by seven members of the Board of Regents)

The New York State Board of Regents is the oldest state education policy board in the nation, established in 1794 as part of the state constitution. The current 17-member board is “elected’ by a joint meeting of the State legislature. The selection as Regents was an honorific, a reward for long service in the state. Members served term after term in anonymity, met monthly from September to August, selected a commissioner and set policy, generally the policy originated with the commissioner. Typically the commissioner was a state superintendent with long and distinguished service.

All that changed with the election of Merryl Tisch to replace Bob Bennett as chancellor. The quiescent board became an activist board under the leadership of Tisch. An outsider, David Steiner was selected as commissioner, a report was commissioned that exposed the state testing program, and, with the resignation of Steiner and the appointment of John King the board pursued aggressive polices championed by the new commissioner. In line with the Arne Duncan game plan one major initiative after another was approved by the Regents, frequently over the objections of Regents Cashin and Rosa.

In order to qualify for the Race to the Top grant the state entered into a long negotiations with the state teacher and principal unions and an agreement was reached – called the Annual Personnel Performance Review, aka, APPR, teachers and principals would be assessed annually using a combination of student progress on standardized tests, locally negotiated measures of student achievement and supervisory observations.

In spite of trepidation the APPR plan was benign, with exceptions in a few districts, the overwhelming percentage of teachers received “highly effective” and “effective” scores and only 1% received ” an ineffective score.

The governor was displeased. Last spring the governor accepted millions in charter school campaign contributions and forced New York City to either co-locate charters in public schools or pay the rent for non-school space. Teacher anger grew, the teacher union made no endorsement for governor and it was clear that many teachers in the Democratic primary voted for Zephyr Teachout, an unknown law professor from Fordham. While Cuomo won handily it was not the landslide he anticipated, and, again, in the November general election Cuomo won, not by the landslide he had clearly expected.

On December 18th, barely six weeks after the election Jim Malatras, the governor’s chief of operations penned an accusatory letter to chancellor Tisch and commissioner King. The letter listed a series of condemning questions, demanding responses, and threatened to take over the policy-making role of the board.

As you know, Governor Cuomo has little power over education, which is governed by the Board of Regents, The Governor’s power is through the budget process and he intends the reforms during that process.

Read the full text of the Malatras letter here.

A few weeks later chancellor Tisch and acting commissioner Berlyn responded meekly; agreeing with the accusation of Cuomo through Malatras.

Differentiation is a necessary component of any evaluation system intended to
support professional development and growth. However, as the Governor has previously indicated, changes in State law are necessary in order to achieve better differentiation and to fulfill the goal of a Statewide evaluation system that identifies those who are excelling so that they can be mentors for their colleagues, identifies those who are struggling so they can get support to improve, and informs high‐quality professional development for all educators.

Read the full text of the Tisch-Berlyn response here.

During the last hours of the budget process the Governor did exactly what he threatened; he forced through a totally new teacher evaluation system based on a Massachusetts model known as the “Matrix.”

The board convened an “Education Learning Summit,” and an all-day series of speakers, experts and stakeholders, to express opinions. Three of the four experts were critical of the use of student growth data for high stakes teacher assessment.

At the May Regents meeting acting commissioner Ken Wagner outlined a 56-page proposal to implement the new teacher evaluation law, although, the reconstituted board was much less sanguine; the four newly appointed Regents and the original critics, expressed displeasure with the new proposals.

Seven members of the board, let’s use Diane Ravitch’s term and call them the dissidents, objected to the role of the governor and outlined an alternative plan (Read Ravitch’s blog post here).

Read the entire position paper at the end of the post.

On Monday the board convenes in Albany, under the current law the board must turn the law into regulations. The guidelines introduced by the “dissident” seven directly challenges the December 18th Malatras letter.

The law clearly gives the board substantial leeway to establish regulations; will the remainder of the board support the dissident seven, reject their assertions, or find another path?

The Board of Regents, for the first time in anyone’s memory is asserting itself and reminding the governor that the state constitution grants to the Regents, not the Governor, the power to determine educational policy.

Unfortunately the P-12 Committee, the meeting at which the action will take place is not webcast. I will be tweeting (@edintheapple) – as fast as my fingers can “tweet.”

Maybe, just maybe, a brave group of board members will listen to their constituents and return education to parents and educators.

Position Paper Amendments
to Current APPR Proposed Regulations
BY SIGNATORIES BELOW JUNE 2, 2015
We, the undersigned, have been empowered by the Constitution of the State of New York and appointed by the New York State Legislature to serve as the policy makers and guardians of educational goals for the residents of New York State. As Regents, we are obligated to determine the best contemporary approaches to meeting the educational needs of the state’s three million P-12 students as well as all students enrolled in our post-secondary schools and the entire community of participants who use and value our cultural institutions.
We hold ourselves accountable to the public for the trust they have in our ability to represent and educate them about the outcomes of our actions which requires that we engage in ongoing evaluations of our efforts. The results of our efforts must be transparent and invite public comment.
We recognize that we must strengthen the accountability systems intended to ensure our students benefit from the most effective teaching practices identified in research.
After extensive deliberation that included a review of research and information gained from listening tours, we have determined that the current proposed amendments to the APPR system are based on an incomplete and inadequate understanding of how to address the task of continuously improving our educational system.
Therefore, we have determined that the following amendments are essential, and thus required, in the proposed emergency regulations to remedy the current malfunctioning APPR system.
What we seek is a well thought out, comprehensive evaluation plan which sets the framework for establishing a sound professional learning community for educators. To that end we offer these carefully considered amendments to the emergency regulations.
I. Delay implementation of district APPR plans based on April 1, 2015 legislative action until September 1, 2016.
A system that has integrity, fidelity and reliability cannot be developed absent time to review research on best practices. We must have in place a process for evaluating the evaluation system. There is insufficient evidence to support using test measures that were never meant to be used to evaluate teacher performance.
We need a large scale study, that collects rigorous evidence for fairness and reliability and the results need to be published annually. The current system should not be simply repeated with a greater emphasis on a single test score. We do not understand and do not support the elimination of the instructional evidence that defines the teaching, learning, achievement process as an element of the observation process.
Revise the submission date. Allow all districts to submit by November 15, 2015 a letter of intent regarding how they will utilize the time to review/revise their current APPR Plan.
II. A. Base the teacher evaluation process on student standardized test scores, consistent with research; the scores will account for a maximum of no more than 20% on the matrix.
B. Base 80% of teacher evaluation on student performance, leaving the following options for local school districts to select from: keeping the current local measures generating new assessments with performance –driven student activities, (performance-assessments, portfolios, scientific experiments, research projects) utilizing options like NYC Measures of Student Learning, and corresponding student growth measures.
C. Base the teacher observation category on NYSUT and UFT’s scoring ranges using their rounding up process rather than the percentage process.
III. Base no more than 10% of the teacher observation score on the work of external/peer evaluators, an option to be decided at the local district level where the decisions as to what training is needed, will also be made.
IV. Develop weighting algorithms that accommodate the developmental stages for English Language Learners (ELL) and special needs (SWD) students. Testing of ELL students who have less than 3 years of English language instruction should be prohibited.
V. Establish a work group that includes respected experts and practitioners who are to be charged with constructing an accountability system that reflects research and identifies the most effective practices. In addition, the committee will be charged with identifying rubrics and a guide for assessing our progress annually against expected outcomes.
Our recommendations should allow flexibility which allows school systems to submit locally developed accountability plans that offer evidence of rigor, validity and a theory of action that defines the system.
VI. Establish a work group to analyze the elements of the Common Core Learning Standards and Assessments to determine levels of validity, reliability, rigor and appropriateness of the developmental aspiration levels embedded in the assessment items.
No one argues against the notion of a rigorous, fair accountability system. We disagree on the implied theory of action that frames its tenet such as firing educators instead of promoting a professional learning community that attracts and retains talented educators committed to ensuring our educational goals include preparing students to be contributing members committed to sustaining and improving the standards that represent a democratic society.
We find it important to note that researchers, who often represent opposing views about the characteristics that define effective teaching, do agree on the dangers of using the VAM student growth model to measure teacher effectiveness. They agree that effectiveness can depend on a number of variables that are not constant from school year to school year. Chetty, a professor at Harvard University, often quoted as the expert in the interpretation of VAM along with co-researchers Friedman & Rockoff, offers the following two cautions: “First, using VAM for high-stakes evaluation could lead to unproductive responses such as teaching to the test or cheating; to date, there is insufficient evidence to assess the importance of this concern. Second, other measures of teacher performance, such as principal evaluations, student ratings, or classroom observations, may ultimately prove to be better predictors of teachers’ long-term impacts on students than VAMs. While we have learned much about VAM through statistical research, further work is needed to understand how VAM estimates should (or should not) be combined with other metrics to identify and retain effective teachers.”i Linda Darling Hammond agrees, in a Phi Delta Kappan March 2012 article and cautions that “none of the assumptions for the use of VAM to measure teacher effectiveness are well supported by evidence.”ii
We recommend that while the system is under review we minimize the disruption to local school districts for the 2015/16 school year and allow for a continuation of approved plans in light of the phasing in of the amended regulations.
Last year, Vicki Phillips, Executive Director for the Gates Foundation, cautioned districts to move slowly in the rollout of an accountability system based on Common Core Systems and advised a two year moratorium before using the system for high stakes outcomes. Her cautions were endorsed by Bill Gates.
We, the undersigned, wish to reach a collaborative solution to the many issues before us, specifically at this moment, the revisions to APPR. However, as we struggle with the limitations of the new law, we also wish to state that we are unwilling to forsake the ethics we value, thus this list of amendments.
Kathleen Cashin
Judith Chin
Catherine Collins
*Josephine Finn
Judith Johnson
Beverly L. Ouderkirk
Betty A. Rosa
Regent Josephine Finn said: *”I support the intent of the position paper”
i Raj Chetty, John Friedman, Jonah Rockoff, “Discussion of the American Statistical Association’s Statement (2014) on Using Value-Added Models for Educational Assessment,” May 2014, retrieved from:
http://obs.rc.fas.harvard.edu/chetty/value_added.html. The American Statistical Association (ASA) concurs with Chetty et al. (2014): “It is unknown how full implementation of an accountability system incorporating test-based indicators, such as those derived from VAMs, will affect the actions and dispositions of teachers, principals and other educators. Perceptions of transparency, fairness and credibility will be crucial in determining the degree of success of the system as a whole in achieving its goals of improving the quality of teaching. Given the unpredictability of such complex interacting forces, it is difficult to anticipate how the education system as a whole will be affected and how the educator labor market will respond. We know from experience with other quality improvement undertakings that changes in evaluation strategy have unintended consequences. A decision to use VAMs for teacher evaluations might change the way the tests are viewed and lead to changes in the school environment. For example, more classroom time might be spent on test preparation and on specific content from the test at the exclusion of content that may lead to better long-term learning gains or motivation for students. Certain schools may be hard to staff if there is a perception that it is harder for teachers to achieve good VAM scores when working in them. Overreliance on VAM scores may foster a competitive environment, discouraging collaboration and efforts to improve the educational system as a whole. David Morganstein & Ron Wasserstein, “ASA Statement on Using Value-Added Models for Educational Assessment,” Published with license by American Statistical Association, April 8 2014, published online November 7, 2014:http://amstat.tandfonline.com/doi/abs/10.1080/2330443X.2014.956906. Bachman-Hicks, Kane and Staiger (2014), likewise admit, “we know very little about how the validity of the value-added estimates may change when they are put to high stakes use. All of the available studies have relied primarily on data drawn from periods when there were no stakes attached to the teacher value-added measures.” Andrew Bacher-Hicks, Thomas J. Kane, Douglas O. Staiger, “Validating Teacher Effect Estimates Using Changes in Teacher Assignments in Los Angeles,” NBER Working Paper No. 20657, Issued in November 2014, 24-5:http://www.nber.org/papers/w20657.
ii Linda Darling-Hammond, “Can Value Added Add Value to Teacher Evaluation?” Educational Researcher, March 2015 44, 132-37:http://edr.sagepub.com/content/44/2/132.full.pdf+html?ijkey=jEZWtoEsiWg92&keytype=ref&siteid=spedr.