Feds Release ESSA Alternate Assessment Pilot Regulations: Will New York State Apply? Will Parents, Teachers and School Districts Choose to Participate?

The feds have posted the regulations for the ESSA Alternative Assessment pilot. The competitive pilot allows seven state pilots and encourages states to apply as consortia. The regulations (Read full text here) sets an April 2^nd filing date and the regulations sets forth specific requirements. The pilot is three years, possibility for a fourth year, with the goal the moving the pilot to the statewide assessment tool.

I know there is enthusiasm among many parents in the state, especially among the opt-out parents, moving from an examination-based accountability system to a project-based system, at first glance, is attractive.

I have heard: “Instead of a test at the end of the year students can submit a portfolio and a project.”

Unfortunately the application is far more specific.

Generate results, including annual summative determinations …. that are valid, reliable, and comparable for all students and for each subgroup of students;

  Provide for the participation of all students, including children with disabilities and English learners;  

 As a significant portion of the innovative assessment system in each required grade and subject in which both an innovative and statewide assessment are administered, items or performance tasks from the statewide assessment system that, at a minimum, have been previously pilot tested or field tested for use in the statewide assessment system.

 Align with the challenging State academic content standards … including the depth and breadth of such standards, for the grade in which a student is enrolled;

 The regulations are 45-pages long and includes the specificity noted in the sections above.

What do the terms “valid, reliable and comparable for all students” mean?

If you move to a system in which teachers grade/evaluate or assess student work: how do you assess inter-rater reliability? How do you assure the teachers/raters in Buffalo, Rochester, New York City, Scarsdale and Great Neck grade/assess projects/portfolios at the same level?

Vermont moved to a portfolio system in the early nineties and asked the Rand Corporation to assess the program, Daniel Koretz, now a professor at Harvard conducted the study.

“For a variety of reasons, such as the variability of tasks used, it may be unrealistic to expect a portfolio program to reach as high a level of reliability as a standardized performance-assessment program” … the report states. “However, the reliabilities obtained in Vermont in 1992 are sufficiently low to limit severely the uses to which the results can be put.”

 On the positive side, the study also found no evidence that teachers assigned higher or lower scores to their own students than did other raters.

 In the ensuing years technology has improved the rater reliability issue; in many schools in New York City regents essays are scanned and teachers grade anonymous papers, the LEA or the SED can review and monitor reliability, although with 700 school districts in the state, a complex process.

A number of states currently have alternative assessments waivers under No Child Left Behind, New Hampshire is in its fourth year and each year the state has added two school districts to a performance task system.

What are performance tasks?

A complicated question: SCALE, Stanford Center for Assessment, Learning and Equity, has developed a data bank of tasks,

… SCALE provides task and resource materials to schools and districts that have committed to adopting performance-based assessment as part of a multiple measures system for evaluating student learning.

 Check out the SCALE performance assessment resource bank here.

What does the New Hampshire alternative assessment look like?

The principles,

• common performance tasks that have high technical quality,
• locally designed performance tasks with guidelines for ensuring high technical quality,
• regional scoring sessions and local district peer review audits to ensure sound accountability systems and high inter-rater reliability,
• a web-based bank of local and common performance tasks, and
• a regional support network for districts and schools.

The New Hampshire pilot has changed the face of teaching and learning, teaching in a performance task system is very different from teaching in a current classroom.

I suggest accessing the New Hampshire site here.

The state works with a consultancy, 2Revolutions, that has played a major role in the training of staffs, much more than training, working with teachers and schools to change cultures, to change the face of teaching and learning in a process that totally engages all the stakeholders.

Are the parents, teachers and school leaders willing to jump off the diving board, to walk into a new world, to move away from rigid testing accountability to performance tasks, to move to a student-centered, highly individualized classroom?

The fed proposal requires consultation with all stakeholders, in a limited period of time.

How will participants be selected? Do you consult with stakeholders, submit the application, and choose actual participants after the application has been approved? Or, work with high opt-out districts in the application creation process? Do you choose a subset of schools within districts, for example, the PROSE schools in New York City? Or, do you expand the Internationals Network for new immigrant arrivals? And. all these decisions within a ten week window.

Another core issue: funding. The fed regs do not come with any additional dollars; the governor/legislature will have to add funds to the budget in a restrictive funding year, or, State Ed will have to find funding from external grants.

Daniel Koretz, the current Harvard scholar who wrote the 1992 Rand Report criticizing the Vermont Portfolio Project has a new book, The Testing Charade, Pretending to Make Schools Better (2017); although he is not anti-testing he does skewer the current use of testing – Read review here.

Can you sever testing from accountability and simply use testing a tool to guide instruction?

A weighty nuanced discussion that would normally take many months is squeezed into a narrow time frame; the folks in Albany have an extremely difficult task.

ESSA, Next Generation Standards, Curriculum: The Beginning of a Trip Up a Steep Learning Staircase

Once a month on a Monday morning the seventeen members of the Board of Regents trickle into the ornate Regents Room in Albany. A long table stretches across the center of the room, the members sit along both sides along with the commissioner; a video screen in one corner and automated cameras (the meetings are live streamed). The coffee is strong!!  Three rows of folding chairs surround the table: journalists, State Ed staff, lobbyists, advocates for some issue on the agenda, an occasional legislator, and me. Most of us know each other, at least by sight. On the walls portraits of bearded white men, the many chancellors who have led the Board of Regents since the origin of the Board in the late eighteenth century.

Slavery ended in New York State in 1827, I wonder if any of the first chancellors owned slaves?  (For another day)

The meeting begins with a sort of invocation. The chancellor, or other board member, muses about the role of the board, the million plus students, parents, what the member thinks and/or what should guide their decisions.

The first order of business on Monday was the approval of the ESSA Plan, all hundreds of pages.

View a 10-slide presentation here.

View presentations of various sections of the Plan here.

ESSA fatigue had set in, there was very little discussion. The year long effort is off to Washington with a decision by the feds expected in early 2018.

The meeting moved upstairs to the P-12 Committee and  a discussion of the Next Generation Standards, aka Common Core 2.0. The SED has been juggling the standards for two years. Hundreds of teachers, school leaders, college folk and SED staff have added and subtracted from what was the Common Core. I sat through an offsite meeting of teachers a few months ago. The very engaged teachers were totally committed, parsed words and phrases, and, I wondered: do classroom teachers, the ordinary Joe’s and Jane’s ever look at the standards?

Check out Highlights of the Revisions here.

In the discussion Regent Cashin, who was a teacher, a staff developer, a principal and a superintendent, asked a simple question:  Is there a curriculum that emerges from the standards?  The commissioner responded: curriculum is the responsibility of the school and/or school district and suggested that the BOCES centers, the regional State Education Department Support Centers could work with their surrounding school districts. In New York City the current leadership has been promising a curriculum for years.

Regents Cashin related that twenty schools in her district had used the Core Knowledge Curriculum with outstanding results. (The Joel Klein driven Board of Education did not renew the funding).

While the Common Core and it’s spin offs have dominated discussion for the last few years the importance of curriculum has emerged.

Charles Sahm in Knowledge Bank/US News wrote,  “A Compelling Case for Curriculum: Growing Evidence Suggests High Quality Curriculum is a Key Component of Student Success .”

One of the odd features of education policy is that while a plethora of research exists on the effects of systemic reforms (e.g., class size, charter schools, teacher and school accountability mechanisms), on student achievement there is very little data on whether curriculum – what kids are actually being taught – makes a difference.

 As Rebecca Kockler, Louisiana’s assistant superintendent of academic content, explained at the Hopkins/Hunter forum, over the past four years the state has reviewed more than 100 curricular programs according to their alignment with state content standards. The state leaves it up to districts to select curricula but helps them make informed decisions and facilitates professional development for the most highly rated curricula. Louisiana keeps track of what curricula districts are using and about 80 percent now employ materials from the state’s top rating tier. The state’s emphasis on curriculum appears to be generating results. Louisiana fourth graders achieved the highest growth among all states on the 2015 NAEP and the second highest in math.

Louisiana doesn’t write curriculum, they assess the alignment of curriculum to state standards on a public site.

Chester Finn, in Education Next, (“Curriculum Becomes a Reform Strategy”) hits the nail on the head,

Curriculum … is generally left to districts, which frequently leave it to individual schools and often to individual teachers or departments within them. When that classroom door closes, Ms. Smith and Ms. Gonzalez can teach pretty much whatever they want, using pretty much whatever materials they want, subject only to budgetary constraints, what’s in the “bookroom,” how fast are their internet connections, and what’s apt to be on their pupils’ end-of-year state test, which of course doesn’t exist for many subjects and high school courses.

New York State points with pride to the curriculum modules on the open source Engage NY web site. Unfortunately the modules are not aligned with current shifting standards; in New York, the state has spent the last two years amending the standards and will spend another set of years rolling out the standards. In other words, “ever-changing” standards, not aligned to modules and not aligned to state tests.

There is an increasing body of research that points to curriculum as the key to increasing student achievement.

Finn concludes,

• the cumulative impact of a well-formulated curriculum over several years can be very large indeed.
• “Changes in curriculum are relatively cost-neutral.” In other words, this is a low-budget reform. A powerful curriculum isn’t more expensive than a weak one.

Traditionally the New York City Labor Day parade takes place the Saturday after Labor Day. I met up with the UFT group on 47th Street waiting for our turn to march up Fifth Avenue. It was a spectacular day. For a couple hours you catch up with friends and make new friends. I chatted with teacher after teacher, did they know that New York State had standards? If so, did the standards impact their lesson planning and instruction? The answer was universal: they taught whatever program the school used, maybe Lucy Calkins, Teachers College Reading and Writing Project or Lucy West  or Singapore Math  or even ED Hirsch Core Knowledge or whatever the principal and/or the superintendent is partial to, in other words, the flavor of the day.

In high schools the curriculum is frequently driven by the contents of the book room.

I know this is radical, but, if we’re going to test kids wouldn’t it be a good idea to test them on the content that is actually taught?

What have we learned:

* We really good at crafting a plan, a high level of engagement across the state, every stakeholder having input, using some of the finest minds in the country to guide the process; however, will the implementation of the plan change outcomes for low performing kids and schools? Can ESSA overcome the impact of poverty, funding inequities, corrosive politics at the national and local levels?

* In spite of the federal law requiring testing state educational leaders are beginning to have doubts over the efficacy and impact of the state tests. The opt-out parents are clearly impacting thought across the state.

* Hopefully, now is the time to actually begin to explore alternatives to the traditional standardized tests

A number of states (for example New Hampshire) are piloting performance-based assessments. The Stanford Center for Assessment, Learning and Equity (usually referred to as SCALE) partners with school districts to create and implement performance assessments. New Hampshire is in the fourth year is an expanding pilot, partnering with 2Revolutions, an education design lab, embedding performance assessments and working with teachers to change the practice and culture of teaching and learning.

For some the submission of the ESSA plan is viewed as a major achievement, for me, a first step up a long and steep staircase, without handrails.

How Would We Assess Student Progress Without Standardized Tests?

In a recent blog post Diane Ravitch wrote,

After twenty years of trying, we should have learned by now that what matters most is having expert professional teachers and giving them the autonomy to do their job with out interference by the governor or legislature.

and Diane points to Finland as the model,

My favorite model remains Finland, where schools are free of standardized testing, teachers are highly educated, teaching is a high-status profession, and politicians and think tanks don’t have the nerve to tell teachers how to teach.

Without getting into a detailed “back and forth,” OECD data differentiates among nations, some data for Finland and the United States.

* poverty rate: Finland the fourth lowest poverty rate,  the US the 30th highest, we only beat out Israel.

* income inequality: Finland is the least inequitable, we only beat out Mexico.

Comparing high wealth schools with high poverty schools is as meaningless as comparing Finland to the United States. If we want to be compared to Finland we should sharply reduce the poverty and inequality gaps within the United States.

Let’s get back to the question of assessing student performance: if our goal is providing the best education, we have to define what we mean by the “best education.” If teaching a student to be literate and numerate is the “best education” we have to set benchmarks and some method of measuring if students are reaching benchmarks.

We currently use what are called “standardized” tests, meaning all kids in the state take the same tests. The grades 3-8 tests required by federal statutes as are exit exams in high school, in New York State, the Regents Exams.

When New York State precipitously  adopted the Common Core State Standards and Common Core tests proficiency rates on the test moved from 2/3 proficient to 2/3 not proficient; thereby angering parents and creating the opt-out movement.

About 20% of parents opt their kids out of the grades 3 – 8 exams, the opt-outs are concentrated in high wealth school districts (meaning folks pay high property taxes) in the suburbs and high achieving schools in New York City.

Tests are not new, prior to No Child Left Behind (NCLB) we tested kids in grade four and eight, and, New York City has a long history of testing; local school districts gave tests to monitor student progress along with citywide tests. Regents exams have been around since the 1880’s

The difference is tests are now used to assess teacher, principal and school performance, and, the results are accountability based; meaning possible school closing and teacher ratings. The new Every School Succeeds Act (ESSA) may, we’ll find out in a few weeks, include in the plan “growth” as well as “proficiency, and, perhaps an “equity” measure.

If we ditch tests, it is unlikely we can move to the Finland system: a nation with very low childhood poverty and among the lowest income inequality among the (Organization for Economic and Cultural Development) nations.

There are other tools that are currently being used to assess student progress.

A number of school districts in California are utilizing performance tasks developed by SCALE, a Stanford-based program that has developed a bank of performance assessments,

Unlike multiple-choice “bubble” tests, performance assessments require students to construct an original response rather than simply recognize a correct answer. The Performance Assessment Resource Bank includes high-quality tasks that engage students in multiple-step and extended performances, such as researching and developing mathematical models to write an article on the rising cost of college tuition. As tasks become more complex and require greater student direction they assess more complex and integrated aspects of learning and require the planning, problem-solving, and persistence that are necessary for success in the real world. This means that the use of performance assessment can both measure and encourage the development of many of the 21st century skills—critical thinking, inquiry, communication, collaboration—that are essential for success in college, career, and life.

See an example of a 9th grade Social Studies performance task/assessment here.

The New York City-based Performance-Based Assessment Consortium  (PBAC), currently 39 high schools, has been receiving waivers from the NYS commissioner, students utilize portfolio/roundtable assessment procedures in lieu of three regents (They still take the mathematics and English regents exams). The State Department of Education has been granting waivers for a cohort of CPBC schools since the nineties. The current waiver expires at the end of this school year. Check out the PBAC site here.

In the nineties Vermont moved to a statewide attempt to replace standardized with a portfolio system; after a number of years Vermont abandoned the initiative – an external report, authored by Harvard scholar Daniel Koretz and others, found inter-rater reliability was absent.

In 2004 Jay Mathews at Education Next explored a number of authentic assessments of student work alternatives to testing, and had doubts,

Lisa Graham Keegan, chief executive officer of the Washington-based Education Leaders Council, said she thinks portfolios can help teachers assess their students’ progress, but are not a good tool for determining how a school or a district is doing. She remembers a visit to a northern Arizona school where “the writing teacher was showing me a portfolio of a student’s work in which the student was writing about kamikaze pilots during World War II.” Keegan was state school superintendent for Arizona at the time and saw that “the essay was horribly written, with glaring spelling and grammatical errors, and yet had received a score of 23 out of 25 points.

“The teacher was just glowing with what a mature and moving topic the student had chosen without any direction from her. I was less impressed and said so–something along the lines of how I could appreciate that the student had something interesting to say, but my first impression was that he didn’t know how to say it–and wasn’t that the first order task for the teacher?”

Having students display their personal strengths is fine, Keegan said, as long as they still learn to read, write, and do math capably before they graduate. “A collection of student work can be incredibly valuable,” she said, “but it cannot replace an objective and systematic diagnostic program. Hopefully, we will come to a place where we incorporate both.”

Daniel Koretz and others, raise questions about quality control in performance assessments,

 … direct assessments of complex performance do not typically generalize from one task to another and thus require careful sampling of tasks to secure an acceptable degree of score reliability and validity for most uses. These observations suggest the pressing need for greater quality control in the design and execution of performance assessments. If such assessments are to have lasting effects on instruction and learning, then their technical properties must be understood and appreciated by developer and practitioner alike.

A more recent report explores these questions, The Center for Educator Compensation Reform, “Measuring and Promoting Inter-Rater Agreement  of Teacher and Principal :Performance Ratings,” February 2012, is a comprehensive look.

Moving from testing to performance tasks/assessments and portfolios will be challenging; however, now is the time for New York State to begin to move forward.

I suggest a number of pilots,  maybe in high opt-out schools, a few in New York City, others in suburban school districts.

For example, a number of schools in New York City are high achieving, high opt-out schools, perhaps candidates for pilots. On Long Island and a few other suburban districts, high opt-out schools/school districts might be candidates for district pilots.

Pilots must be partnerships with teacher unions and higher education institutions, moving to performance tasks and/or portfolios is a major instructional shift and will require both buy-in and an enormous dose of support. New Hampshire, the major example of a state that is moving towards performance tasks is hugely invested in supporting the folks on the front lines – classrooms teachers. Read an description of the New Hampshire efforts here.

We should not tarry.

There is an absence of leadership at the US Education Department, ironically, a good thing. Previously Washington administrations (Arne Duncan, John King) were intrusive, they attempted to drive their views of education down to the classroom level. The current administration clearly has no interest in teaching and learning, they are concerned with choice, i. e., charters and vouchers.

As soon as the ESSA plan is submitted, September, the state should begin the process of creating pilot schools and school districts, exploring the complexities of moving away from standardized tests to a system of performance tasks and portfolios. We don’t need a state-wide system, at this point let’s begin the process. Down the road we may have a system in which some schools/school districts stay with standardized testing while others move to other assessment systems.

There are times not being first, waiting and seeing how initiatives work out makes sense; other times being out front allows you to set the rules. Vermont and New Hampshire are well along the path, also, far different than New York State. A window has opened, teacher unions and some schools/school districts, are ready to move away  tests, it will be a complex task, very complex:  let’s get started.