The feds have posted the regulations for the ESSA Alternative Assessment pilot. The competitive pilot allows seven state pilots and encourages states to apply as consortia. The regulations (Read full text here) sets an April 2nd filing date and the regulations sets forth specific requirements. The pilot is three years, possibility for a fourth year, with the goal the moving the pilot to the statewide assessment tool.
I know there is enthusiasm among many parents in the state, especially among the opt-out parents, moving from an examination-based accountability system to a project-based system, at first glance, is attractive.
I have heard: “Instead of a test at the end of the year students can submit a portfolio and a project.”
Unfortunately the application is far more specific.
Generate results, including annual summative determinations …. that are valid, reliable, and comparable for all students and for each subgroup of students;
Provide for the participation of all students, including children with disabilities and English learners;
As a significant portion of the innovative assessment system in each required grade and subject in which both an innovative and statewide assessment are administered, items or performance tasks from the statewide assessment system that, at a minimum, have been previously pilot tested or field tested for use in the statewide assessment system.
Align with the challenging State academic content standards … including the depth and breadth of such standards, for the grade in which a student is enrolled;
The regulations are 45-pages long and includes the specificity noted in the sections above.
What do the terms “valid, reliable and comparable for all students” mean?
If you move to a system in which teachers grade/evaluate or assess student work: how do you assess inter-rater reliability? How do you assure the teachers/raters in Buffalo, Rochester, New York City, Scarsdale and Great Neck grade/assess projects/portfolios at the same level?
Vermont moved to a portfolio system in the early nineties and asked the Rand Corporation to assess the program, Daniel Koretz, now a professor at Harvard conducted the study.
“For a variety of reasons, such as the variability of tasks used, it may be unrealistic to expect a portfolio program to reach as high a level of reliability as a standardized performance-assessment program” … the report states. “However, the reliabilities obtained in Vermont in 1992 are sufficiently low to limit severely the uses to which the results can be put.”
On the positive side, the study also found no evidence that teachers assigned higher or lower scores to their own students than did other raters.
In the ensuing years technology has improved the rater reliability issue; in many schools in New York City regents essays are scanned and teachers grade anonymous papers, the LEA or the SED can review and monitor reliability, although with 700 school districts in the state, a complex process.
A number of states currently have alternative assessments waivers under No Child Left Behind, New Hampshire is in its fourth year and each year the state has added two school districts to a performance task system.
What are performance tasks?
A complicated question: SCALE, Stanford Center for Assessment, Learning and Equity, has developed a data bank of tasks,
… SCALE provides task and resource materials to schools and districts that have committed to adopting performance-based assessment as part of a multiple measures system for evaluating student learning.
Check out the SCALE performance assessment resource bank here.
What does the New Hampshire alternative assessment look like?
- common performance tasks that have high technical quality,
- locally designed performance tasks with guidelines for ensuring high technical quality,
- regional scoring sessions and local district peer review audits to ensure sound accountability systems and high inter-rater reliability,
- a web-based bank of local and common performance tasks, and
- a regional support network for districts and schools.
The New Hampshire pilot has changed the face of teaching and learning, teaching in a performance task system is very different from teaching in a current classroom.
I suggest accessing the New Hampshire site here.
The state works with a consultancy, 2Revolutions, that has played a major role in the training of staffs, much more than training, working with teachers and schools to change cultures, to change the face of teaching and learning in a process that totally engages all the stakeholders.
Are the parents, teachers and school leaders willing to jump off the diving board, to walk into a new world, to move away from rigid testing accountability to performance tasks, to move to a student-centered, highly individualized classroom?
The fed proposal requires consultation with all stakeholders, in a limited period of time.
How will participants be selected? Do you consult with stakeholders, submit the application, and choose actual participants after the application has been approved? Or, work with high opt-out districts in the application creation process? Do you choose a subset of schools within districts, for example, the PROSE schools in New York City? Or, do you expand the Internationals Network for new immigrant arrivals? And. all these decisions within a ten week window.
Another core issue: funding. The fed regs do not come with any additional dollars; the governor/legislature will have to add funds to the budget in a restrictive funding year, or, State Ed will have to find funding from external grants.
Daniel Koretz, the current Harvard scholar who wrote the 1992 Rand Report criticizing the Vermont Portfolio Project has a new book, The Testing Charade, Pretending to Make Schools Better (2017); although he is not anti-testing he does skewer the current use of testing – Read review here.
Can you sever testing from accountability and simply use testing a tool to guide instruction?
A weighty nuanced discussion that would normally take many months is squeezed into a narrow time frame; the folks in Albany have an extremely difficult task.