Tag Archives: Performance-Based Assessment Consortium

How Would We Assess Student Progress Without Standardized Tests?

In a recent blog post Diane Ravitch wrote,

After twenty years of trying, we should have learned by now that what matters most is having expert professional teachers and giving them the autonomy to do their job with out interference by the governor or legislature.

and Diane points to Finland as the model,

My favorite model remains Finland, where schools are free of standardized testing, teachers are highly educated, teaching is a high-status profession, and politicians and think tanks don’t have the nerve to tell teachers how to teach.

Without getting into a detailed “back and forth,” OECD data differentiates among nations, some data for Finland and the United States.

* poverty rate: Finland the fourth lowest poverty rate,  the US the 30th highest, we only beat out Israel.

* income inequality: Finland is the least inequitable, we only beat out Mexico.

Comparing high wealth schools with high poverty schools is as meaningless as comparing Finland to the United States. If we want to be compared to Finland we should sharply reduce the poverty and inequality gaps within the United States.

Let’s get back to the question of assessing student performance: if our goal is providing the best education, we have to define what we mean by the “best education.” If teaching a student to be literate and numerate is the “best education” we have to set benchmarks and some method of measuring if students are reaching benchmarks.

We currently use what are called “standardized” tests, meaning all kids in the state take the same tests. The grades 3-8 tests required by federal statutes as are exit exams in high school, in New York State, the Regents Exams.

When New York State precipitously  adopted the Common Core State Standards and Common Core tests proficiency rates on the test moved from 2/3 proficient to 2/3 not proficient; thereby angering parents and creating the opt-out movement.

About 20% of parents opt their kids out of the grades 3 – 8 exams, the opt-outs are concentrated in high wealth school districts (meaning folks pay high property taxes) in the suburbs and high achieving schools in New York City.

Tests are not new, prior to No Child Left Behind (NCLB) we tested kids in grade four and eight, and, New York City has a long history of testing; local school districts gave tests to monitor student progress along with citywide tests. Regents exams have been around since the 1880’s

The difference is tests are now used to assess teacher, principal and school performance, and, the results are accountability based; meaning possible school closing and teacher ratings. The new Every School Succeeds Act (ESSA) may, we’ll find out in a few weeks, include in the plan “growth” as well as “proficiency, and, perhaps an “equity” measure.

If we ditch tests, it is unlikely we can move to the Finland system: a nation with very low childhood poverty and among the lowest income inequality among the (Organization for Economic and Cultural Development) nations.

There are other tools that are currently being used to assess student progress.

A number of school districts in California are utilizing performance tasks developed by SCALE, a Stanford-based program that has developed a bank of performance assessments,

Unlike multiple-choice “bubble” tests, performance assessments require students to construct an original response rather than simply recognize a correct answer. The Performance Assessment Resource Bank includes high-quality tasks that engage students in multiple-step and extended performances, such as researching and developing mathematical models to write an article on the rising cost of college tuition. As tasks become more complex and require greater student direction they assess more complex and integrated aspects of learning and require the planning, problem-solving, and persistence that are necessary for success in the real world. This means that the use of performance assessment can both measure and encourage the development of many of the 21st century skills—critical thinking, inquiry, communication, collaboration—that are essential for success in college, career, and life.

See an example of a 9th grade Social Studies performance task/assessment here.

The New York City-based Performance-Based Assessment Consortium  (PBAC), currently 39 high schools, has been receiving waivers from the NYS commissioner, students utilize portfolio/roundtable assessment procedures in lieu of three regents (They still take the mathematics and English regents exams). The State Department of Education has been granting waivers for a cohort of CPBC schools since the nineties. The current waiver expires at the end of this school year. Check out the PBAC site here.

In the nineties Vermont moved to a statewide attempt to replace standardized with a portfolio system; after a number of years Vermont abandoned the initiative – an external report, authored by Harvard scholar Daniel Koretz and others, found inter-rater reliability was absent.

In 2004 Jay Mathews at Education Next explored a number of authentic assessments of student work alternatives to testing, and had doubts,

Lisa Graham Keegan, chief executive officer of the Washington-based Education Leaders Council, said she thinks portfolios can help teachers assess their students’ progress, but are not a good tool for determining how a school or a district is doing. She remembers a visit to a northern Arizona school where “the writing teacher was showing me a portfolio of a student’s work in which the student was writing about kamikaze pilots during World War II.” Keegan was state school superintendent for Arizona at the time and saw that “the essay was horribly written, with glaring spelling and grammatical errors, and yet had received a score of 23 out of 25 points.

“The teacher was just glowing with what a mature and moving topic the student had chosen without any direction from her. I was less impressed and said so–something along the lines of how I could appreciate that the student had something interesting to say, but my first impression was that he didn’t know how to say it–and wasn’t that the first order task for the teacher?”

Having students display their personal strengths is fine, Keegan said, as long as they still learn to read, write, and do math capably before they graduate. “A collection of student work can be incredibly valuable,” she said, “but it cannot replace an objective and systematic diagnostic program. Hopefully, we will come to a place where we incorporate both.”

Daniel Koretz and others, raise questions about quality control in performance assessments,

 … direct assessments of complex performance do not typically generalize from one task to another and thus require careful sampling of tasks to secure an acceptable degree of score reliability and validity for most uses. These observations suggest the pressing need for greater quality control in the design and execution of performance assessments. If such assessments are to have lasting effects on instruction and learning, then their technical properties must be understood and appreciated by developer and practitioner alike.

A more recent report explores these questions, The Center for Educator Compensation Reform, “Measuring and Promoting Inter-Rater Agreement  of Teacher and Principal :Performance Ratings,” February 2012, is a comprehensive look.

Moving from testing to performance tasks/assessments and portfolios will be challenging; however, now is the time for New York State to begin to move forward.

I suggest a number of pilots,  maybe in high opt-out schools, a few in New York City, others in suburban school districts.

For example, a number of schools in New York City are high achieving, high opt-out schools, perhaps candidates for pilots. On Long Island and a few other suburban districts, high opt-out schools/school districts might be candidates for district pilots.

Pilots must be partnerships with teacher unions and higher education institutions, moving to performance tasks and/or portfolios is a major instructional shift and will require both buy-in and an enormous dose of support. New Hampshire, the major example of a state that is moving towards performance tasks is hugely invested in supporting the folks on the front lines – classrooms teachers. Read an description of the New Hampshire efforts here.

We should not tarry.

There is an absence of leadership at the US Education Department, ironically, a good thing. Previously Washington administrations (Arne Duncan, John King) were intrusive, they attempted to drive their views of education down to the classroom level. The current administration clearly has no interest in teaching and learning, they are concerned with choice, i. e., charters and vouchers.

As soon as the ESSA plan is submitted, September, the state should begin the process of creating pilot schools and school districts, exploring the complexities of moving away from standardized tests to a system of performance tasks and portfolios. We don’t need a state-wide system, at this point let’s begin the process. Down the road we may have a system in which some schools/school districts stay with standardized testing while others move to other assessment systems.

There are times not being first, waiting and seeing how initiatives work out makes sense; other times being out front allows you to set the rules. Vermont and New Hampshire are well along the path, also, far different than New York State. A window has opened, teacher unions and some schools/school districts, are ready to move away  tests, it will be a complex task, very complex:  let’s get started.

Is It Time to Review High School Graduation Requirements? Regents Exams? Computer Science as a Required Course? Authentic Assessments?

The Commissioner and the Board of Regents have been totally focused on writing a new school accountability plan under the provisions of the new Every School Succeeds Act (ESSA).  Hopefully the plan will be more equitable, the plan will identify the Title 1 schools in the lowest five percent as defined by the metrics in the state plan.

Will the plan impact teaching and learning?  Will we be identifying the same schools we would have identified under the prior law, No Child Left Behind?

While I am hopeful that the new plan will be an improvement larger questions emerge: How do we define “college and career ready?” Do our current graduation requirements, courses and assessments, i. e., regents exams, lead to college/career readiness?

David Conley, “Four Keys to College and Career Readiness” is the national expert and has written extensively.

New York State uses a narrow definition: The City University (CUNY) defines college and career readiness as grades of 75 on the Algebra 1 Regents and 80 on the English Regents.  State Ed, under the leadership of acting commissioner Ken Wagner was planning to move to aspirational regents grades: five “levels” of achievement.

Level 5: Exceeds Common Core expectations

Level 4: Meets Common Core expectations

Level 3: Partially meets Common Core expectations … comparable to students who pass current Regents exams with a score of 65

Level 2: (Safety Net) Partially meets Common Core expectations (required for local diploma purposes), expect comparable percentages of students who pass current Regents exams with a score of 55.

Level 1: Does not demonstrate Knowledge and Skills.

These “levels” would be scale scores, the test would undergo psychometric massage to determine the level.

The Commissioner, quietly, backed away from the plan to move from the current  0-100 grading system with 65 passing to aspirational scale score levels.

An underlying issue: courses and assessment exams.

The high school graduation requirements are below:  22 units (44 one-term courses) click on the link for a more detailed explanation.

  1. English, four units of commencement level credit;
  2. social studies, four units of credit … ;
  3. science, three units of credit of commencement level science, at least one course shall be life sciences and at least one in the physical sciences, the third may be either life sciences or physical sciences;
  4. mathematics, three units of credit of mathematics, which shall be at a more advanced level than grade eight, shall meet commencement level learning standards as determined by the commissioner, provided that no more than two credits shall be earned for any Integrated Algebra, Geometry, or Algebra 2 and Trigonometry commencement level mathematics course;
  5. visual arts and/or music, dance, or theatre, one unit of credit; and
  6. health education, one-half unit of credit in accordance with the requirements set forth in section 135.3(c) of this Title. Learning standards in the area of parenting shall be attained through either the health or family and consumer sciences programs or a separate course.

In addition to the courses students must pass exit exams – the Regents Exams.

English

Mathematics (usually Algebra 1)

Science (usually Living Environment)

American History and Government (usually at the end of the Junior year)

Global History and Geography (currently covers two years (9th and 10th grades) of work, in June 2018 the exam will only cover 10th grade work)

Check here for a detailed description and alternative pathways

Let’s ask some essential questions:

* Should we continue to “nibble around edges,” namely, making it incrementally easier to graduate, or, address the essential questions?

Should we adopt a state-wide core curriculum with required readings? The current EngageNY curriculum modules are not required and the state tests are not based on a curriculum, they are based on a set of standards. Should state tests be curriculum and standards based?

Should instruction be grade level regardless of the level of the students?  Some argue that by teaching to the level of the kids we are assuring that kids will never reach grade level or higher?

There are school and grade organizational models that are far more instructionally impactful than others – is it the role of the state to “strongly encourage” evidence-based grade/school organizational/instructional models?

Should coding and computer science be part of  school curriculum and graduation requirements? New York City has announced a Computer Science for All initiative,

Through an unprecedented public-private partnership, by 2025, all NYC public school students will receive meaningful, highquality Computer Science (CS) education at each school level: elementary, middle, and high school. Over the next 10 years, the DOE will train nearly 5,000 teachers who will bring CS education to the City’s ~1.1 million public school students. 

Hunter College made a presentation at the last Regents Meeting asking the State to approve a new teacher certification area: Teacher of Computer Science. – Grades 9 – 12. (Read proposal here).

Over 18 million students have code.org accounts – has New York State adopted code.org? Has/should the state add computer science to the state curriculum? State graduation requirements?

And, the elephant in the room: moving from pencil and paper (or computer screen) examinations to performance task and portfolio/roundtable assessments, aka, authentic assessments. Are alternative assessments evidence-based assessments, or, the “softening” of assessments?

A cluster of New York City high schools have been granted waivers from Regents exams for twenty years, although the number of schools and the conditions of the waivers have changed (see the Performance Based Assessment Consortium here).

The state of Vermont spent years in the nineties trying to create a state-wide portfolio system that was eventually abandoned primarily due to the absence of inter-rater reliability (Check discussions here and here); Vermont is once again making an effort to move to classroom-based authentic assessments, read here.

The California Performance Assessment Consortium (CBAC) has created a bank of assessments and is working with a wide cohort of schools. Watch a live U-Tube of an  in depth discussion of the program here, including benchmarks and student work, the site of excellent!!

I am not advocating for any specific change – I am advocating for an investigation, moving beyond “playing” with graduation/testing requirements and exploring taking a deep dive into the base questions:

* Graduation requirements, are we requiring the “right” courses, and

* Should  the assessments reflect the curriculum as well as the standards, and

* Are authentic assessments, namely performance tasks and portfolios, “reliable” indicators of the quality of student work, and, if so, should we be moving forward with pilots?

Completing the ESSA school accountability plan is a beginning, a baby step, self-reflection is at the heart of effective teaching, and, effective leadership.

If we’re not satisfied with where we are now how we can we make the system better?