Advocacy Research: So-Called Research Must Not Be Misused to Defend Politically Popular Educational Policies

It has become commonplace for electeds, journalists and advocates to begin a speech or article with “Research says ….” The use of the word “research” adds credibility to whatever argument they’re pursuing.  I was taught that research begins with a hypothesis, moves to a research design, the collection of data or access to data sets, the application of the appropriate statistical tools, the analysis of the results commonly expressed in a standard deviations  (“…quantity calculated to indicate the extent of deviation for a group as a whole”) and the reporting of the significance of the findings.

Sadly, advocacy researchadvocacy research is commonplace. Organizations hunting for research findings that support their positions or conducting research to “prove” positions already established.

(See Howard Wainer, “Uneducated Guesses: Using Evidence to Uncover Misguided Education Policies, 2011 – watch U-Tube here)

The New Teacher Project (TNTP) published “The Widget Effect,” research alleging that teachers were unsupervised and urging a more formalized teacher evaluation system. The New Teacher Project has been a critic of teacher tenure and supports using value-added measurements of teacher effectiveness to drive salary schedules, promotions and discharge decisions.  The report has been quoted hundreds of times, see examples here  and here.

States jumped on board and many states adopted teacher evaluation plans based on data – the application of a dense algorithm usually called value-added measurement. When the dust cleared tens of millions of dollars later, there is no evidence that “teaching” is more effective, in fact, the reliance on complex statistical tools did not meet generally accepted standards of validity, reliability and stability and has had the unintended consequence of discouraging prospective teachers. Not only has the reliance on data not separated teachers by ability the entire initiative alienated teachers, created an opt-out movement and the number of students entering teacher preparation programs sharply declined.

The TNTP Widget Effect report ignored teacher retention. Almost 40% of teachers in New York City leave within five years and in high-poverty, low -achieving middle schools up to 70% of teachers leave within three years. Isn’t the question of why are teachers leaving, and, who are the “leavers” more important than staining each teacher with a variable numeric score?

The Gates-funded three year Measures of Effective Teaching  (MET) study spent millions to parse day-to-day teaching, 360 degree cameras recording teaching in thousands of classrooms – breaking teaching down teaching into discrete increments,

The findings will be useful to school districts working to implement new development and evaluation systems for teachers. Such systems should not only identify great teaching, but also provide the feedback teachers need to improve their practice and serve as the basis for more targeted professional development

Three years after the release of the report New York State has placed a moratorium on the use of student test scores to evaluate teachers, the MET Study is gathering dust on book shelves.

Teacher value-added measurement (VAM) scores vary widely from year to year, they are unstable. Highly effective teachers one year are ordinary the next, ineffective teachers are effective the following year. We know that teachers who teach the same students over multiple years achieve superb results; however, schools rarely move teachers from grade to grade with students.

Suspensions of students lead s to poor academic achievement, dropping out of high school, trouble with the law and incarceration: the school to prison pipeline. Study after study shows that Afro-American males are suspended at much higher rates than all other students and multiple suspensions correlate closely to dropouts and incarceration. The conclusion: decreasing suspensions will lead to higher graduation rates and break the pipeline. You rarely hear discussions of the behaviors that precede the suspension.

In New York City the District has tightened the suspension rules and suspensions have dipped 40%, some schools are implementing Restorative Justice and Positive Behavioral Interventions and Supports (PBIS) approaches. Principals complain they don’t have capacity to deal with serious behavioral problems, aka social and emotional issues: no psychologists, no social workers, guidance counselors limited to working with special education students, limited budgets and new, inexperienced teachers. Other schools place these programs at the top of their funding priorities.

Is PBIS effective? How do you define effective? What is the impact of student backgrounds?  (See Goldhaber, Brewer & Anderson, Education Economics, 7 (3), 1999)

The “school to prison pipeline” is a mantra, widely quoted and universally accepted, without exploring the possible root causes of unacceptable behaviors.

Other areas of research, less popular, might be more useful:

* Who are the students who move from suspension to incarceration?

* What do we know about them both inside of school and outside of school?  Family structure? Housing? Family background?  School Attendance?  School Achievement?

* Can we identify students, as early as possible, who are candidates for incarceration and are there early intervention strategies? Who are the students in the same schools who survive and prosper, and, a crucial question: Why?

Anna Devere Smith, the acclaimed playwright and actor has produced a new project,

Urgent and inspiring, Notes from the Field: Doing Time in Education outlines the civil rights crisis currently erupting at the intersection between America’s education system and its mass incarceration epidemic.

Unfortunately we live in an age in which “politically popular solutions” precede research.  The school to prison pipeline indicts teachers, if only teachers had the skills the “mass incarceration epidemic” could be halted. If only teachers understood “white privilege” and “culturally relevant pedagogy” the pipeline could be broken. 

Maybe.

As teachers gain experience the pedagogical toolkit grows; however, the skills of the teacher may not be able to overcome factors external to school.

The culture, the mores, the rituals of impoverished communities surrounding schools may be antithetical to the behaviors associated with the participating in collaborative, learning environments.

Yes, research can be time-consuming, expensive and frustrating, widely held beliefs may not be valid.

Before we jump on the next new “thing” bandwagon it would be nice to have some assurance we’re moving in the right direction.