The single book driving education policy, on the surface, is about baseball. Moneyball: The Art of Winning an Unfair Game (2003), posits that decisions in baseball driven by mathematical algorithms will determine winners and losers.
The so-called science of baseball statistics, sabermetrics, is all the rage with a list of acronyms that baseball junkies, as well as baseball executives rely upon as gospel. In a prior age batting averages, HRs and RBIs, now OBA, WHIP, WAR, etc. drive personnel decisions.
If we can predict the value of a second baseman why can’t we use the same methodology to assess teachers?
Teacher selection has been tied to the economy, teacher surpluses in good times and teacher shortages in bad times.
The Great Consolidation, the formation of New York City in 1898 merged the five boroughs into one city, and one school system. As part of the reform movement a Board of Examiners was created, an autonomous body that created competitive examinations resulting in a rank order list – exam scores determined your rank on the eligible list. In the 1930’s during the Depression teaching was a highly sought after job and the Board of Examiners was the gate keeper. In 1950’s and 60’s a burgeoning economy pushed teaching aside, school districts scrambled to fill classrooms.
During the Vietnam War teachers in Title 1 schools were given draft exemptions and a surprising number of males sought refuge as teachers, at the end of the war some left for other professions while many stayed and made teaching a career.
By 1995 17% of teachers in New York City were provisional preparatory teachers (PPT); they accumulated the minimum number of credits but could not pass the low level exams. In hard-to-staff inner city schools teachers left faster than they came – many simply quit, others found jobs in higher achieving schools.
For decades supervisors evaluated teachers, the weak teachers left on their own or were counseled out, teachers were observed a couple of times a year – the vast percentage were rated “satisfactory.”
The New Teacher Project (TNTP), in 2009, issued the “Widget Effect” report,
“When it comes to measuring instructional performance, current policies and systems overlook significant differences between teachers. There is little or no differentiation between teachers, from good to fair or fair to poor. This is the Widget Effect: a tendency to treat all teachers as roughly interchangeable when their teaching is quite variable. Consequently, teachers are not developed as professionals with individual strengths and capabilities and poor performance is rarely identified or addressed.”
Just as Money Ball changed the way general managers viewed baseball the Widget Effect changed the way the feds and the states viewed teacher assessment.
The answer for small market baseball teams and education systems was the analysis of numbers – creating complex mathematical formulas and judge both players and teachers by the numbers.
They both sorely overestimated the use of data to substitute for human judgment.
Arne Duncan pegged Race to the Top (RttT) dollars to creating teacher evaluation plans and states scrambled to create systems anchored in student growth data (VAM) with the end product, a numerical score, a system in which all teachers are rated and compared to each other.
In New York State all 700 districts created plans (view all the plans here).
In October the state education department released the preliminary results of all the districts (with the exception of NYC),
The preliminary statewide composite results, based on data submitted by school districts and BOCES as of the October 18 deadline, found that 91.5 percent of teachers are rated Highly Effective (49.7 percent) or Effective (41.8 percent); 4.4 percent are rated Developing; and 1 percent are rated Ineffective.
The Widget Effect thesis was simply wrong – the algorithm created by one of the most well-respected research institutions in the nation produced a teacher evaluation plan that is useless.
Principals and teachers have no idea what the score means, it is useless as a guide for professional development and the “instability,” the year to year variability puts into question whether the score can serve any purpose.
The threat of a bad rating, the threat of a school closing does not result in teachers improving practice; in fact, it impedes improving practice, and diminishes the effectiveness of all teachers. Fear and test prep is no way to run a school system.
American Institutes for Research (AIR) suggests that teacher evaluation take into account the following factors,
• Student performance on annual standardized achievement tests
• Student performance on classroom tests (e.g., curriculum-based measures)
• Evaluation of student artifacts and work judged according to rubrics
• Unique assessments for teachers in nontested grades and subjects
• Unique assessments for teachers of at-risk populations
• Review of teacher portfolios
• Student surveys
• Parent surveys
• Self-report measures
• Principal evaluation
• Goal-driven professional development
• Classroom observation
In ideal systems communities of professionals, principals and teachers create a rubric, a list of the qualities of effective teaching, and use the list to guide both professional development as well as normative and summative assessments
The one element missing from the list is involvement of peers – of colleagues.
If we view ourselves as professionals we must begin to take ownership of our profession. I hear constant chirping, you can’t use test scores, principals are unfair and prejudiced, kids vary from year to year, and on and on, all complaints have some validity; however, how do we assess performance, and, more importantly, how do we create a culture of constant improvement, constant growth in practice?
The identification of the 1%, the teachers who are inadequate, must not drive an assessment system for the profession – what must drive the profession is teachers striving, always striving to become more effective teachers. The hardest working athletes are the best athletes, they never rest on their laurels, their goal is the next ring, the next championship, and, improving the team. LeBron James makes the other four players on the court better.
The process of creating a school-based assessment system, a system anchored in on-going professional development, directed by the school leader and the staff, supported by the district leadership which produces desired outcomes.
I believe that peer collaboration, involving ourselves in reviewing our own instructional practices and the practices of our colleagues is the most effective path, we must move away from a factory model to a professional model.
An Annenberg Institute report suggests,
One of the perennial criticisms of public education in the United States is the reliance on the traditional “egg-crate” model of teaching and learning, whereby teachers instruct students within isolated, closed-door classrooms with little interaction or sharing of effective practices. Peer learning among teachers and leaders, when it does happen, has traditionally been scattered and informal, with the exception of some district- and philanthropically supported efforts.
An emerging literature supports the idea that peer networks, both within and across schools, can improve teaching and learning
This lack of widespread formal knowledge sharing is coupled with an increased emphasis on evaluation systems that reward individual teachers and schools for producing higher test scores … the push for these “new teacher-evaluation systems that rely primarily on matching individual teachers with their students’ test scores threatens to exacerbate [a] competitive, rather than collaborative, system of teaching,” a system that does not lend itself to high-quality practice
[The Annenberg report recommends,]
• allow individuals in schools adequate time to learn about and share effective practices;
• build rapport and “safe spaces” for principals and practitioners to discuss challenges openly and honestly;
• understand the importance of building social capital within and across schools, and that teaching and leadership are joint enterprises
• emphasize inter-school collaboration and outward facing approach, rather than the competitive models that are increasingly popular in urban districts; and
• mix both technological and face-to-face interactions to build effective communities of practice.
Not only do these guidelines support collaboration, rather than competition and isolation, they are consistent with recent international research that illuminates the traits of the world’s most successful educational systems (Fullan 2010; Gurria 2011).
The sharing of best practices — across classrooms and schools — are tools that schools, districts, and states can use to create and retain highly effective teachers and school leaders. This recent Annenberg study found that U.S. teachers used peer-to-peer networks when available and found them valuable. But without formal structures and built-in time for such networks — and absent a culture of collaboration and teamwork — they will remain tools never used to their full potential.
The Governor, the Chancellors, both of the Regents and the NYC Education Department and the Commissioner must recognize that no matter how many regulations you pass, no matter how top-down and proscriptive the message, no matter the express or implicit threats we will only improve outcomes by involving principals and teachers, by creating cultures that honor and respect working in teams, and we urge unions to support programs that encourage teachers not to fear a culture of collaboration that includes exposing themselves to the inspection of colleagues.
There is a narrow window to make substantive changes, a new administration that is anxious to put its stamp on teaching and learning and extinguish twelve years of folly; a new administration that must not look to the past for answers; a district leadership that must look to teachers and school leaders, the folks “in the trenches” each and every day.
Windows close quickly.
Read description of a New York City high school peer review exemplar: http://www.wnyc.org/story/303290-teachers-peer-review-would-strengthen-the-profession/
Read articles discussing labor-management collaboration here