APPR (Teacher Evaluation) Complexity: If Principals, Teachers and Parents Don’t Understand It, It Will Fail.

On Wednesday night the UFT and the City submitted their versions of a teacher evaluation plan to the state and over the next two weeks the areas of disagreement will be narrowed and a plan will be agreed upon or imposed by the commissioner.

Otto Von Bismarck was the architect of the unification of the German nation. He was an apostle of “realpolitik,” (politics based on practical and material factors rather than on theoretical or ethical objectives), both reviled and admired as a master of international political wheeling and dealing.

Two provinces on the Denmark-German border, Schleswig and Holstein, were part of an extremely complex dispute.

Years later Bismarck was asked to explain the details of the dispute, he replied, “Only three people have fully understood the issues, one died, one is mad, and I’ve forgotten.”

I fear the New York State APPR teacher and principal evaluation plans (see all approved plans here) are so complex, that the implementation has so many moving parts; it is highly unlikely that mere mortals fully understand the plans.

* 20% of the plan is made up of NYS student test data – a Value-Added Model (VAM), measuring student growth adjusted for a number of variables (Disability, English Language Learners, Title 1 eligibility, etc.), well, not exactly – only 21% of teachers teach students who are tested – 79% of teachers are not impacted by state tests. Teachers in grades pre-K – 2, all elementary school specialty teachers, all middle school teachers other than English and Math and all high school teachers are not impacted by the state tests.

Additionally middle and high school teachers teach five classes a day across multiple grades – how do you “measure” teachers who are teaching multiple grades? Check out the “50% rule” on the EngageNY website. Think you’re principal will be able to figure this out?

* locally negotiated 20% depends on Student Learning Objectives (SLOs) – sort of a pre-test/post-test measurement of students that is highly subjective and in no way passes any test of “validity and reliability”

Once again, take a look at Student Learning Objective 101 on the EngageNY website. 70,000 teachers in 1600 schools creating SLOs to measure the progress of 1.1 million students, probably can knock it off in a prep period!! Or, hire a few hundred SLO Creation and Implementation Developers (SLOCID) to scurry from school to school…

* The 60% supervisory/peer assessment through classroom observations are the major section of the APPR. I doubt many (any?) plans include peers in the classroom observation section. In my random sampling of plans many call for “one announced and one unannounced observation per year” utilizing a scoring rubric approved by the bargaining agent, in New York City, the Danielson Frameworks.

The Danielson Frameworks 2013 are a 114-page document describing in minute detail classroom behaviors divided in Domains, Components and Elements. Last year I sat through two days of “training” on an earlier version of the Frameworks – we watched videos, discussed, took a “mini quiz,” and assessed lessons. I sat with six principals – we watched a video of lesson – “scored” the lesson, and compared our scoring – with wide differences among us. At the end of the session Teachscape made a pitch – for many thousands of dollars principals could buy software to train themselves and staffs – I was told more than half of the schools in NYC purchased all or part of the Teachscape package.

Danielson, strangely, is not a fan of standardized testing ,

I do understand the policy people desperately wanting to assure their public that they are paying attention to, not only what teachers do, which is teacher practice but to also the effect that their teaching has on student learning. I think that is a reasonable request.

The real question is what counts as evidence and how can you attribute that evidence to individuals? That is the challenge and I do not believe anyone has figured that out yet. There is a great deal of psycho-metric evidence that indicates that any standardized tests are a poor measure of student learning for this purpose, but in many cases that’s all that’s available.

I am not thrilled about saying that a teacher’s performance is a level 2 and not a level 3. However, we are in a world of high-stakes evaluation, and so it may be necessary to do so.

Can a teacher receive a high grade in one category and a low grade in another category, and, if so, what does that say about the system?

When New York City distributed scores to teachers, percentiles scores, they also gave the teacher a wide range – a standard error of measurement,

The standard error of measurement (SEM) is an estimate of error to use in interpreting an individual’s test score. A test score is an estimate of a person’s “true” test performance. Using a reliability coefficient and the test’s standard deviation, we can calculate this value:

SEM = s  1 – r), Where:
S = the standard deviation for the test
r = the reliability coefficient for the test

I’m sure your supervisor will have no problem working out the formula to assure that you fully understand your grade and the ramifications.

Do not fear: the folks at the State Education Department have provided a number of detailed reports and webinars – click here for “Resources about State Growth Measures.”

After you complete reading the resources an online test will be administered (only kidding!!)

Teachers complain that supervisors “walk-throughs” are “drive-bys,” implying that supervisors are using a few minutes to derogate the lesson. I demur: as you walk into a classroom sometimes the class is jumping with high levels of student engagement, other classrooms are boring, kids filling in worksheets or a teacher rambling, most classrooms fall somewhere in the middle.

After a presentation by Charlotte Danielson I asked, “Justice Potter Stewart in a Supreme Court decision wrote, ‘ …pornography was hard to define, but that I know it when I see it.'” Isn’t it the same with a good lesson?

Charlotte was not pleased.

An irony: Charlotte Danielson’s other book, “Talk About Teaching: Leading Professional Conversations,” 2009, discusses the conversations supervisors have with teachers after a classroom visit that are not intended to be evaluative.

Take six minutes and watch an excellent video (http://www.youtube.com/watch?v=NqeMKQhTcLQ).

Maybe we should just dump the entire APPR plan and coach teachers and principals how to talk with each other about teaching.

About these ads

4 responses to “APPR (Teacher Evaluation) Complexity: If Principals, Teachers and Parents Don’t Understand It, It Will Fail.

  1. we all know what happened to Bismarck in the end…When I hear pontificators such as Danielson say they are anti stansardized testing, Im always remindede of a long time staanding principal in a HArlem elementary school, who when asked if she would join the bandwagon to stop standardized testing, replied, I’ll do so when Harvard stops theirs…The danger in this whole TCR/Prin evaluation process, is that it plants in the hedas of misinformed parents that it is now going to be easy to just get rid of teachers and supervisors that they think should be gotten rid of. With that said, In all my travels to schools since my retirement, I have never before seen such a brain drain with regard to so many principals and assistant principals Ive encountered. THe common denominator amongst most is that they pretty much all come from Mayor Bloombergs School Leadership Academy..The same Mayor who fortunately for all of us (children included) will fall short of his ambition to UNION bust both the UFT and CSA..It is for this reason that in terms of next years mayoralty election when considering Ms.Quinn, you better believe the seed doesn’t fall far from the tree. Bill Thompson whose late wife was a Teachre, Ass’t.Prin and Prin is very proeducation and knows well how to work with our unions and above all, respects educators and the jobs they do. I know this for a fact.

    Like

  2. The role of education in a democracy is above all to produce citizens who can think and make informed judgments and decisions. Has anyone developed a quantifiable rubric for THINKING? How do you judge the quality of thinking on any level from pre-k to 12th grade? Andi if THINKING is that essential to the eduction/democrady continuum aren’t we going to continue to “test” for the least important issues>

    Like

  3. The rebirth (or redefinition) of the word “rubric” coincides with the birth of the BS that is now controlling education. Go looking for what a rubric is—everything from a “drop cap in red in an ecliastical writing” to a holistic way of assigning a grade. This mishmash of a word virtually defines what the for profit sector of education has made of the schools, and of public education.
    First, make a mess. Decree that every child is equally capable of learning in the exact same time. If you can’t sell that lie, change the language. Then change the standards. Then send in a pile of 30 day TFA wonders to show how anyone can teach. Add Geoffrey Canada, Michelle Rhee, Eli Broad, Walton’s, Gates, Bloomberg, and top it off with a few Governors, and self aggrandizing “superintendents”, chancellors, mayors, Ed Secretaries, Presidents, etc. and you have a “rubric” for the end of the melting pot that was public education in the USA for over a hundred years.

    Rubric my a**!l

    Aside from his use of the “R” word, I agree with Paul.

    Like

  4. What about the new after school program for next year? Private contractors will meet with students after school in BOE buildings while the teachers go home. Schools have to opt in-after some pressure from the principal(oops sorry,school leader)-to the program. What happens to the regular after school programs in these schools? Bloomberg is a genius. He found a way to eliminate overtime for teachers while giving one of his buddys a wonderful new contract. Can he make it system wide before he is done? Opt in suckers!

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s