Drive to the east of the Georgian Bay of Lake Huron and you will see some of the oldest exposed rock on earth. Drive to a certain small town there one day about thirteen years ago, and you would have seen some venerable outcroppings of humanity. Mr. S., the guest of honor at a party I attended, was retiring after thirty-five years of teaching the same grade in the same room of the same schoolhouse. One of the guests was the teacher he succeeded, who had taught him in that grade in that schoolhouse. Does it get any stabler than that?
By contrast, one of the schools I taught at had a 25 – 30% annual turnover in students. Its student body was drawn from families with parents on the move. In such circumstances one does one’s best with the students who appear in class only to vanish all too soon. (Well, some of them couldn’t vanish soon enough, but a teacher is nothing if not patient. I sometimes wonder whether being a farmer and cattle baron was Job’s second career, he having learned patience during a first career in the classroom.)
Schools in the U. S. operate across this spectrum, but in spite of the difficulty it would encounter working across such a spectrum, and in spite of other difficulties of implementation, schools and districts are moving towards the “acceptance” of a “formula” that “grades” teachers’ “effectiveness.” I am referring to “value-added learning,” which is not a formula, does not grade, does not measure effectiveness of teaching, and is meeting widespread (though not enough) resistance. It is not a formula because the variables are not commensurable; it does not give an accurate grade because it does not evaluate things of an identifiable kind against a common standard; and it does not measure effectiveness because “effectiveness” here is the unjustified reification of sets of statistics into The Thing Effectiveness.
Take a number of students in a ninth-grade classroom in their first year of high school. At the beginning of the school year they take a test whose writers claim it can assess what they know in, say, English. In their second year, at the start of tenth grade, they take a test attached to the same claim. It is in some cases the very same test, kept without provisions for security from year to year. Call the average score of the 9th-grade class AS1 and the average of their 10th-grade scores AS2. According to value-added learning, if AS1 is less than AS2,“value” has been “added” to the test-takers’ growing minds. This “value” is deemed by the proponents of this idea to have been “added” by their 9th-grade teacher.
There’s some illicit deeming going on here. The late great biologist Stephen Jay Gould in his book The Mismeasure of Man warned against reifying statistics, that is, taking numbers and turning them into “things” such as “intelligence” or “value.” Alfred Binet was allowed to finesse a concern about the nature of intelligence when, asked what intelligence was, he was reported to have said, “It is what my test measures.” We might be charmed by such insouciant superbity if we did not know the mischief caused by the misuse of intelligence tests since Binet invented the first one. This time we must be on guard against the mischief, and the first question for the guard to ask is On what basis or authority is it asserted that the score on a multiple-choice achievement test is “value”?
In any case, this view of value omits to consider the long-term good a teacher can do for students—what a former colleague of mine called “seed-planting.” She thought, perhaps naïvely or unbusinesslikely, that a good teacher’s benefit to a student can be lasting or can unfold only in the fullness of time. Professor Barzun thought it wise to give a teacher’s instruction ten or twenty years to see what it amounted to. The present-orientation of “value-added learning” discounts all that. To my former colleague’s pedagogy its question is, “Why are you planting all those trees and perennials—why are you cultivating that joyous and varied field—when you should be concentrating on a patch of what can be sown, grown, and mown in the same year?”
Cut grass lies frail;/Brief is the breath/Mown stalks exhale.—Philip Larkin
Cut grass has worth:/Better mown field/Than tree-clad earth.—Value-added Learning
It also asserts that all “value” is to be imputed to the teacher with no control for other variables. Did the student do independent reading? Did a newly invented gadget distract a number of students from their studies? Did a curtailed school lunch program leave students hungry and unready to learn? No matter, says “value-added learning.”
But even if we unwisely take the assertion of “value” as given, we still have two problems. One is that tests like these typically measure what they claim to measure only to a certain degree of accuracy. Where is the “value” in such tests? If someone wanted to say that “our tests allow us to make some tentative claims with a certain amount of confidence,” we might find in that statement a trustworthy caution. But that is not what these tests will be asked to do. They will be asked to show an exact and definite amount of “value” in a class, whose addition to prior values will in turn yield exact determinations of teachers’ effectiveness.
The other problem is what I will call Big Mac Immodesty. The Economist publishes from time to time its Big Mac Index to the relative value of the world’s major currencies. This index’s inventors assert with tongue firmly in cheek that the Big Mac hamburger is a mini-test of that value. They advise their readers to take the index with a “generous pinch of salt.” Do the proponents of “value-added learning” approach their instruments with the same becoming modesty? Not at all. In the Big Mac Index at least a cow is a cow and a bun is a bun, but in “value added learning” what bits of pedagogical produce are fungible across years? No matter: they will yield their numbers and we will use them. Our Big Mac index is to be swallowed salt-free and uninspected.
Then we get to the problem not faced by Mr. S before his retirement. In his school there was little or no turnover of students from year to year. What about schools whose populations turn over rapidly? Regardless of whether scores go up or down from year to year, they will not be measurements of the same kids. In such cases how can they be said to add anything to anything? If a statistical fluke leaves a school with an influx of poor students to replace an outflow of good ones, are its teachers to be blamed when “value” is “subtracted”? What if a school is reported to have poor test scores, prompting an exodus of students to neighboring or charter schools? Evidence suggests that these educational migrants tend to be brighter and more motivated than the students they leave behind. “Value-added learning” will not report an exodus; it will report a decline in “value,” and the teachers “responsible” for this “value-subtraction” will be deemed “ineffective.”
We also have problems with Campbell’s Law (See my posting “℞ Stone Tablets”): testing is causing corruption, with schools scheduling outings for their Special Ed groups on Testing Day, subjecting poor students to “counseling out,” etc.
Mr. S. briefly touched on his relief that he was retiring when he did, this even though he survived educationist vicissitudes from the early ‘60’s to the late ‘90’s. (Another former colleague, when trying to achieve detachment from the ups and downs of his career in teaching, would sometimes look at me with a smile and say, “Dem vicissitudes!” Indeed.) I sometimes wonder what Mr. S. thinks of this nonsense, but more, I hope that he is having a delightful retirement, secure from depredations on his reputation and pay, and free at last of the bad consequences of educationist baloney.