California Teachers: Hot or Not?

Value Added and Reduced.


This week, the LA Times published the findings of a study of teachers and effectiveness. 6,000 third-fifth grade teachers, from all over California, were rated and ranked from “most effective” to “least effective.” The top 100 were highlighted in their own column.

Bombshell! Teachers across California must be rattled. In a world where tenure has reigned supreme for decades, suddenly teachers are being reduced to numbers. A California fourth grade teacher can look herself up, on the LA Times website, and find her rank. And so can anyone else.

How did they decide who was Hot and who was Not?

The LA Times developed this ranking on one statistical value derived from one place: standardized tests. While not complicated, it does take a little explanation:
1. Researchers compared a student's performance on state-wide standardized tests from one year to next year.
2. The difference in scores reflected a higher score, a lower score, or the same score.
3. After each student's net change score was tabulated, the students were grouped by teacher.
4. Each teacher group's average net change score was determined based on these numbers.

This “Value Added” score, the Times argues, allows for anyone to see what “value” any teacher has added to any of his students. Simple. Clean. A raw piece of data-- just what we love.

Ah, but not so fast. Data's beauty is in the eyes of the beholder.

Those who challenge the data have some great arguments against the analysis. First, the data all hangs from the chain of the quality of the standardized tests. A 'good standardized test' has proven to be the the single hardest thing to build in the realm of student evaluation. Holy Grail hard.

Second, it's not that ONE test has to be terrific: the first year AND the second year test must give an accurate measure of a student's acumen. And then every subsequent test that is used in this manner. Furthermore, these tests must be built together in order to measure growth; the tests must work in concert in order to have comparative value.

Finally, the 'Value Added' score doesn't measure ability-- it only measures change. Take one example of the many scenarios that skew the data. Think about what happens when comparing a teacher with kids who start with an average of 95/100 to a teacher with kids who start out with an average of 45/100. The teacher with the better kids, weirdly, has a terrific chance of being ranked as a poor teacher.

However, those who support the data build a strong case, too. First, every teacher (and student) is measure using the same test. Subsequently, no matter how 'good' or 'bad' the test is, everyone is handled with equal fairness (and unfairness). This is true even in comparing first year to second year tests-- it's fair because everyone is measured with, theoretically, the same stick.

Second, no small number of students who had 'a bad day' will skew the data simply because all of the students took the test. Arguably, a nearly equal number of students would have had 'a great day' taking the test. The size of the sample smooths out the inconsistencies or anomalies. The more years this data is collected, the less the effect of accidentals.

Finally, supporters say, the data show that where a kid comes from (class, ethnicity, and gender) are far less important than the kind of teacher she has. This is good because we can finally separate the data about student achievement from the data about teacher achievement. The Value Added model focuses on growth as opposed to talent or natural endowment. (Richard Buddin, who wrote a paper about the data collection, loves to say the word 'endowment.' You have to love using that word earnestly. I implore you to work the word 'endowment' into conversations you have today.)

Find Buddin's work here http://www.latimes.com/media/acrobat/2010-08/55538493.pdf

These are just the arguments about the data and its collection. The hot and nasty arguments are about philosophy, cultural values, and professionalism. Rightfully, some of the argument centers on whether or not information about teachers' evaluations should be made public.

I'd like to offer a broader perspective.

The idea of ranking teachers is ridiculous. It doesn't serve to improve the profession; it only serves the media's love of bite-sized information that allows for thin conclusions. Henry I. Braun, in his treatment of Value Added Assessment, warns of the dangers of "casual interpretation" of data. The amount of ugliness that will follow the Times posting this information may very well swallow any other productive dialogue.

It takes a quality blindfold to pretend that something is rank-able just because it has a number attached to it. It's a shame that the solid journalism that supported this research has been shared in this way. The LA Times decision to rank the teachers has turned California into a Miss America contest. A beauty pageant isn't about beauty: it's about winners and losers. Revamping education will require a much wider, more informed view than that.

That said, evaluating teachers (like evaluating anything) requires some hard statistical data. Toss the ranking; keep the data. The “Value Added” Score provides non-anecdotal information about a teacher. For this reason, this statistical information, despite its potential flaws, is not only useful. It is important.

Stats like “Value Added” are essential. And, they are the future.

As you will read in my long-awaited final installment of “Scapegoats and Saviors IV,” at least part of teacher evaluation must come from a number like this. It cannot-- and should not-- be avoided.

http://www.latimes.com/news/local/teachers-investigation/
Check out the LA Times to read the article yourself. I'd love to hear what you think about it.


If you really want to dork out (and I highly recommend doing so), peruse the scholarly work (including that of Buddin and Braun) on how and why they did the Value Added model.
lookee here:
http://www.latimes.com/news/local/la-me-value-added-sources,0,6109096,full.story

Cartoons and Mirrors

A film review.


Once again, Pixar uses brightly colored playthings to smash us to pieces, only to glue us together again.

Prickly pieces have punctuated Pixar's productions in the past. In Finding Nemo, we find a desperate father absolutely devastated by the death of his long lost son. In The Incredibles, a family gets ripped to pieces by secrets that question identity and trust. Wall-E gives us harshly dystopian world in which an orphaned, trash picking robot is the only 'life' left on a planet destroyed by humans. And who can forget the brilliant opening sequence in Up? The perfect confection of boy meets girl leads unstoppably to the bitterness of loss and widowhood.

Dark stuff for kiddie movies. These people at Pixar aren't messing around.

This summer's Toy Story 3 avoids none of the bleakness, either. The end of childhood comes to toy-stewarding Andy, and never in the film do we see him happy to go to college. The focus remains on the toys, however, and here we find abandonment, attempted homicide, and the annihilation of a family.

I'm reminded of the unvarnished Grimm tales. Before Disney (ironically the parent company of Pixar) got a hold of a Grimm tale like "Cinderella," the story was filled with harsh consequences. For instance, the “evil” stepfamily abused Cinderella and constantly lied to her. Cinderella's name comes from the fact that she picked her food from the ashes in the fireplace- often for the sport of her family. Late in the story, when faced with a too-small glass slipper, one of the sisters cut off her own heel to make a gory fit. The prince recognized that Cinderella as the true owner of the shoe simply because “there was no blood in” the shoe when she slipped it on. Birds that spent the story helping Cinderella finish the tale by pecking out the eyes of the sisters, leaving them blinded for a life of darkness.

Yikes.

What makes a Grimm “children's story” tick are the abject desolation and violence that many stories of our time whitewash. Oh, sure, our stories have explosions and death; we love a good chase scene and showdown. But most of our stories often leave us where they found us, counting on the idea that all will be right in the end. The status quo will be preserved. The good guys (or girls) will win and everything will be returned to normal.

The storytellers at Pixar make no deals with us. They grasp that an authentic representation of the world must include irreversible consequences. On one hand, Pixar films seem to offer escape and bliss: we go to these movies expecting to have our heartstrings pulled while we giggle and laugh. On the other hand, Pixar films abuse us with the burdens of our sympathies: we lean forward, inhaling, worried that these little dolls/fish/robots might not actually survive how awful everything is.

We adults watching these films know that the world is an unforgiving place. Bad things happen to good people. Friends leave us. Homes are destroyed forever. People die. We know what's on the line for Nemo's dad when he breaks down in the face of how incredibly large and dangerous the ocean is. We understand that Wall-E's trash dump world is a reasonable, if avoidable, future to our own fragile green planet. And we know that Andy's loyal and determined toys, like all playthings, are all destined to be tossed off by a boy who outgrows his own childhood.

These Pixar characters are destined to be changed forever by their travails, left with permanent scars and irretrievable pasts. In Toy Story 3, Woody and Buzz and the toy gang go through a one-way door, forever leaving the familiar comforts of grown-boy Andy's bedroom. And Andy, too, goes through his own threshold, leaving his emptied bedroom for college. Andy, like his erstwhile plastic and fuzzy friends, will never be the same.

And we, watching from the chilly dark of our cineplex, recognize these primal forces at work. We know that these animations are merely ephemeral pixies, pretty things that glow from hard drives. (It's just a cartoon!) But something about the distance of it all-- the remove of the cartoon coupled with the remove of the cinematic experience, opens a window into the very center of us.

In the dazzling color of all, something wonderful happens. Through the unique alchemy of animation, Pixar makes us kids again. Witches are real, as are giants and monsters. We re-encounter the way we experienced stories as children. We look up at towering kitchen tables, looking through belt buckle perspectives and are reminded just how big and scary and thrilling this world can be.

Where nostalgia helps us shine up our memories, focusing on the good and the beautiful, Pixar's Toy Story 3 gives us the entire memory. We see mom's heartbreak at our newly emptied room. We remember our own favorite and worn toys from our childhood, knowing that these well-loved pieces are likely in a landfill somewhere.

And we remember that first, last time we looked at the place we grew up, that first time we left home. We clearly see that one way door that led from our technicolor childhood to our complex and ever-shifting adulthood.

For a moment, in that half light of the cinema, the fragility and durability of these animated toys pulls back the curtain. And the beauty is more than enough.