Who’s Afraid of Data in Schools?

It’s hard to have a school-wide faculty meeting, or open an education magazine, without encountering the word “data.”  Once highly idiosyncratic, teaching has rightly become more and more a skill to be learned with outcomes to be measured.  I could not be happier about this move from teaching-as-witchcraft (or worse, merely bad theatre) to teaching-as-skill.

Those opposed to the use of data in schools are legion.  Some teachers say gathering data gets in the way of instruction.  Lots of stakeholders argue that it couldn’t possibly capture the complexity of good schools and teaching.  The most troubling objection is that data isn’t necessary to improvement; it’s somehow just a sideshow for accounting fetishists.

A recent post by Robert Wachter in New York Times puts it like this: “the objections [to using data] became harder to dismiss as evidence mounted that even superb and motivated professionals had come to believe that the boatloads of measures, and the incentives to ‘look good,’ had led them to turn away from the essence of their work…. The focus on numbers has gone too far. We’re hitting the targets, but missing the point.”

Then, quoting a “hard-nosed scientist” and measurement expert about how to improve the helping professions, Avedis Donabedian, Wachter claims he said on his deathbed: “The secret of quality is love.”

There you have it: love.  Retool, everyone!  Make loving schools and the rest falls into place.

Hardly.  Under the rubric of “love,” lots of pretty awful things happened: low expectations, damning “ability” streaming, corporal punishment, whole-language instruction, and on and on. Yes, teachers should be loving: but we know that because ample meta-analyses suggest that teacher-student relationships have a huge impact on student outcomes.  That’s data.

Why We Should Embrace Data

“Count something.” That’s the advice of Atul Gawande, surgeon, writer, health care reformer.  I am fond of borrowing his ideas, and this is no exception.  We ought to be counting more, not less; we ought to be asking research questions in our own schools and classrooms; we ought to be finding the best evidence about the effectiveness of our individual schools and making decisions based on it.

Ask Yourself: Without “Data” or “Information” or “Evidence”…

… How would you know if your classroom or school is functioning at its best?  Rather than gathering data, would it be better to merely guess?  Or perhaps listen to the loudest teacher complaining in the faculty lounge?

… How can you find improvements?  How do you know if your students are being well-served? How do you know if program changes are aiding student learning or the opposite?

Models of Using Data That Are Likely to Work:

– Validity: Measure what is valued instead of valuing only what can easily be measured

– Balance: Create a balanced scorecard

– Insist on credible, high quality data that are stable and accurate.

– Design and select data that are usable in real time.

– Develop shared decision-making and responsibility for data analysis and improvement.

(This list is taken from Andy Hargreaves, sometimes quoted as being opposed to the use of data in schools.)

Imagine This

Using the above criteria, I suggest that each school (or department within) embrace as their operating principles – something they just do as a matter of course – a set of measures.

ImagineColleagues getting together to decide, informed by stable accepted research (yes, it does exist) and professional judgment, what things really matter.  A few possibilities to illustrate: student engagement; preparation for the next step; effective instruction; and so on.

Then, consciously asking: how can we collect data – qualitative and quantitative, in valid and practical ways – to figure out how we are collectively doing across these domains?

Then, embracing an ongoing effort to track and discuss the data to improve programming or approaches.

The Scientist in the Room

I’m fond of saying that the best teachers are scientists in the room, looking for data about how the students are doing. That data could come from a lot of different sources: it could come from examining exams after students have written them; it could come from moderated or parity marking with colleagues; it could come from student surveys; it could come from student exit interviews; it could be routine (brief “stop/start/continue” surveys for students at the end of units). The more conscious we are of the information we base decisions on the better our decisions are likely to be.

The important thing is that we count, measure, and gather information, from a variety of sources to paint the most accurate picture of what we’re doing and how well it’s working.  We need to start now.  And we need to make it part of our regular practice.

We need more data, not less.  It needs to be collected efficiently and routinely, used consciously and conscientiously, for the perpetual betterment of our schools.  People have said the use of data harms children; I argue the careful use of data is the best defence against a complacent or dysfunctional approach to schooling.

If we won’t tolerate the random best guesses of heart surgeons, why would we demand any less from those that teach?

(Photo: JD Hancock)

Results, Not Hours, Matter

A recent edition of Planet Money addressed a topic near and dear to the hearts of all teachers: the question of long working hours.  In lots of fields, but maybe especially in teaching, we equate long working hours with “doing a good job.” But there are lots of reasons to question this idea.  Steve Henn, a reporter on the show, explains with a story:

One of my favorite economists, Dan Ariely, tells this story about a locksmith. When the locksmith was new at his job, when he was an apprentice, he took a really long time to open a lock. And people saw him working away, struggling, really having a hard time. And often they’d end up giving him a tip. But then when locksmith got better at his job, when he got so good at his job he could open pretty much any lock in just a minute or two, then his customers started complaining. They were like, you want $200 for that? This took you, like, 30 seconds…

Maybe hard work is irrelevant. Maybe what should matter is what we create. Maybe companies should be measuring our output and not keeping track of our input. What would happen if you ran a company based on that idea? What would that look like?

The Scope of the Problem

Ask most teachers, and they will tell you: teaching requires long hours.  

The BBC reports that in Britain, teachers work between 55.2-63.3 hours per week, a number that includes teaching time, in-school non-teaching work, and work that gets taken home.  The Guardian more or less echoes this view, with teachers responding that they frequently work 8-10 hours a day, plus time during evenings and weekends.  And the 2012 National Study on Balancing Work, Life and ­Caregiving in Canada: The Situation for Alberta Teachers makes the case that teachers in that province work an average of 60.8 hours per week, 10 hours more than the general population of professional workers.

Any teacher has probably heard many a humble-brag along these lines.  The comment, “I spent my entire weekend marking…!” serves at once as a complaint, a brag, and a reinforcement of the narrative: good teachers work long hours.

I can recall, early in my career, a senior leader of mine telling me that the only way to teach well is to “skip watching the late show”; if you had an evening to yourself, you couldn’t be a very effective teacher.

What’s to Be Done?

Planet Money offers an interesting, if cautionary, tale.  They profile practices in the tech world where output can be measured, and tentatively conclude that the results of your work matter more than the time spent doing it.  Sounds reasonable, but as I’ve argued before, in many fields – schooling one of them – it is hard to develop an easy measurable metric for the value of a teacher’s work.

That need not mean we abandon the notion of valuing teachers by their results entirely; there are lots of ways we can reward effectiveness and discourage merely “spending time.”

How?  Three Humble Proposals:

1. Let’s change the conversation in schools.  Let’s reduce the number of times we praise faculty for merely being there, and find more and more interesting ways to reward the fruits of their labour.

2. Speaking of which, let’s move towards a more evidence-based practice.  We’re not going to be able to measure lots and lots of important aspects of a teacher’s day, but we can start looking for some.  The challenge of measuring our effectiveness is central to any improvement regime we want to make.  And evidence can come in lots of forms.  Let’s think broadly about this important aspect of perpetual betterment.

3. Let’s focus on what really matters.  I’ve written before about the need for priorities in schooling.  We too often think changing the carpet can achieve results.  The problem: these less-than-effective-efforts all take time.  Imagine we could simply stop doing some of the things that don’t matter much – how much time could we free up? 

Teaching will always be a difficult job – what’s required is infinitely complex.  But surely we can stop thinking that effective teachers are merely the ones who work longest, or who take home the biggest bags of marking.

Image credit: Kevin Dooley

Unschooling – Again

It’s that time of year again. By “that time of year,” I don’t mean the insidious back to school displays to tide over retailers until Halloween and Christmas, though that is also true. It’s the time of year when we ask ourselves: why bother with schooling?

I’ve written before about what some call the unschooling movement, a loose collection of folks who hold that we deprive kids of authentic, meaningful, and creative experiences in our grand confinement of young people in millions of schools across the world. The cousin of unschooling, homeschooling, has similar issues with modern school systems.

Here is an interview from yesterday’s Globe and Mail. It’s with Zander Sherman, “Home-schooled until the age of 13, he was the odd man out when he finally joined a public high school, a vegetarian who played classical guitar, read his grandfather’s Marxist literature – and found himself wondering about the strange entity called ‘school.’”

I’d like to take on Sherman’s central claims. Of course, Sherman is a well-meaning, intelligent, and insightful person. But he repeats claims often trotted out about the schooling system, and I think they need some exposure.

Claim: “Most people look at the specifics – standardized testing, the number of homework hours a week, teacher tenure – but not the bigger issues. What is an education? What are we supposed to take away from it? As a home-schooler, though, I felt like an outsider, like I didn’t necessarily belong. At the time, it was kind of excruciating, but in retrospect I was able to look at this thing called “school” with fascination and curiosity.”

I think many people do obsess about what we supposed to take away from formal schooling. There are heated policy debates all the time (full year kindergarten, anyone? Homework policies?), politicians running on education platforms (Ontario’s premier styles himself along educational lines first and foremost, as did Davis and Robarts to a lesser degree before him), and discussions around dinner tables every day. As for the claim that he felt like an outsider, I grant that schools can be mean places – and to their detriment. But so can any important public institution. The remedy is not the destruction of schools.

Claim: “Schools have historically turned out citizens and voters; today, though, you could say we’re focused on human resources – schools have become standardized, and that’s because it makes for a good labour pool, it’s convenient for the economy.”

Writers like Michael Apple have made this case, too. But in response I want to develop a line of thinking that suggests the school-as-training-for-jobs is more complicated than both of them suggest. First, schools are usually accused of not preparing students for the world of work; usually, schools are seen as irrelevant wastes of time, a theme Sherman himself flirts with. I think the schooling system, in its insistence on the importance of literacy and numeracy, with mandatory exposure to liberal arts, physical education, and science is a good balance of the exact kinds of things nearly any parent would like his or her child to experience. I would like more specific examples of instances our schooling system is “convenient” for capitalism. Second, to the extent that schooling is directed at employment (there is a half-year course in grade 10 in Ontario, Careers, that helps students prepare for interviews and make resumes), it is quite reasonable.

Claim: “Education should be about instilling a sense of wonder and a love of learning. If people aren’t galvanized by curiosity, what’s the incentive to go to work?”

I grant that there is a difference between formal schooling and education, and that not all moments of schooling are about nurturing curiosity. Though, the system has been stressing a sense of wonder and curiosity for a long time – and has, at least since 1950, lamented the perceived lack of it (see the Introduction to the 1995 Royal Commission, For the Love of Learning, for a brief summary of both the Hope Commission of 1950, and the Hall-Dennis Report of 1968). Our many Royal Commissions and reports over the past 60 years indicate we aren’t as unthinking as Sherman would argue.

Claim: “Finland is a great example. They don’t value standardized tests (although they perform well on them) and there’s less schooling-per-year than elsewhere. Students learn, then bundle up and go skiing. It’s a wonderfully eccentric system.”

There has been quite a bit written about Finland and its “eccentric” system. It’s hard to separate the truth from the hype, but let’s grant that it’s a high-performing system that serves its students well.

What concerns me is the general opposition to a systematic and standardized approach. In all professions, practitioners have benchmarks and protocols and standards bigger than their own offices. In medicine, doctors follow international guidelines; why should teachers not benefit from the collected work of a hundred years of research into teaching and learning?

If we didn’t collect standardized data on student performance on reasonable and accurate measures of our major priorities (literacy and numeracy), how would we know if we were doing a good job? Teachers in individuals classrooms (including yours truly), often lack the perspective to be able to objectively determine their students’ success. In Ontario, our approach to gathering data across the whole system allows us to see if students are learning or not. How would we ever improve the system if we didn’t know such basic data as how many of our students can read?

Claim: “Teachers in Finland are venerated above doctors and lawyers. Why can’t we look at our own teachers the same way? It’s totally baffling.”

A lot of studies have pointed to the lack of respect in the teaching profession as a reason lower-performing undergraduates enter the field, and I think there is a lot of truth to the lament that teachers aren’t esteemed enough. But surely the appropriate response is not to get rid of system-wide data collection and merely increase the amount of nordic skiing. The path to greater respect, at least if medicine is any guide, involves greater transparency, rigorous standards, and the dedicated pursuit of meaningful goals – in the case of teaching, goals like literacy and numeracy. I think teaching is where medicine was in the 19th century; on the way to professionalization through the increasing use of evidence-based techniques, not snake oil.

Claim: “I think a growing number of educators are disillusioned with international comparisons. They often put the economy first – these are not necessarily the subjects that make for the best education. These countries are at war to be economic superpowers, and math, technology and engineering are the sectors that generate the most capital.”

First we should emulate Finland because of its high performance on international standardized tests, but we should also abandon international comparisons. Which is it?

And again, a variation on the claim that the economy drives the curriculum – at least, more than it should. In Ontario, the highest number of high school credits needed is in English, hardly a capitalist bastion. The second highest? Math. Then science.

Is it the case that math and science have been turned over to General Electric? Hardly. Corporations continue to complain that our school system is not geared enough to the needs of the economy.

Claim: “I’m currently working on an article about the importance of Latin and Greek. In the schools of yesteryear, knowledge of the classical languages was part of a pedagogy known as ‘formal discipline.’ The idea was that the human brain is a muscle; learning Latin and Greek gave the brain a workout, students’ minds were toughened, sculpted.

“In the 20th century, the curriculum no longer focuses on simple knowledge and wisdom, but what’s required for the work world.”

How would we define “simple knowledge and wisdom”? While I adore the classics, and have taught ancient history and philosophy throughout my career, the path to greater relevance is teaching Latin and Greek?

Claim: “I was home-schooled for creative reasons. But many home-schoolers are from religious families, and I think the temptation there can be for parents to indoctrinate instead of developing inquiring minds.”

So, his parents taught out of a love of creativity, but the rest can’t be trusted to. In light of this claim against homeschoolers, what’s to be done?

“I like what I see at the local level, when teachers take things into their own hands. One of my best friends is a public high-school teacher. Every day he practices what I preach: He chooses material that engages his students – that gets them excited and curious. He also avoids an emphasis on testing, grading and data in general. That’s what excites me most.”

All teachers can currently do this. There is no prohibition against it, nor has there been much restriction over day-to-day curriculum for several generations (in Ontario, at least). There are no daily suggested lessons in the slightest. Teachers have a tremendous degree of latitude over their daily practices. And of course, no student goes to school at a system-level – every last one is in a local school and the daily experience is made up of relationships with (mostly) caring teachers and peers. (Teachers, a highly unionized bunch, are unlikely foot soldiers of creeping capitalism.)

I still object to the treatment of “data” here. I think it’s important to know how your students are doing, ideally every class period. But that does not mean – in the slightest – that this data collection is only in paper-and-pencil tests. Talking, as any psychiatrist or journalist can tell you, is also data. So is debating. So is conferencing with a small group of students. As is when a student paints in art class. This is all data towards the same aim: namely, to know how we are doing.

If we’re not assessing our students, how do we know if they can read or write, or have the wisdom that Sherman is fond of?

(And again on standardized tests, Ontario has a standardized test – with no individual accountability for individual students or teachers – in grades 3, 6, 9 and 10. Hardly every day are the students exposed to the dangers of these insidious tormentors.)

School is there to do what not every parent can: instill in young people the skills that we have deemed important through our democratic process. Currently, that is literacy, numeracy, with some exposure to science, and a smattering of liberal arts and physical education courses. Evidence of the undue influence of capitalism is hard to find.

And while alternatives to the current system exist, of course, how could they be efficiently deployed? Can everyone be homeschooled? As I’ve written before, the wholesale opposition to modern schooling is the prerogative of the wealthy. Universal, government-funded schooling has been transformative for those not born into wealth. I say we celebrate that success, while dedicating ourselves to improving the system further through a systematic approach guided by – gasp – the most trustworthy data we can find.

A Contagious Vagueness

Last year, Salon ran an interview by Alice Karekezi with a New York City public school educator and curriculum advisor Diana Senechal. Senechal had recently released a book, The Republic of Noise. Her critique of some standard educational practices is intriguing, and while not empirically verifiable, rings true. (I am definitely going to be stealing the phrase, contagious vagueness.)

Below is an abridged version of the interview. (The whole thing can be found here.)

What’s your definition of solitude?
The idea of solitude as an attribute of the mind goes back to antiquity. The Greek Stoic philosopher Epictetus distinguished between a negative sort of isolation (helplessness, removal from others) and the strength that comes from relying on one’s own mental resources. Quintilian wrote about the importance of overcoming distractions through mental concentration and separation. “In the midst of crowds, therefore, on a journey, and even at festive meetings,” he wrote, “let thought secure for herself privacy.”
Solitude is not about being in a hut out in the woods or being out in the desert or living without other people around. I define solitude as a certain apartness that we always have, whether we’re among others or not. It is something that can be practiced — maybe to think just on one’s own, even when in a meeting or in a group and so forth — but that also has been nurtured by time alone. So there’s an ongoing solitude that’s always there, and there’s also a shaped or practiced solitude, which requires both time alone with things, to be thinking about things and working on things, and time among others when you nonetheless think independently.

You’re critical of certain educational philosophies in practice in schools today, especially the workshop model. Why?
The workshop model has an emphasis on group work and a de-emphasis on teacher presentation. What happens is the teacher is supposed to give a mini-lesson which is about 10 minutes long. From there students are supposed to work in groups on something related to that mini-lesson, sometimes independently, but most of the time in groups. At the end they are supposed to share about what they learned. This was mandated across the board, across the grades and subjects, in many schools. Every lesson is supposed to follow a workshop model. (Of course some schools were a little bit more flexible about this than others.)
The problem with that is that the workshop model is very wonderful for certain lessons and topics, but when you apply it across the board, you are constraining the subject matter. You need a variety of approaches in order to deal with a topic. You may need a lesson where the teacher gives an extended presentation to give the students necessary background. Or an extended discussion. For instance, the students may have a project that they will have to do together, but they have to work on their own to build up to that point.
Also, schools have put an enormous emphasis on skills – or what are called skills – at the expense of content. This has been going on for decades. No one wants to specify what students should read, but they say that they should be analyzing and comparing and contrasting. Well, none of this has meaning unless you know what it is you’re comparing and contrasting or analyzing. What happens is, students write essays that show that they haven’t read very closely, and yet this passes because it meets the checks on the checklist: that it has the right number of paragraphs; it has an introduction, body, conclusion; it seems as though they’re comparing something with something. There is a contagious vagueness because we don’t specify what we’re talking about and what students should learn. We then encourage in them a certain vagueness and carelessness. The problem perpetuates itself, and it turns up much later when students enter college and don’t know how to write a coherent essay. Well, the reason this comes up is that they’re in courses where they’re expected to read on specific topics, and that’s where things fall apart and it’s no longer about the rubric.
So the problem lies in the idea of putting the model above the actual subject. You have to think about the subject and think about how you’re going to bring this to the students, and think about the type of lesson that will do that best. Often you’ll find that you need a combination of types of lessons.

You write that we “mistake distraction for engagement”? How so? How does it affect even mental cognition?
I’m not a psychologist, but in the classroom and in many discussions on education, what I see is an emphasis on keeping the students busy from start to finish. Not letting a moment creep in where they don’t have something specific to do, something concrete where they are actually producing something. So if you keep them busy, busy, busy, and doing something at every moment, then supposedly they’re engaged. And when supervisors walk into classrooms and look and see the students writing and turning and talking, their conclusion is “Oh! What an engaged class!” The problem with that is then students don’t learn how to handle moments of doubt, or moments of silence, or moments where they have to struggle with a problem and they can’t produce something right on the spot. So, the students themselves come to expect to be put to work at every moment. If you want to give them something more difficult, you have to expect a little uncertainty.

In Teaching, Peace is Every Step

I’ve recently enjoyed reading Peace is Every Step by Thich Nhat Hanh, Buddhist monk and popularizer of many of the modern traditions associated with mindfulness. Anyone who’s spent any time in a school at all usually begins his description of his experiences in emotional terms – “I felt miserable… I loved my music teacher… I hated gym class…” – and so I’ve gotten used to conceiving of schools in emotional terms as well as rational ones.

This brief passage serves as a reminder of an important lesson about human interactions – one that might remind teachers that merely pushing harder does not always result in the changes we’d like to see in our pupils.

“When you plant lettuce, if it does not grow well, you don’t blame the lettuce. You look to the reason it is not doing well. It may need fertilizer, or more water, or less sun. You never blame the lettuce. Yet if we have problems with our friends or our family, we blame the other person. But if we know how to take care of them, they will grow well, like lettuce. Blaming has no positive effect at all, nor does trying to persuade using reason and arguments. That is my experience. No blame, no reasoning, just understanding. If you understand, and you show that you understand, you can love, and the situation will change.”

Voodoo Teaching

It must be a rare person who hasn’t seen this clip from Ferris Bueller’s Day Off. Of course, it’s funny. But also absurd: students in such a class wouldn’t be nearly so quiet. The desire for students to find engagement in something is stubborn – if the explicit lesson isn’t doing it, then socializing will.

Still, a great clip. And as an adult who knows the answers, enjoyable to watch.

On Knowing, Not Just Saying

Readers of this space know of my obsession with philosophical (really, epistemological) issues. Mostly, I am concerned with separating the wheat from the chaff in ideas – isolating the truly silly, from the potentially true, from the probably true, from the certain. In education, these categories tend to get tossed around together, without much reference to the important question: How Would We Know?

I recently had the pleasure of seeing John Hattie speak at the University of Toronto. To say it was refreshing is an understatement. I’ve been drawn to his work for a while now, and before I found it, was convinced such a project was possible.

(Here’s Hattie giving a similar talk.)

His interest is in measuring the effects of various inputs in the education system – inputs and interventions like team teaching, outdoor education, whole language versus phonetics, and practically every question in pedagogy. His premise: we can determine the effect size of all of the things we do in schools – how well they work. His conclusion: as a profession we should move towards those things that work well (have a large effect size), and away from those that do not.

Simple enough. I’m not able to provide a census of his detractors, but I know lots of people who critique his philosophical assumptions. They say that teaching practice cannot be reduced to such certainties. They argue that to strive to capture the subtle human interactions and nuances in teaching quantitatively is absurd. They sometimes argue that to gather such data is to take aim at low-performing schools and groups within them – they sometimes argue it’s culturally imperialist.

Obviously, they have warm hearts. All of us want students to do well, and most of us root for the disadvantaged. But until we gather reliable data on what works and what does not, we are continuing to impoverish our students. And what better way to ensure fairness in our society than by providing all students with the best possible teaching techniques and the best possible practices? And how else to do that then by measuring in the most precise way possible the effect size of what we do in schools?

Like Hattie, I am hostile to the idea that we are not professionals with something special to give. I reject outright the notion that “all teachers have their own way.” If an old-timer said, “Well, I hit the students; that’s how I get them to learn,” we would be outraged. I don’t see how it’s much different to say we are all equally successful using whatever techniques and approaches we “feel” are right. (Also, if teaching were so much based on whim, we would let absolutely anyone walk in and teach our classes; we do not, evidence we think it matters who teaches and how it is done.)

But perhaps the most satisfying element of the whole thing is the humility its adoption would bring to our field. Teaching suffers from a strange paradox of ego: on the one hand, most teachers feel like imposters, and denigrate the value of their practice; on the other, many teachers act the role of Superteacher, where everything he or she does is magical. What teacher hasn’t bristled in the staff meeting where one of their colleagues bellows, “Well, in my class, students love doing X,” or “I’ve never had that problem in my class…”, or “My students learn best when…”? I always want to ask, “How would we know?” and get a response more fulsome than “Because I’ve been teaching for 19 years, and I just know.”

Measurement projects like Hattie’s sweep all that nonsense away by asking, “What is the effect of our labours?” We can know, within some margin of error, what works and what doesn’t. At least, if we can ever know at all, it will be with an approach like Hattie’s, not our gut feelings and egotistical rantings. And in that kind of regime is comfort – it can depersonalize teaching somewhat, diminish the notion that teaching is a cult of personality, or a kind of mystical alchemy. Some approaches work better than others; let’s determine those, reliably, use them more often than not, and continue to measure the effects of our work – forever.

I dream of a rigorous measurement approach in a school setting, a unit of organization too small to hide in. You would probably need to set up long-term measurement indicators – perhaps a few basic assessments used for years within each grade or course, evaluated with clear rubrics and exemplars, and copies of old student work kept for years – and determine if students are improving by virtue of our efforts, and of course, by how much. (It isn’t enough to merely improve: as Hattie points out, we need to know the magnitude of the improvement. A student in any class will achieve some level of improvement over the year just through maturation.) Other indicators I like: success after high school, student feedback,

We could finally, without reference to our own whims, begin to address genuine “best practices” in our schools. Does team teaching work at our school? Let’s check this year’s assessment and see. Are students benefitting from the Advanced Placement regime? Let’s see how our graduates have done over the past 10 years and compare that number against our graduates from the pre-AP days. Is our program rigorous enough? Let’s gather data from 1st- and 2nd-year students in post-secondary studies. It is for these reasons I’m, in principle, a fan of large-scale assessments like the EQAO.

A proper measurement regime would provide some justification for the claims we make about our schools, our classes, our practice. In fact, it’s the only thing that ever has, and ever will. Without it, merely the loudest voice in the room wins.

Flip this Video

I’ve recently written with some suspicion about the promised revolution the Flip the Classroom folks like Salman Khan have been preaching. I said that while the videos are interesting, and promising, the evidence of a revolution is hard to see. I think the word revolution should be saved for actual revolutions.

A colleague passed this video onto me this week.

It is a beautiful and inspiring video. It is the clearest explanation of the value of this kind of resource. Or is it? In fact, it seems to imply the premise of flipping – the practice of having students watch videos as homework to allow more higher-order tasks in class – is something altogether different. As the folks at TedEd put it:

“Flip” is meant to indicate that teachers of all stripes can propel/catapult/slingshot the video to a wider audience. And “flip” is also a reference to a nascent and evolving teaching method called Flip Teaching.

I smell a revolution in trouble. Their own buzz word now has two meanings.

Of course, the videos are often superb. And who could complain about some beautifully created lessons by some of the world’s great minds? (Its stated mission: “To capture and to amplify the voices of the world’s greatest teachers.”) I have shared many dozens of these clips with friends, and yes, used them in class – and as homework.

But let’s call it what it is, at least for now: better filmstrips than a generation ago. The evidence to prove me wrong would be simple: let’s examine the usage stats for the videos. If the usage is high among 14 year-olds, then I’m wrong. It is a bona-fide revolution. Young people clamouring for videos of middle-aged intellectuals doing Power Point presentations in front of other middle-aged intellectuals.

I think you’d find something else in the usage stats, though: an incredibly low number of young people visiting when not prompted by their teachers. And when they are prompted, a huge number of them watching the first few seconds, finding it boring, then clicking away.

It might just be, as I’ve argued before, we expect a revolution because the videos speak to us as adults. I’m not completely convinced they speak to those we want to reach. If they do, why aren’t the masses of teenagers showing us the videos, having found them the night before, on their own, as they do with memes? Why aren’t Ted Talks memes for kids?

(Whose?) Character Education

James Q. Wilson
A little while back, David Brooks, from his usual column in the New York times, praised a recently-deceased political scientist, James Q. Wilson – an academic most famous for his “broken window” theory of crime. Wilson, a professor at Harvard for most of his working life, had a larger impact that that single theory, though; the core of Wilson is his writings on character, says Brooks.

‘When Wilson began looking at social policy, at the University of Redlands, the University of Chicago and Harvard, most people did not pay much attention to character. The Marxists looked at material forces. Darwinians at the time treated people as isolated products of competition. Policy makers of right and left thought about how to rearrange economic incentives. “It is as if it were a mark of sophistication for us to shun the language of morality in discussing the problems of mankind,” he once recalled.’

And later in the NYT piece:
‘When Wilson wrote about character and virtue, he didn’t mean anything high flown or theocratic. It was just the basics, befitting a man who grew up in the middle-class suburbs of Los Angeles in the 1940s: Behave in a balanced way. Think about the long-term consequences of your actions. Cooperate. Be decent. He did not believe that virtue was inculcated by prayer in schools. It was habituated by practicing good manners, by being dependable, punctual and responsible day by day.’

Many school boards (and certainly many independent schools) have been in the embrace of character education for some time now, and – good or bad – some thanks goes to Wilson. Many fine schools have used the discourse of character to redefine their programs and restructure their school cultures. Some hold celebrations of one particularly important character trait each month.

Ontario, in particular, has turned to it in the past decade or so. As a jurisdiction with over 12 million people, one that spends a lot of money on education, and is possessed of many well-respected university faculties of education, Ontario’s influence (in Canada and more broadly) is weighty. And on this issue, it has led.

But an examination of the question is surely complicated. After all, if Wilson discounts ‘Marxist’ (though certainly more than Marxist) ideas of materialism, we’re left with little to explain divergent crime stats besides cultural and character matters. People steal not because of inequity, but because of personal failing? This might be so (I doubt it, but it’s not logically impossible), but as casual supporters of character education we often forget about the consequences.

Then there is the question of whose character – from which culture should we find the appropriate character traits and attributes? The Ontario government writes, “Character development is the deliberate effort to nurture the universal
attributes upon which schools and communities find consensus.” Character development is a-cultural, they say.

Are noble character traits universal, as the Government of Ontario suggests? In the words of Wilson, what does being ‘decent’ mean – and does it mean the same thing to all people? If contemporary character development isn’t religious education (something both Wilson and secular school authorities suggest), then how do we explain its similarity to Protestant values? I was raised with such values, and do admire them. But it seems hard to explain its coherence with a very sacred ideal: equity. Indeed, “respect for diversity” is another heading in the same curriculum document.

Is it possible that students who come from some cultures express respect differently than our own? That they might manifest as lacking in character, when other forces might be at play? Is there a tendency, a conservative one at that, to blame kids for things beyond their control – either material or cultural? (In this case, independent schools are on more solid ground by virtue of the choice involved.)

What would it mean, in real practice, to achieve the following government mandate: “The multiple perspectives that exist within our communities demonstrate the need for school boards to be increasingly responsive to the needs and aspirations of their diverse communities.”

The general consensus – and the easy answer – is to ask the community itself to identify attributes to reinforce and inculcate. But that presumes several things that might not be true: first, that the community speaks with one voice; second, that the voices heard at meetings represent all voices; and third, that the community is right. Where I grew up, the community was often vocally opposed to gay rights. It was a place where in the local Santa Claus parade, one ‘float’ was a man in blackface in a white Ford Bronco with a sign on the side: “OJ’s Back for Christmas.” This was 1995. That community, at that time, often showed poor values. Some of us would have benefitted from a defence from community norms.

I think of Catholic schools in Ontario and their discomfort with homosexual relationships. Is being gay (or expressing it at prom, say) bad character? If so, says who? And if not, what do we say to the folks who are upset by an acceptance of gay relationships? The complication arises here: it is easy to merely assign a judgment to behaviour in accordance with a shared understanding. But when various factors are at play – community values, the Constitution, school board directives – and when a ‘shared understanding’ is quite unlikely, any declaration that character matters is easy seems somewhat foolish.

Shame and Blame

It is a tired axiom that teachers get blamed for things that aren’t always their fault. And while that matter deserves exploration on its own, for today let’s leave it at a judicious reprinting of two worthy commentators.

The first is, William Johnson, a special education teacher from Brooklyn, NY, who writes in the New York times:

As you might imagine, my job can be extremely difficult. Beyond the challenges posed by my students, budget cuts and changes to special-education policy have increased my workload drastically even over just the past 18 months. While my class sizes have grown, support staff members have been laid off. Students with increasingly severe disabilities are being pushed into more mainstream classrooms like mine, where they receive less individual attention and struggle to adapt to a curriculum driven by state-designed high-stakes tests.

On top of all that, I’m a bad teacher. That’s not my opinion; it’s how I’m labeled by the city’s Education Department. Last June, my principal at the time rated my teaching “unsatisfactory,” checking off a few boxes on an evaluation sheet that placed my career in limbo. That same year, my school received an “A” rating. I was a bad teacher at a good school. It was pretty humiliating.

Like most teachers, I’m good some days, bad others. The same goes for my students. Last May, my assistant principal at the time observed me teaching in our school’s “self-contained” classroom. A self-contained room is a separate classroom for students with extremely severe learning disabilities. In that room, I taught a writing class for students ages 14 to 17, whose reading levels ranged from third through seventh grades.

When the assistant principal walked in, one of these students, a freshman girl classified with an emotional disturbance, began cursing. When the assistant principal ignored her, she started cursing at me. Then she began lobbing pencils across the room. Was this because I was a bad teacher? I don’t know.

I know that after she began throwing things, I sent her to the dean’s office. I know that a few days later, I received notice that my lesson had been rated unsatisfactory because, among other things, I had sent this student to the dean instead of following our school’s “guided discipline” procedure.

I was confused. Earlier last year, this same assistant principal observed me and instructed me to prioritize improving my “assertive voice” in the classroom. But about a month later, my principal observed me and told me to focus entirely on lesson planning, since she had no concerns about my classroom management. A few weeks earlier, she had written on my behalf for a citywide award for “classroom excellence.” Was I really a bad teacher?

In my three years with the city schools, I’ve seen a teacher with 10 years of experience become convinced, after just a few observations, that he was a terrible teacher. A few months later, he quit teaching altogether. I collaborated with another teacher who sought psychiatric care for insomnia after a particularly intense round of observations. I myself transferred to a new school after being rated “unsatisfactory.”

Behind all of this is the reality that teachers care a great deal about our work. At the school where I work today, my “bad” teaching has mostly been very successful. Even so, I leave work most days replaying lessons in my mind, wishing I’d done something differently. This isn’t because my lessons are bad, but because I want to get better at my job.

In fact, I don’t just want to get better; like most teachers I know, I’m a bit of a perfectionist. I have to be. Dozens and dozens of teenagers scrutinize my language, clothing and posture all day long, all week long. If I’m off my game, the students tell me. They comment on my taste in neckties, my facial hair, the quality of my lessons. All of us teachers are evaluated all day long, already. It’s one of the most exhausting aspects of our job.

Teaching was a high-pressure job long before No Child Left Behind and the current debates about teacher evaluation. These debates seem to rest on the assumption that, left to our own devices, we teachers would be happy to coast through the school year, let our skills atrophy and collect our pensions.

The truth is, teachers don’t need elected officials to motivate us. If our students are not learning, they let us know. They put their heads down or they pass notes. They raise their hands and ask for clarification. Sometimes, they just stare at us like zombies. Few things are more excruciating for a teacher than leading a class that’s not learning. Good administrators use the evaluation processes to support teachers and help them avoid those painful classroom moments — not to weed out the teachers who don’t produce good test scores or adhere to their pedagogical beliefs.

Worst of all, the more intense the pressure gets, the worse we teach. When I had administrators breathing down my neck, the students became a secondary concern. I simply did whatever my assistant principal asked me to do, even when I thought his ideas were crazy. In all honesty, my teaching probably became close to incoherent. One week, my assistant principal wanted me to focus on arranging the students’ desks to fit with class activities, so I moved the desks around every day, just to show that I was a good soldier. I was scared of losing my job, and my students suffered for it.

That said, given all the support in the world, even the best teacher can’t force his students to learn. Students aren’t simply passive vessels, waiting to absorb information from their teachers and regurgitate it through high-stakes assessments. They make choices about what they will and won’t learn. I know I did. When I was a teenager, I often stayed up way too late, talking with friends, listening to music or playing video games. Did this affect my performance on tests? Undoubtedly. Were my teachers responsible for these choices? No.

The other voice is Bill Gates, responding to a recent decision in New York State, also writing in the Times:

LAST week, the New York State Court of Appeals ruled that teachers’ individual performance assessments could be made public. I have no opinion on the ruling as a matter of law, but as a harbinger of education policy in the United States, it is a big mistake.

I am a strong proponent of measuring teachers’ effectiveness, and my foundation works with many schools to help make sure that such evaluations improve the overall quality of teaching. But publicly ranking teachers by name will not help them get better at their jobs or improve student learning. On the contrary, it will make it a lot harder to implement teacher evaluation systems that work.

In most public schools today, teachers are simply rated “satisfactory” or “unsatisfactory,” and evaluations consist of having the principal observe a class for a few minutes a couple of times each year. Because we are just beginning to understand what makes a teacher effective, the vast majority of teachers are rated “satisfactory.” Few get specific feedback or training to help them improve.

Many districts and states are trying to move toward better personnel systems for evaluation and improvement. Unfortunately, some education advocates in New York, Los Angeles and other cities are claiming that a good personnel system can be based on ranking teachers according to their “value-added rating” — a measurement of their impact on students’ test scores — and publicizing the names and rankings online and in the media. But shaming poorly performing teachers doesn’t fix the problem because it doesn’t give them specific feedback.

Value-added ratings are one important piece of a complete personnel system. But student test scores alone aren’t a sensitive enough measure to gauge effective teaching, nor are they diagnostic enough to identify areas of improvement. Teaching is multifaceted, complex work. A reliable evaluation system must incorporate other measures of effectiveness, like students’ feedback about their teachers and classroom observations by highly trained peer evaluators and principals.

Putting sophisticated personnel systems in place is going to take a serious commitment. Those who believe we can do it on the cheap — by doing things like making individual teachers’ performance reports public — are underestimating the level of resources needed to spur real improvement.

At Microsoft, we created a rigorous personnel system, but we would never have thought about using employee evaluations to embarrass people, much less publish them in a newspaper. A good personnel system encourages employees and managers to work together to set clear, achievable goals. Annual reviews are a diagnostic tool to help employees reflect on their performance, get honest feedback and create a plan for improvement. Many other businesses and public sector employers embrace this approach, and that’s where the focus should be in education: school leaders and teachers working together to get better.