Tag: balanced scorecard

This is Your Brain on Test Scores

Posted on | by Sam

There are two seemingly unrelated columns in today’s Opinion page of the New York Times that provide a crisp summary of where we stand in our current thinking about school reform — and where we need to go.

The first is a piece about charter schools in New York City, in which the editors reference “a national study finding that only 17 percent of charter schools offered students a better education, as measured by test scores, and that an astounding 37 percent offered a worse one.”

This is not the first time the Times has uncritically conflated something as comprehensive as “a better education” with something as singular as student reading and math scores. I imagine it won’t be the last. But it is, thankfully, a funhouse-mirror brand of “business thinking” that is on its death bed. Indeed, it hasn’t characterized actual business thinking for decades — ever since Robert Kaplan’s notion of the balanced scorecard first demonstrated the danger of focusing too narrowly on net income as a metric of overall success.

As I have said repeatedly, reading and math scores are valuable — and overvalued. Even KIPP, the poster child for exponential test score growth in high-poverty environments, recognized this when, a year ago, it shared the results of its own study that showed just a 33% college completion rate for its graduates. Since that time, as Paul Tough reports in his new book, KIPP has rightfully sought to round out its own portrait of a successful graduate by identifying a set of actual skills and habits its young people can use to successfully navigate the awaiting worlds of college, career and citizenship.

Ironically, neuroscientist David Eagleman calls for a similarly comprehensive vision on the same Op-Ed page. Writing about President Obama’s recent decision to invest in a multiyear effort to map the human brain, Eagleman makes a series of statements the Times editors would be wise to apply to their own thinking about school reform. “You can’t pull a piece of circuitry out of your smartphone and expect the phone to function,” he writes. “Looking at the brain from a distance isn’t much use, nor is zooming in to a single neuron. A new kind of science is required, one that can track and analyze the activity of billions of neurons simultaneously.”

What excites Eagleman is the potential to understand the brain as a system, and not just as a series of isolated parts. “While we have improved our ability to diagnose problems,” he writes, “we have yet to understand how to remedy them.”

The same can be said for our efforts to diagnose which school provides the “better education.” Now we just need to courage to admit it.

The Testing Carousel Goes Round and Round . . .

Posted on | by Sam | 2 Comments on The Testing Carousel Goes Round and Round . . .

Today’s Washington Post reports that the test scores of elementary school kids slipped this year after two successive years of growth, “a setback to Chancellor Michelle A. Rhee as she seeks to overhaul the city’s schools.”

No doubt, this news is being used by Rhee’s critics to point out that her particular brand of reform can’t bring the city the lasting change in its public schools that everyone desires. Meanwhile, Rhee responded to the news with equanimity. “We like to celebrate when we do well, and when we don’t, we have to take responsibility,” she said. “We have to own this and figure out how to move forward.”

Can I please make a wish to the education fairy and ask that this be the last of this sort of story I ever see? For those of us who believe that the best way to assess a school’s overall health involves a balanced scorecard of assessments, we can’t have it both ways — you can’t criticize Rhee for focusing on tests, and then lambaste her when those same scores are poor. It either is or isn’t a viable way to assess the health of a school.

In that same spirit, why aren’t folks like Rhee proactively diffusing these sorts of stories by getting out in front of the curve and releasing their own bundle of assessment measures, as a way to diffuse the potential power of the test scores when viewed in isolation? Rhee could do this immediately, without even getting into the contentious issue of using performance assessments. The city could stitch together an interim scorecard, made up entirely of existing measures (student and faculty absenteeism rates, student disciplinary data, graduation rates, a balanced set of course offerings, school climate surveys, and yes, test scores) and use it to educate the public about the many elements that go into a high-quality learning environment? Depending on what the data tell us, it might even lead to some insights that could drive future policy proposals. So let’s stop bickering over the wrong thing — otherwise, we’ll be stuck interminably on this basic-skills testing carousel, and forced to watch it go round and round while other countries are actively revising their education systems to become more effective at imparting higher-order skills and preparing children for the 21st century.

We can do better.

Data-Driven Decision Making . . . and Soccer?

Posted on | by Sam | 2 Comments on Data-Driven Decision Making . . . and Soccer?

Great timing.

A week after I wrote about what the World Cup can teach us about school reform, the New York Times published an article about the growing push for more detailed data in the relatively data-free world of professional soccer.

I am not, for what it’s worth, against the use of more sophisticated data in making decisions about how to improve the learning conditions for kids (or, for that matter, how to make better decisions on the soccer pitch). Who would be? In fact, I’ve written in the past about how a balanced scorecard in schools would help educators do their jobs more effectively.

That being said, I am very much against the glorification of data as a way to make extremely subjective, non-linear things — like learning how to use one’s mind well, or watching a collective burst of creativity and synchronicity that leads to a beautiful soccer goooooooaaaaaaal — into extremely objective, linear things for which we can appropriately plan and script out a desired, predictable response.

I don’t think it’s coincidental that this new push for soccer data is reported the same week as an announcement in my home city that Chancellor Michelle Rhee intends to significantly expand the use of standardized tests so that “every D.C. student from kindergarten through high school is regularly assessed to measure academic progress and the effectiveness of teachers.” What’s afoot in both instances is, on one hand, the (appropriate) desire to take human ingenuity and apply it to situations that in the past have lacked specificity, and, on the other, the (inappropriate) effort to make everything quantifiable, resulting in an overreliance on that which can be measured — at the expense of everything else.

Notably, the push for soccer data seems far more measured than what I see in education. According to Mark Brunkhart, the president of a company that provides soccer data for a fee to clubs and news organizations, he and his staff do not blindly evangelize statistics. Every month or two, he says, he gets a call from a professor or graduate student who is a rabid soccer fan and just finished Moneyball, the book that brought sabermetrics into the mainstream in 2003. (I wrote about Moneyball and its potentially positive implications for school reform in a 2009 column titled “What Would Theo Do?”)

“Every single one comes with the idea that they’re going to solve soccer with the ‘Moneyball’ approach,” Brunkhart said, “and I try to talk them all down.” Similarly, the president of the Society for American Baseball Research pointed to Miroslav Klose’s second goal in Germany’s 4-0 victory against Argentina in the World Cup quarterfinals as an example of how statistics seem to overlook the nuance and elegance of soccer. “A series of three or four absolutely beautiful passes — how do you capture that?” he said. “It’s just the nature of the game.”

Would that I were seeing similar restraint among our education leaders. As longtime educator Ted Sizer once said, “Inspiration, hunger: these are the qualities that drive good schools. The best we educational planners can do is to create the most likely conditions for them to flourish, and then get out of their way.”

More Tests on the Way in DC?

Posted on | by Sam | 1 Comment on More Tests on the Way in DC?

In yesterday’s Washington Post, reporter Bill Turque wrote that Michelle Rhee is seeking an outside contractor to help dramatically expand DCPS’ use of standardized tests, so that every grade from K through 12 will have some form of assessment to measure student progress and teacher effectiveness.

Is this what happens when we pray too long at the altar of “data-driven decision making?”

Don’t get me wrong: I’m all for what that concept actually means — using information to guide all decisions about how to help children learn more effectively — but the faulty logic here is that adding more standardized tests at the end of every school year will achieve that worthy goal. Wouldn’t it be better to start exploring how to strategically bundle other existing measures that tell us a lot about a school’s overall health (such as attendance, graduation rates, faculty absenteeism, and, yes, attitudinal surveys of the students themselves)? Wouldn’t it be better to start experimenting with ways to have other schools in the District implement student portfolio assessments as effectively as the good people at Thurgood Marshall Academy — recently profiled on CBS News — have done?And wouldn’t it be better to stop pretending that systemic reform, and the impact those changes will have on individual students, can be as easily monitored and measured as these tests suggest?

Bring on the information revolution, I say — and this ain’t it.

The Big Picture on School Performance

Posted on | by Sam

On Feb. 1, President Obama vowed to toss out the nation’s current school accountability system and replace it with a more balanced scorecard of school performance that looks at student growth and school progress.

I love the idea. Mr. Obama and education secretary Arne Duncan have repeatedly criticized the No Child Left Behind Act for keeping the “goals loose but the steps tight.” On their watch, both men aspire to introduce a new law that keeps the “goals tight but the steps loose.”

With that more flexible standard in mind, I have a scorecard to propose: the ABC’s of School Success.

Why Send My Son to Public School?

Posted on | by Sam

Earlier this week, Schools Chancellor Michelle Rhee announced the latest hopeful sign for D.C.’s public schools – a spike in citywide student reading and math scores. “We’re thrilled at the progress we’ve made this year,” said Rhee. “We still have an incredibly long way to go.”

I’m grateful for the early improvements in the D.C. schools – and I share Chancellor Rhee’s caution. We all know standardized test scores offer just one window into the health of a school system. Any business school student also knows it’s foolish to judge an organization’s overall health based on a single measure of success. And yet the United States is the only nation with an accountability system based solely on standardized test scores.

We can do better. That’s why local leaders like Michelle Rhee, and national leaders like Arne Duncan, should lead the charge in demanding a better accountability system for our schools.

Here are four things we could do that would make a difference: