How Should We Evaluate Our Preschools?

Imagine, for a second, that you are in charge of more than $600 million in taxpayer money. You live in a city that has made deep investments in early education, and that aspires to provide universal preschool by 2014. You have a thriving network of public charter schools, and you want to help parents make more informed choices about where they send their children.

What would you do?

Continue reading . . .

The PCSB Responds

As the deadline for public comment on the PCSB’s proposed accountability framework for early childhood programs nears (August 28) — and as public reactions to the proposal intensify — PCSB’s executive director, Scott Pearson, published a formal reply to the change.org petition that is asking for greater balance in how schools are evaluated. Here’s Scott’s response:

I appreciate your sharing your concern about assessments in early childhood testing.

It’s important to note that the proposed early childhood PMF was developed in close collaboration with charter school leaders who broadly support this proposal.

Among most important features of the PMF was to incorporate the very assessments that these schools are already giving to students, and that have been used for years for school accountability purposes. Schools may choose from a menu of over two dozen assessments – a menu that includes virtually all of the assessments already used by our schools. These bear little resemblance to “fill in the bubble” tests. They are administered one-on-one, by teachers. They are authentic and designed to be developmentally appropriate and to feel low-stakes for the child.

Let me repeat. All of our schools already use these assessments for accountability purposes. The Early Childhood PMF does not impose new tests. It creates a framework for comparing the results at one school using one test with the results at another school using a different test.

Indeed, far from imposing new assessments, the Early Childhood PMF restricts the use of these assessments. For example many schools currently evaluate their youngest students for literacy and numeracy proficiency. The Early Childhood PMF now looks only at whether these students abilities have improved, not at the absolute proficiency levels of students.

PCSB has a vital role to ensure that charter schools, which are public schools paid for with taxpayer dollars, are high quality. We seek to play this role in a way that preserves the maximum autonomy and flexibility for school leaders to run their schools as they see fit. While every school is different, virtually all agree that the vast number of students who enter the third grade unable to read or perform basic math is a crisis in our city and that a basic function of a school is to prepare students to be literate and numerate.

The core bargain of charter schools is greater autonomy in exchange for greater accountability. This means there will inevitably be high stakes reviews of charter schools. Almost all charter schools prefer to be left alone to develop their own educational programs. High stakes reviews focusing on *how* they run their schools run counter to the philosophy of most of our school leaders. They want the freedom to decide the “how” and to be judged on the outcomes.

While it would be hard to find an educator who did not value the social and emotional development of children, most would also agree that valid assessments of social and emotional progress are not yet well-established. For this reason many school leaders are reluctant to have significant portions of an evaluation of their school be based on an assessment of their students’ social and emotional development. This concern influenced the work the task force developing the Early Childhood PMF.

We take the public comment phase of the process very seriously. The views of those who signed this petition, along with those who submitted comments directly to me and to PCSB, will be taken into account. Beyond this, we recognize that no system of accountability is perfect. For this reason, the committee of schools who designed the Early Childhood PMF will continue to meet to consider ongoing improvements.

We continue to discuss this issue on our blog, which can be found at www.dcpcsb.org.

Scott Pearson
Executive Director
DC Public Charter School Board

I’m grateful for Scott’s response, though I don’t feel it addresses the central complaint. As I wrote yesterday, “the problem with the PCSB’s proposal has less to do with requiring schools to choose from a common pool of math and reading assessments and more to do with attaching a disproportionate weight (60-80% of the total) to reading and math. This immediately transforms those assessments from diagnostic to accountability tools. It is guaranteed to modify the behavior and strategic planning of the schools. And it perpetuates the shell game of American public education, in which we use partial information to pronounce complete judgment on whether a given reform effort is working or not.”

I don’t see how Scott’s response addresses that central critique. Am I missing something? What do you think needs to be done?

 

Is It The Tests or the Stakes?

As DC grapples with whether or not to adopt an accountability framework that would assign between 60-80% of a charter preschool’s overall rank to its students’ reading and math scores, it’s worth asking: What’s at the root of the problem here — the tests, or the stakes attached to those tests?

When it comes to the debate over the Early Childhood Framework (which you can access here — just click “Public Hearing” in the left column and then click “Early Childhood PMF”), I would suggest it is overwhelmingly the latter.

Here’s the thing: just about every early childhood program worth its salt is already assessing children in a variety of ways, including, and not limited to, reading and math. Usually, these assessments are administered in a 1-on-1 format, and if a teacher does his or her job skillfully, the child won’t even register that it’s a “test.” So the Public Charter School Board is correct when it says that testing children in these subjects is not new.

What their explanation glosses over is the impact children, educators and parents can expect once these assessments have high-stakes attached to them. Rankings matter to all schools, but they are particularly important to charter schools, which are always in the midst of raising money, searching for a permanent facility, and recruiting families. Here in DC, “Tier 1” status is a gateway to better fundraising, preferential treatment when it comes to facilities, and longer waiting lists.

The problem with the PCSB’s proposal, therefore, has less to do with requiring schools to choose from a common pool of math and reading assessments (indeed, the list boasts more than thirty options to choose from, many of which were submitted by the charters themselves), and more to do with attaching a disproportionate weight to reading and math. This immediately transforms those assessments from diagnostic to accountability tools. It is guaranteed to modify the behavior and strategic planning of the schools. And it perpetuates the shell game of American public education, in which we use partial information to pronounce complete judgment on whether a given reform effort is working or not. (And, more importantly, in which we allow ourselves to assign credit or blame in ways that correspond to a political timetable.)

In reality, as both common sense and research would tell you, it’s not that simple. “True standards of intellect do not lend themselves wholly to quickly collected, precise, standard measurement,” wrote Ted Sizer in his classic book, Horace’s Compromise. “We need to devise clusters of instruments to probe our students’ ability to think resourcefully about important things. Indeed, we need time to reflect deeply on what we mean by ‘think resourcefully’ and what we feel are the most ‘important things.’”

In his 2009 book Measuring Up: What Educational Testing Really Tells Us, Harvard professor Daniel Koretz makes a similar point. “Careful testing can give us tremendously valuable information about student achievement that we would otherwise lack,” he says. The question is how well we understand what standardized tests can, and cannot, tell us about American schools. “There is no optimal design,” he asserts. “Rather, designing a testing program is an exercise in trade-offs and compromise – and a judgment about which compromise is best.

Systems that overvalue one set of test scores fail to heed this advice, but they also fail to highlight a sufficiently clear picture of what is actually happening in a school. “Certainly, low scores are a sign that something is amiss,” Koretz says. “But the low scores themselves don’t tell why achievement is low and are usually insufficient to tell us where instruction is good or bad, just as a fever by itself is insufficient to reveal what illness a child has. Disappointing scores can mask good instruction, and high scores can hide problems that need to be addressed.”

So let’s vigorously debate ideas like the PCSB framework, and when we do so, let’s follow the lead of experts like Lydia Carlis, the chief of Research and Innovation for the AppleTree network of preschools. “The ongoing conversation will be so useful to all children if we focus on both/and rather than either/or when discussing teaching and (developmentally appropriate) assessments of academics and SEL in early childhood, ” she says. “The early childhood interventions that have been evaluated longitudinally and shown effective demonstrate that low-income children at risk for school failure need early access to high quality, evidence based academic and social supports. Social emotional skills are both necessary and insufficient for closing the academic achievement gap. Children who begin demonstrating cognitive gaps as early as nine months need skill and concept development on academic content and strategic support for social emotional learning. And clearly, all children will benefit from a high quality, balanced instructional program.

Works for me. PCSB, are you listening?

 

High Stakes Tests For 3-Year-Olds?

If you’re a parent of a young charter school student in DC – or just someone who cares about early education – you need to know what’s happening here in the nation’s capital, and fast.

In less than a week, all charter schools that serve young children will start being held accountable to their students’ test scores on reading and math.

Just to clarify: we’re talking about three-, four-, and five-year-olds. Being Tested. In Reading and Math. With High Stakes attached for the schools that care for them.

First, some context: Like many other cities, DC is a place where daycare waiting lists can last for years, and where the costs of childcare can amount to a second mortgage. Unlike other cities, however, Washington ranks first in the nation for its percentage of 3- and 4-year-old children enrolled in preschool programs – 88% in all, and at an expense of nearly $15,000 per child. That’s a huge advantage for DC’s families, and a huge influence on the overall development and growth of the city’s youngest residents.

As DC inches closer to its goal of providing universal preschool by 2014, our civic leaders are rightfully asking themselves what else they should do to ensure that our deep investments in early childhood reap deep civic returns. And in their effort to provide an answer, DC’s sole authorizing and oversight body for charter schools – the Public Charter School Board – has proposed an accountability plan for the youngest children that would mimic the format that’s already in place for the oldest.

If the plan is approved – and it will be, barring significant community objections – all of the city’s Pre-K and lower elementary charter school programs will forthwith be ranked according to a weighted formula that assigns between 60 and 80% of a school’s overall performance to student reading and math scores. And although the proposal includes the possibility for schools to “opt-in” to adding an assessment that measures the social and emotional (SEL) growth of children, it would count for just 15% of the total for Preschool and PreK, and 10% for Kindergarten.

This sort of weighted formula squares neatly with the latest trends in education policy. It does not, however, align with the latest research on the brain.

“Everything that happens to us affects the way the brain develops,” says Dan Siegel, a clinical professor of psychiatry at UCLA and the author of The Whole Brain Child. “The brain is a social organ, made to be in relationship. What happens between brains has a great deal to do with what happens within each individual brain . . . [And] the physical architecture of the brain changes according to where we direct our attention and what we practice doing.”

Where we direct our attention, then, matters greatly when it comes to determining what our children will practice doing, and how their brains will develop.  And what scholars like Siegel are saying is that the worst thing we can do is disproportionately weight one piece of the developmental puzzle. “We want to help our children become better integrated so they can use their whole brain in a coordinated way,” he explains. “We want them to be horizontally integrated, so that their left-brain logic can work well with their right-brain emotion. We also want them to be vertically integrated, so that the physically higher parts of the brain, which let them thoughtfully consider their actions, work well with the lower parts, which are more concerned with instinct, gut reactions, and survival.”

Siegel’s suggestions align with the recommendations of other leading researchers, all of who confirm that the foundation of learning is social, not academic. In fact, according to the Collaborative for Academic, Social & Emotional Learning (CASEL), an organization that works to advance the science and evidence-based practice of social and emotional learning, the best way for schools to provide the optimal foundation for learning is by helping students develop five core competencies: self-awareness, or the ability to accurately recognize one’s emotions and thoughts and their influence on behavior; self-management, or the ability to regulate one’s emotions, thoughts, and behaviors effectively in different situations; social awareness, or the ability to take the perspective of and empathize with others from diverse backgrounds and cultures; relationship skills, or the ability to establish and maintain healthy and rewarding relationships with diverse individuals and groups; and responsible decision-making, or the ability to make constructive and respectful choices about personal behavior and social interactions. CASEL has even published a compendium of the available assessment measures when it comes to measuring these sorts of skills in children.

In other words, the research is clear, the tools are out there, and the common sense is self-evident to anyone who is a parent to young children. So I ask you: will an accountability framework that places as much as 80% of its weight behind reading and math scores engender a generation of children with the skills CASEL identifies as the foundation of all learning, or lead to the sort of neurobiological integration scholars like Siegel are calling for?

If you think the answer is yes, sit tight. But if you think the answer is no, I urge you to call or email the PCSB’s executive director, Scott Pearson (spearson@dcpubliccharter.com, 202.328.2660) and insist that any accountability system assign equal weight to the different components of a healthy, high-functioning learning environment – including, and not limited to, social and emotional growth. (You can also sign this petition.)

The past twelve years of federal policy have taught us that when it comes to assessing the upper grades, reading and math are valuable – and overvalued. Let’s not make the same mistake twice.