As readers of this bog know we believe Michigan needs to move away from standardized testing. Anytime we make that case, the first question we get is “what is the alternative?” In a column for Bloomberg entitled Testing Craze Is Fading in U.S. Schools. Good. Here’s What’s Next. Andrea Gabor tackles just that question.

Before we delve into the alternatives to standardized testing, let’s review why Michigan Future has turned away from supporting a test-based assessment and accountability system:

• The book Crossing the Finish Line details findings that a student’s high school GPA – made up of grades given by individual teachers across four years of high school – was far more predictive of eventual college graduation than her SAT/ACT score. Why? Because while test scores measure a student’s ability on a narrow band of math and reading skills, GPA measures a diverse set of capacities, encompassing academic habits, content knowledge, and non-cognitive skills, exhibited day after day across four-years of high school.
• Research by Northwestern economist C. Kirabo Jackson found that a non-cognitive index of grades, attendance, and disciplinary records was more predictive of long-term success than test scores. He also found that the set of teachers that were able to improve this index was an entirely different set of teachers than those that were adept at raising test scores. The message from both Jackson and Crossing the Finish Line: when we focus only on test scores, we miss the really important stuff.
• Our learnings about the skills that are rewarded in today’s labor market. Far broader, and in many ways more rigorous, than the skills that are measured by standardized tests––particularly single right answer multiple choice tests. As you know we would define career- ready skills as the 6Cs which includes content (what is on the test) but much more.
• The unintended consequence of test scores being the only measure that schools/educator are held accountable for has led to non-affluent kids (in cities, suburbs and rural areas) attending schools with curriculum and pedagogy that is both non engaging and way too narrow. Focusing almost exclusively on what is on the test at the expense of all the other Cs as well as extracurriculars, electives, the arts, even writing that are so important to a quality education that we all want, if not demand, for our kids. To us this is a key equity challenge of our times: non-affluent kids in schools that are designed to build too narrow skills.

You can delve into why we have turned against standardized testing in our previous blogs which can be found here, here, and here.

For those interested in learning more about the damage to both curriculum and pedagogy caused by test-based assessment and accountability systems, the book In Search of Deeper Learning is highly recommended.

To be clear our reason for wanting to de-emphasize standardized testing is not to lower standards. In fact it is the opposite. We strongly believe the bar for student achievement in Michigan is too low. We need to hold school management accountable for more rigorous and broader student outcomes. To do that requires better assessments. The reality is that the non-content specific skills that make up the 6Cs are higher-order skills than what is measured by today’s standardized tests.

If we are serious about college and 40 year career success (not a first job) we had better get what we are holding schools accountable for right. What we need to come up with is an assessment system that actually predicts college and career success. If we don’t we are harming, most importantly, our kids as well as the Michigan economy.

Which brings us back to the question “What is the alternative to standardized testing?” Gabor writes:

Now states from Arizona to Wyoming are retreating from high-stakes testing. The announcement last month that New York’s education commissioner, a testing proponent, will resign in August, signals another reversal.

It might be easy to say good riddance, but schools still need ways to measure student progress. The accountability movement that pushed testing was a response to a genuine need to improve K-12 education. Since the 1983 publication of “A Nation at Risk,” a bipartisan report by a commission appointed by President Ronald Reagan, business leaders have warned that schools weren’t developing the knowledge workers modern industry needs, and progressive educators have criticized traditional factory-style schools for not fostering an engaged and informed citizenry.

So schools need to find new ways to show accountability advocates that test retrenchment won’t weaken standards, and this presents an opportunity to develop more robust assessments and better education.

The country’s best under-the-radar experiments are a useful guide. Chief among these is the New York State Performance Standards Consortium, a decades-old effort led by progressive educators and involving 38 high schools, which won exemptions from all standardized tests except English. Instead, students complete ambitious projects known as performance-based assessments — think mini theses with lots of research, writing and real-world projects in everything from social studies to physics, which students present to expert panels, including teachers (often from different schools) and community members.

Since launching in the 1990s, the consortium has racked up far higher graduation rates and college matriculation rates for its schools than New York’s traditional public schools.

… In 2015, New Hampshire won a waiver under a federal pilot program that opened the door to alternative assessment programs, and is introducing performance-based projects like New York’s that are designed almost entirely by teachers

Another answer to the what is the alternative question is the MIT Playful Journey Lab in collaboration with Albemarle County School District, Portola Valley School District, and San Mateo County Office of Education, Beyond Rubrics Toolkit. Really worth checking out.

The skills the Toolkit is designed to assess:

• Agency: The capacity to make intentional choices and to understand that you have such a capacity. With agency, you see yourself as a contributor and an agent of change in the world surrounding you.
• Design Process: A way to approach challenges by brainstorming, prototyping, testing, and iterating. Designers are aware of the many steps to reach a solution and deliberately work on each step to improve a design.
• Social Scaffolding: An active participant in a community that supports everyone’s learning.
• Productive Risk-taking: To try an idea or a solution beyond your comfort zone. Even when an action ends in an unexpected way, you can identify lessons learned and connect it to the next iteration or future projects.
• Troubleshooting: A capacity to persist and to find solutions. If a project is not progressing as expected, you can use different strategies to diagnose and fix the problem. Not giving up requires patience, resilience, and resourcefulness.
• Bridging Knowledge: Using knowledge from your lived experiences at home, community, and culture as well as from OST experiences and other subject areas to benefit the project you are working on.
• Content Knowledge: You may develop stronger conceptual understanding, be able to accurately understand why this do or do not work, or be able to use materials in safe and effective ways.

Does anyone really believe these skills are not a better predictor of life outcomes than the skills assessed on the single right answer multiple choice tests that we use today?

The post Moving away from standardized testing appeared first on Michigan Future Inc..

But it turns out that there’s one, very uncomplicated measure of college readiness that repeatedly trumps everything else in the research: a student’s high school grades.

In nearly all the research that’s been done on student success in college, the most predictive indicator, time and again, is a student’s high school GPA. While for many this is now common knowledge, no matter how many times we share this finding it continues to shock people, and some simply don’t believe it. Grades are too subjective, they’ll say. And some high schools are far easier than others. How can a measure composed of inconsistent inputs (classroom grades) provided by a range of people with inconsistent training (teachers) produce a measure so predictive of future success?

The answer is that a student’s GPA is likely so predictive precisely because of the many varied inputs and the lack of standardization that go into the creation of that GPA. It captures a student’s mastery of academic content to be sure, but also ends up capturing their ability to pay attention in class, take notes, participate, complete assignments on time, seek feedback, seek help, advocate for themselves, manage their time, and create study systems. In other words, it provides a composite measure of many of the skills required for college success.

The research

The most convincing research on GPA comes from the book Crossing the Finish Line, the authoritative study on college completion in America. The book utilized a massive national dataset that enabled the authors to follow students from high school through to college graduation, and run a bunch of tests to figure out what high school characteristics predicted college success.

And they found that a student’s high school GPA was one of the strongest predictors of their eventual college graduation – far more predictive than a student’s SAT/ACT score. In the words of the authors:

High school grades are a far better predictor of both four-year and six-year graduation rates than are SAT/ACT test scores…(our) analysis reinforces the point that high school grades measure a student’s ability to ‘get it done’ in a more powerful way than do SAT scores – a conclusion that holds, regardless of the high school attended.

This last point bears repeating. The predictive validity of a student’s GPA held regardless of the high school that the student attended. So whether a student attends a “good” high school or a “bad” high school, the GPA they graduate with says an awful lot about whether or not they will end up with a college degree.

Research from the Consortium on Chicago Schools Research (CCSR), which regularly studies the educational outcomes of students in Chicago Public Schools, backs up the findings from Crossing the Finish Line. Their research found that a student who got an 18 on their ACT – below the ‘college ready’ benchmark of 21 – but graduated from high school with a 3.5 GPA, was nearly 20 percentage points more likely to graduate from college than a student who scored a 24 on the ACT, but graduated with a GPA below 2.5.

Echoing the findings from Crossing the Finish Line, the CCSR authors write:

Grades are so important because they capture many of the noncognitive aspects of students’ work habits that test scores miss, such as executive functioning, academic perseverance, and growth mindset.

Finally, in Paul Tough’s latest book, Helping Children Succeed, he describes a study by Northwestern economist C. Kirabo Jackson, in which Jackson, using data from all high school students in North Carolina, tried to figure out what high school characteristics predict college and life success. Jackson created a “noncognitive index” using student grades, attendance, and discipline records, and pitted that measure against test scores. As you might have guessed, Jackson found that the noncognitive index was a better predictor of college attendance and adult wages than was test scores.

All of this research is important because it suggests that with our obsessive focus on test scores, we’re taking our eye off the ball. What we want from our k-12 education system are students who’ve developed a whole range of skills that will help them navigate postsecondary education, and the rest of their adult lives. Test scores are an insufficient measure of these capacities, yet it’s the only measure of quality that gets mentioned in public discussions on education.

Focusing on the student habits, mindsets, and skills needed to attain a good GPA – and be successful in college – would lead to much richer discussion of what we mean by a good education.

The post What actually predicts college success? appeared first on Michigan Future Inc..

Our current school accountability system ranks schools almost exclusively on standardized test scores. But standardized tests capture only a small piece of what it means to be college-ready.

A couple pieces of evidence. The book Crossing the Finish Line used a large, nationally representative set of student data to analyze what truly predicts success in college. And what they found was that a student’s high school GPA – made up of grades given by individual teachers across four years of high school – was far more predictive of eventual college graduation than her SAT/ACT score. And the student’s GPA was predictive regardless of high school attended; whether the student went to a “good” high school or a “bad” high school, a good GPA predicted college success.

Why? Because while test scores measure a student’s ability on a narrow band of math and reading skills, GPA measures a diverse set of capacities, encompassing academic habits, content knowledge, and non-cognitive skills, exhibited day after day across four-years of high school.

Research from Northwestern economist C. Kirabo Jackson came to a similar conclusion. Using a large set of student data from North Carolina, Jackson found that a “non-cognitive” index of grades, attendance, and disciplinary records was more predictive of long-term success than test scores. And he also found that the set of teachers that were able to improve this index was an entirely different set of teachers than those that were adept at raising test scores.

The message from both cases: when we focus only on test scores, we miss the really important stuff.

The case for focusing on long-term outcomes: data from Metro-Detroit schools

The argument we often hear against using long-term outcomes to measure school success is that there are far too many factors that intervene between high school graduation and college graduation to meaningfully hold high schools accountable for postsecondary results.

Our argument, however, is that there is so much a high school can do – outside of improving test scores – to improve postsecondary outcomes that it’s irresponsible not to include this data on an accountability scorecard.

A look at local data is illustrative. Detroit’s selective high schools, Cass Tech and Renaissance, were ranked in the 21^st and 45^th percentile, respectively, in the 2015-16 school rankings. High-ranking suburban high schools, like Birmingham Groves and Saline High School fell in the 84^th and 95^th percentile. To some extent, this is to be expected: test scores are highly correlated with socioeconomic status, and Cass and Renaissance, despite being “test-in” schools, serve a far higher proportion of economically disadvantaged students than their suburban counterparts.

Yet if we look at the high school class of 2008, postsecondary outcomes for these schools look remarkably similar. 57% of Renaissance graduates and 45% of Cass graduates earned a four-year degree in the 6 years after high school, with another roughly 7% at each school collecting a degree in the following two years. Meanwhile, 57% of Groves graduates and 49% of Saline graduates had earned a four-year degree 6 years after high school, with another 6 to 10% earning a degree in the following two years.

The postsecondary outcome data for these schools was roughly the same, despite the fact that a larger chunk of Cass and Renaissance student will likely face a more difficult path to college graduation.

Yet although postsecondary outcomes for these schools look largely similar, in the picture the public receives on school quality, Groves and Saline are exemplary, while Renaissance and Cass are failing.

What might be going on at Cass and Renaissance that we miss by looking only at test scores? Perhaps they’ve developed a rich college-going culture, a strong college-counseling department, or a broad curriculum that targets the wide-range of skills students need to do well in college.

Regardless, this example demonstrates just how much is missed when we only focus on test scores. Groves and Saline may very-well be exemplary. And Cass and Renaissance may very well have a lot of room for improvement. But we can’t tell any of that based on the picture of school success we’re given through the state accountability system.

Including long-term outcomes in our school evaluation system doesn’t give us all the information we need, but it certainly makes the picture just a bit clearer.

The post What we miss when we focus only on test scores appeared first on Michigan Future Inc..

But I also mentioned that there’s a significant research base demonstrating the connection between increased school funding and better student outcomes. This stands in stark contrast to the dominant narrative that school funding is disconnected from student outcomes, so I wanted to spend some time exploring some new school funding research, and how the “money doesn’t matter” myth got started in the first place.

First, the myth. The basis for the myth is that while education funding in the aggregate has increased substantially since the early 70s, reading scores on the NAEP exam, the nation’s report card, have been stagnant. These type of general observations are made by politicians on both sides of the aisle, in seemingly every state in the country.

However, these observations have three major faults. First, they fail to control for factors that have naturally caused the cost of providing education to rise, like increasing human capital costs as wages for college-educated workers have increased across the economy.

In addition, these observations focus only on test scores as the outcome-of-interest. However, it’s entirely possible that school funding increases have significant positive impacts on life outcomes without impacting test scores. It’s also worth noting that there’s substantial research demonstrating that high standardized test scores may not be as correlated with later life outcomes as we like to think.

Finally, these observations don’t account for how school-funding increases impact different populations of students differently. One would think that a school-funding increase would have a larger impact on poor student in an under-resourced school than a student who’s already receiving significant resources both at school and at home.

Which is why the results from a recent school funding study from C. Kirabo Jackson, Rucker Johnson, and Claudia Persico are so welcome. Their study, which followed a nationally representative set of students born between 1955 and 1985, through to 2011, addressed the above faults, and that school funding does, indeed, matter.

Rather than observing general increases in school funding, the researchers analyzed “shocks” in school funding – large increases that came about either through legislative action or court orders – and analyzed what effect these shocks had against students that didn’t receive the same funding. They also analyzed the effect school funding had on low-income versus wealthy children, and analyzed long-term outcomes, rather than short-term test scores.

What they found was pretty astounding. Based on their data, if a poor child attends a school that receives a 20% increase in school funding, that is maintained throughout a child’s 12 years of public education, she is likely to complete nearly one additional year of education, earn 25% more as an adult, and is 20 percentage-points less likely to be poor as an adult, compared to students who didn’t receive the same level of funding in either duration or intensity.

As educational interventions go, these are huge effects. The authors write that the size of the effects are large enough to eliminate two-thirds of the gap in these measures of life outcomes between students raised in poor and non-poor families. In other words, more school funding now has a major impact on future economic mobility.

Also interesting is that increased funding ended up having no impact on children from non-poor families, perhaps because their schools were already well-resourced, and/or the positive effects of their home lives were more important than any school effects.

These findings are not only statistically robust, but also have the benefit of making intuitive sense. It makes sense that more resources directed to low-income children, who are generally lacking resources both at school and at home, would produce significant positive outcomes.

It also makes sense that more resources may not cause dramatic improvements in test scores, but could greatly improve life outcomes. If the money is spent to reduce class sizes or hire a school social worker or college counselor, the benefit to students may not show up in test scores, but instead in stronger relationships, a better connection to their school, and more ownership over their education.

Either way, this study delivers a welcome counter to the narrative that money in education doesn’t matter. If we look at the outcomes that really matter and to which students the money goes, we find that money not only matters, but can change students’ lives.

The post More on why school funding matters appeared first on Michigan Future Inc..

As I wrote in my last post we need to give schools management in public schools––they already have it in charters––more ability to replace low quality teachers and other building level professionals. But doing that based on how well a teachers’ students perform on standardized tests is not a good way of assessing the quality of a teacher.

Paul Tough in his must read new book Helping Children Succeed and in an Atlantic article entitled How Kids Learn Resilience writes about research on teacher effectiveness by Northwestern University economist C. Kirabo Jackson. Tough writes:

… Jackson began investigating how to measure educators’ effectiveness. In many school systems these days, teachers are assessed based primarily on one data point: the standardized-test scores of their students. Jackson suspected that the true impact teachers had on their students was more complicated than a single test score could reveal.

… Jackson did something new. He created a proxy measure for students’ noncognitive ability, using just four pieces of existing administrative data: attendance, suspensions, on-time grade progression, and overall GPA. Jackson’s new index measured, in a fairly crude way, how engaged students were in school —whether they showed up, whether they misbehaved, and how hard they worked in their classes. Jackson found that this simple noncognitive proxy was, remarkably, a better predictor than students’ test scores of whether the students would go on to attend college, a better predictor of adult wages, and a better predictor of future arrests. (Emphasis added.)

… Jackson’s proxy measure allowed him to do some intriguing analysis of teachers’ effectiveness. … Jackson found that some teachers were reliably able to raise their students’ standardized-test scores year after year. These are the teachers, in every teacher evaluation system in the country, who are the most valued and most rewarded. But he also found that there was another distinct cohort of teachers who were reliably able to raise their students’ performance on his noncognitive measure. If you were assigned to the class of a teacher in this cohort, you were more likely to show up to school, more likely to avoid suspension, more likely to move on to the next grade. And your overall GPA went up—not just your grades in that particular teacher’s class, but your grades in your other classes, too.

Jackson found that these two groups of successful teachers did not necessarily overlap much; in every school, it seemed, there were certain teachers who were especially good at developing cognitive skills in their students and other teachers who excelled at developing noncognitive skills. But the teachers in the second cohort were not being rewarded for their success with their students—indeed, it seemed likely that no one but Jackson even realized that they were successful. And yet those teachers, according to Jackson’s calculations, were doing more to get their students to college and raise their future wages than were the much celebrated teachers who boosted students’ test scores.

Wow! Research that demonstrates that so called non cognitive skills are a better predictor than standardized test scores of attending college, adult wages, and future arrests. And shows that, by and large, the teachers that are most effective in developing these skills in their students are not the teachers that are best at getting higher test scores.

Obviously we want schools to hire and retain more teachers that are good at developing students skills that lead to students “more likely to show up to school, more likely to avoid suspension, more likely to move on to the next grade. And your overall GPA went up—not just your grades in that particular teacher’s class, but your grades in your other classes, too.” And yet we have a teacher evaluation system––because we have wrongly equated student success with a test score––that as Tough notes does not reward teachers who are good at building these skills, “indeed, it seemed likely that no one but Jackson even realized that they were successful.”

Not smart!

The post Standardized tests and teacher evaluation appeared first on Michigan Future Inc..