0 A Little Information Could Go A Long Way

THIS GUEST POST COMES FROM ROBERT KELCHEN, DOCTORAL CANDIDATE IN EDUCATIONAL POLICY STUDIES AT UW-MADISON.

In a new report, Filling in the Blanks: How Information Can Affect Choice in Higher Education, Andrew Kelly and Mark Schneider of the American Enterprise Institute examine the role that information can play in the college choice process. One thousand parents in five states were asked which of two similar colleges they would recommend to their high school-age child. Half of the parents were given information about the colleges’ six-year graduation rates, while half were not. The researchers found that parents who were provided information about graduation rates were fifteen percentage points more likely to recommend the college with the higher graduation rate to their child, with larger differentials for parents who reported having less information about colleges and who had lower levels of education.

The intervention shows the importance of providing salient information to the parents of high school students. However, because parents in the study were making a theoretical decision instead of an actual decision that would affect their child, they had less of an incentive to think as carefully about their choice. This might result in effects that are larger than in real life, especially where parents have evenmore information about the two colleges being compared. A logical next step would be to repeat this experiment with high school students to see if the results significantly differ. Encouraging or requiring colleges to publicize their graduation rates may lead parents and students to choose colleges at which the student is more likely to graduate, as they take this information into account. In any case, even a small effect of additional information can make this low-cost intervention sound public policy.

0 Spin Cycle

Education Next apparently has provided a platform for school choice advocate George Mitchell to shill for voucher schools outside of the state of Wisconsin. Here is his latest spin on a study that shows the high school graduation rate to be 12 points higher in seven Milwaukee voucher schools compared with 23 Milwaukee public high schools.

The Milwaukee Journal Sentinel story by Erin Richards provides the crucial quote regarding causation from the study's author, John Robert Warren, a sociology professor at the University of Minnesota:

"We still don't know whether it's going to the voucher school that causes you to be more likely to graduate, or if it's something about the kinds of families that send their kids to voucher schools would make them more likely to graduate," he said.

Then there's the whole question of which and how the voucher and public high schools were chosen for purposes of comparison. More questions than answers. Unlike Mitchell, I neither see this report as providing "another piece of evidence suggesting that urban students benefit when afforded more educational options," nor "new data" to encourage President Obama and Education Secretary Duncan to take "a second look at the power of parent choice."

The study was funded by the voucher-advocacy group School Choice Wisconsin, run by Mitchell's wife, Susan. The Mitchells have split from national school choice leader Howard Fuller who is devoting his current efforts to furthering accountability and quality in the Milwaukee Parental Choice Program.

After the spin cycle, be sure to rinse.

For past perspective on Voucher Inc, please visit here.

0 Premature Conclusions: More Money, No More Grads?

Some members of the media are covering the release of a new Canadian study, associated with the Educational Policy Institute, that examines the effects of a financial aid program on college-going and completion among low-income students. Researchers at the Measuring the Effectiveness of Study Aid Project tried to isolate those effects by examining what happened following a change in student aid policy in Quebec that increase aid eligibility and decreased reliance on loans. By comparing student outcomes both before and after the policy change, and comparing the outcomes of similar student in Quebec to those in other provinces (where such reforms did not occur), analysts attempted to establish a causal effect of aid.

They conclude that the policy affected access (increasing overall enrollment among students from families making less than $20K per year by 4-6 percentage points), and persistence (increasing retention rates by 6 percentage points) but did not affect graduation rates--at least within the 4-year window of time during which graduation was measured.

While noting that the null findings may stem from that short period of observation, the researcher still goes on record with this conclusion: "These results therefore cast doubt on the efficacy of this reform in particular, and of needs-based grants in general, to improve graduation rates."The headline over at Inside Higher Ed reads "More Money Doesn't Equal More Graduates."

This is a distinctly premature and irresponsible conclusion. First, as one of my graduate assistants James Benson pointed out, "if the percentage of college-eligible students that enrolled in college increased by 5 percent, and the persistence and graduation rates remained entirely static, then the program produced a net gain in the proportion of young adults completing semesters and degrees."

Furthermore, there are many reasons why an effect might not be estimated properly in this study. As my colleagues Doug Harris, Phil Trostel, and I explained in a recent paper, a simple correlation between aid receipt and college success is likely to be negative because students from low-income families, in the absence of aid, are for a variety of reasons less likely to succeed. Unless researchers can convincingly account for all of those reasons – and we argue that very few do – the estimated effects of aid are likely to look smaller than they really are. This study is not very convincing and really doesn't move far beyond a correlation, for many reasons. For example, as another graduate assistant, Robert Kelchen, indicates:

1. The comparison groups (Quebec vs. other provinces) have very different rates of financial aid take-up prior to the reform. This calls the validity of the comparison into question. It's also too bad the researcher didn't see fit to post his tables on the website, since we cannot see whether the differences post-treatment are significant.

2. Quebec saw increases in the enrollment rates of high-income students following the reform, in addition to increases in the enrollment rates of low-income students. If financial aid was the real driver, it shouldn't have affected the (ineligible) high-income students.

These are but a few examples-- if a full research paper (such as would be submitted for academic review) was available, I bet we'd have more concerns.

This is a case of the press jumping the gun and running with a story, and a headline, not supported by the empirical work done by the researchers. We're in a recession, and aid programs cost a lot of money. We do need to know if they work, and in particular if they are cost-effective. But the estimation of impacts should be done more carefully, and results discussed in a much more responsible manner. Sexy but un-informed headlines will do little good-- perhaps even casting a shadow on an effective program, reducing its ability to maintain funding. All of us studying financial aid have an obligation to do much, much better.