I recently took a few days off between jobs, and I thought, “Hey, it would be fun to run a marathon while I have some time on my hands, just to see if I can!” I haven’t been training for a marathon, but I have been running, and I’ve run long distances in the past.
On the first day of vacation, I jogged from my house to a nearby lake that is about five miles around, figuring I’d do laps until I got to my 26.2 miles. But just after I passed 13 miles, I was out of energy and walked home with only a half-marathon to show for it.
On the last day of vacation, I decided to give it one more try. Now, one of the principles that I’ve learned from behavioral science is the value of commitment mechanisms, whether it’s a savings account that restricts access once you make a deposit (which increased in savings in the Philippines), committing in advance to a financial loss if you don’t quit smoking (which decreased smoking in the Philippines), or letting farmers pre-commit to purchasing fertilizer (which boosted fertilizer use in Kenya).
So I found an 18-mile trail near my house, parked my car at one end, and ran 13.5 miles in one direction. At that point, I had few alternatives to running the 13.5 miles back to my car. It’s true, I could have run the last 5.5 miles to the other end of the trail, but it would have been a pain to get back to my car. I also could have walked, but that just would have meant hours of walking in the cold with a dying phone and few supplies. (This is what Bryan, Karlan, and Nelson call a “soft commitment,” where the consequences are principally psychological rather than economic.) So I jogged back. Slowly, but jogging all the way. I made it back to my car just as my phone told me I’d clocked 27 miles.
I have a new paper with Fei Yuan on how to communicate learning results more accessibly than in standard deviations. Here’s the paper. Here’s a summary blog post.
Here are the abstract and title of the paper:
Equivalent Years of Schooling: A Metric to Communicate Learning Gains in Concrete Terms
Abstract: In the past decade, hundreds of impact evaluation studies have measured the learning outcomes of education interventions in developing countries. The impact magnitudes are often reported in terms of “standard deviations,” making them difficult to communicate to policy makers beyond education specialists. This paper proposes two approaches to demonstrate the effectiveness of learning interventions, one in “equivalent years of schooling” and another in the net present value of potential increased lifetime earnings. The results show that in a sample of low- and middle-income countries, one standard deviation gain in literacy skill is associated with between 4.7 and 6.8 additional years of schooling, depending on the estimation method. In other words, over the course of a business-as-usual school year, students learn between 0.15 and 0.21 standard deviation of literacy ability. Using that metric to translate the impact of interventions, a median structured pedagogy intervention increases learning by the equivalent of between 0.6 and 0.9 year of business-as-usual schooling. The results further show that even modest gains in standard deviations of learning — if sustained over time — may have sizeable impacts on individual earnings and poverty reduction, and that conversion into a non-education metric should help policy makers and non-specialists better understand the potential benefits of increased learning.
Over at the Development Impact blog, I provide microsummaries of each research paper in a recent edited volume, Towards Gender Equity in Development.
Precision of language is a virtue so lauded as to seldom be questioned. And yet, a new article by Peter McMahan and James Evans in the American Journal of Sociology — “Ambiguity and Engagement” — shows the potential upside of ambiguous language. This, from Evans’s Facebook post:
Everyone from scientists writing a research paper to criminals under interrogation use ambiguity to widen their appeal or claim more or less than they know. In “Ambiguity and Engagement”, we measure ambiguity in language and explore its consequence for social life. We build a measure of ambiguity in language and demonstrate that when calculated on New York Times articles captures most of the ambiguity perceived by surveyed readers. Next, we assessed ambiguity across millions of article abstracts from science and scholarship, revealing that the humanities and social sciences use language most ambiguously, while chemistry, biology and biomedicine use it most precisely. Finally, we show that more ambiguity systematically—in all time periods and subject areas—is associated with greater association and engagement, as readers reference one another in prolonged conversations. While ambiguous language could lead to fragmentation and disconnection, as audiences understand it in conflicting ways, these findings demonstrate that instead it draws competing interpretations together into conversation with one another as they build on it. [emphasis added]
Association and engagement seem to be measured through fragmentation of citations: Greater fragmentation means that articles are cited by other articles in sub-literatures that don’t cite each other: the academic citation version of cliques.
Here’s how different disciplines line up on ambiguity:
Here’s a word from the article’s discussion:
Articles that use more ambiguous language tend to result in more integrated streams of citations tracing intellectual engagement. This pattern underscores the interpretation of ambiguity not only as a limitation but also as a potentially fruitful characteristic of language. Ambiguity leads to individual and collective uncertainty about communicated meanings in academic discourse. Uncertainty drives social interaction and friction, which yields coordination.
Disclosure: James Evans is my brother.
Here are a couple of recent posts I’ve written on other blogs:
Check them out!
Over the course of last year, I worked closely with counterparts in the Government of Rwanda to map what human capital investments would be most likely to lead to high economic growth in the coming decades. It was a satisfying, collaborative process, and it felt like our findings on the quality of education reached high levels of government decisionmaking.
That work is now included in a volume — Future Drivers of Growth in Rwanda: Innovation, Integration, Agglomeration, and Competition. Our chapter — written by Francois Ngoboka, Ignace Gatare, Rose Baguma, Jee-Peng Tan, Deepika Ramachandran, Fei Yuan, and me — begins on page 51.
Rwanda will not achieve upper-middle income status without a dramatic increase in school completion. Even the bottom 25th percentile of upper-middle-income countries have primary completion rates of 94 percent, about 50 percent higher than Rwanda’s current rate. The median primary completion rate in upper-middle-income countries is nearly 100 percent. Likewise, the median lower-secondary completion rate for upper-middle-income countries is 87 percent, more than 2.5 times Rwanda’s current rate. The disparity is even greater for upper-secondary completion. Expanding basic education, together with ensuring quality, is essential for Rwanda’s sustained growth.
Much more on the quality of education, stunting, fertility, training, and more, in the report.
This was a case where we did randomization without a baseline. I highly recommend this when you’re working with a government because the biggest risk is implementation failure. You’ll spend a lot of time doing the baseline – spend time, spend money – and have the intervention not be implemented. So when you’re working with the government, it’s better to get power by doubling the sample of your endline and just randomized with administrative data so you’ll get the same amount of power but you reduce risk up front.
That is Karthik Muralitharan speaking at the RISE conference today. Of course, he didn’t have to say that doubling your sample also increases your likelihood of randomization resulting in balanced intervention and comparison groups, thus making a baseline less necessary.
Update: This prompted a very active discussion on Twitter, which you can read in full here. Below are a few points.
Ultimately, there are a number of factors to consider — the potential sample size, the probability of implementation failure, the importance of baseline covariates for your analysis. But still, where there is serious concern that the program may not be implemented as expected — and especially if there are decent administrative data — it’s worth consideration. I’ll give the penultimate words to Karthik’s co-author, Abhijeet Singh.
First, responding to Pauline and Andrew’s point.
Second, to Cyrus and Seema’s points.
And the last word to Karthik himself.
There are many more comments, but I won’t embed them all here. You can read the full conversation here on Twitter.