A few self-links

  1. This morning I posted “What a new preschool study tells us about early child education – and about impact evaluation” over at Development Impact, about an interesting study “Cognitive science in the field: A preschool intervention durably enhances intuitive but not formal mathematics,” which is a randomized controlled trial in Delhi, India.
  2. You can also just watch the researchers explain that paper below.

3. The French version of my post, “A Framework for Taking Evidence from One Location to Another,” based on the work of Mary Ann Bates and Rachel Glennerster, is now available:  Comment déterminer si un projet avec de bons résultats dans un pays fonctionnera ailleurs ?

4. The Portuguese version of my post, “Are good school principals born or can they be made?” based on the work of Roland Fryer and others, is now available: Os bons diretores da escola nascem ou podem ser criados?

Even if technology improves literacy, is it worth the cost?

Ben Piper reports on insightful work that he and co-authors have done comparing various education technology intervention in Kenya in terms of both effectiveness (do they improve reading ability?) and the cost-effectiveness (what’s the cost per reading gain?).

I recommend his full post (or the research paper it’s based on). Here are a couple of highlights:

When compared to traditional literacy programs, the more intensive ICT interventions did not produce large enough gains in learning outcomes to justify the cost. This is not to say that each of the ICT interventions did not produce improvements in students’ reading ability…. [But] the cost-effectiveness of all of these programs might still be significantly lower than a clear investment in high quality literacy programs…. In additional to monetary cost, an opportunity cost existed…. Many of the teachers, tutors, and students lacked exposure to technology and the time and energy spent on learning how to use the technology reduced the amount of time for instructional improvement activities. 

When costs are considered, there are non-ICT interventions that could have larger impacts on learning outcomes with reduced costs; one such option could include assigning the best teachers to the first grade when children are learning how to read, rather than to the end of primary school as many schools do.

Economists will disagree with the standard errors if I understand the specification right: Randomization is at the district level and I don’t believe the authors cluster the standard errors. 

But I don’t think that will change the fundamental message here: Even if there are some gains from education technology, we have to ask when they will be most likely to be worth the cost.

Economic Development in 20 minutes to middle schoolers

Earlier this week I had 20 minutes each to speak to four classes of middle schoolers about my career. I talked about economic development. I used a presentation (available in full here). Given that it was the antepenultimate day of school, the students and teachers appeared very engaged.

  1. I showed the students four families from Gapminder’s Dollar Street initiative — one from India, one from Burundi, one from Ukraine, and one from Colombia, and I had students vote (by raised hands) on which family they expected was poorest and which was wealthiest.

slide 2

2. Then I introduced the four families in turn, and I expressed their monthly income in terms of the price of school lunches at the middle school where I was speaking. (The Ukrainian family’s monthly income was the equivalent of 3,100+ school lunches, more than the most insatiable student should ever consume.)

slide 3

3. Having highlighted the massive gaps in income between families, I invited the students to vote (again, by raised hands) on the number of people in the world currently in extreme poverty. Hint: According to the latest estimates from Cruz et al. (2015), we’re at 700 million.

slide 4

4. Then I showed — in two ways — how much poverty has decreased over time. First I showed the figure below from Our World in Data. I also showed the evolving chart on income and life expectancy from Gapminder. (Technical difficulties precluded showing the actual evolution over time, but at least I could show screenshots of the beginning and the end.)

slide 9

5. I then highlighted geographical concentrations of poverty.

slide 12

6. Then I gave two very simple definitions of economics:

  • Macro: Why are some countries rich and others poor
  • Micro: How can poor families get out of poverty?

7. What does economic growth look like? Here’s some of the variation, where countries on the top left are those that grew the most: low income in 1960, high income in 2014.

slide 13

8. I then invited the students to suggest what makes countries grow. We talked about a few possibilities.

slide 15

9. We then returned to the growth map and differentiated between two high-growth countries: South Korea, which produces goods for trade (I had all the students with Samsung devices raise their hands) versus Equatorial Guinea, which produces a natural resource for trade (oil). We talked about the different implications for inequality.

10. I then talked through the two objectives of the World Bank: to encourage growth and to end extreme poverty. (To be more precise, the “twin goals” are to encourage “shared prosperity” — growth that benefits the bottom 40% of the population — and to end extreme poverty.)

11. Then, since education is an area I work in actively, I highlighted the relationship between learning and economic growth, using data from Hanushek et al. (2008).

slide 18a

12. I then asked how many of the 7th graders could read a sentence: All of them claimed that ability. I then showed data from the Early Grade Reading Barometer on the percentage of 2nd graders in various countries who couldn’t read a single word, which of course predicts future literacy.

13. I talked about what I do specifically, with a few examples (including a few funny stories).

slide 20

14. And finally, I talked them through how I got to my current job and reminded them that it’s not just economists working in international development.

slide 21

Many thanks to all those who gave suggestions. I used several of them and would have enjoyed using others if I’d had more time (either to prepare or with the classes).

The next day, I received a number of thank you notes from students. This one took the cake.

thank you

The impact of student performance labels on later schooling

One key feature of test-based accountability systems in the U.S. is that every student receives not only a test score but also a label based on their performance. Massachusetts, the state that we study, assigns students labels of Failing, Needs Improvement, Proficient, or Advanced by determining cut-points with which it divides the finer-grained test-score distribution into performance regions. … These labels provide no additional information beyond the test scores on which they are based; they are simply coarse summaries of a student’s performance. We focus on responses to labels that have no state-defined consequences for students.

This, from a paper by John Papay, Richard Murnane, and John Willett, in the Journal of Human Resources.

Using a regression-discontinuity design, we find persistent effects of earning a more positive label on the college-going decisions of urban, low-income students.

Here’s an open-access, earlier version of the paper. And here’s the key figure, on the left comparing students with an “advanced” versus a “proficient” label, and on the right, comparing the “proficient” to the “needs improvement” label:

advanced-proficient cutoff

 

How to improve the quality of education in Africa? Start with the evidence from Africa!

A new study was just published in the Review of Educational Research:  Identifying Effective Education Interventions in Sub-Saharan Africa: A Meta-Analysis of Impact Evaluations, by Katharine Conn from Columbia University. Here is a subset of findings (emphasis added):

I identify educational interventions with an impact on student learning in Sub-Saharan Africa. After a systematic literature search, I conducted a meta-analysis synthesizing 56 articles containing 66 separate experiments and quasi-experiments and 83 treatment arms…. A key finding is that programs that alter teacher pedagogy or classroom instructional techniques had an effect size approximately 0.30 standard deviations greater than all other types of programs combined. Limited evidence further suggests that pedagogical programs that employed adaptive instruction or teacher coaching were particularly effective

In case you don’t have access, the earlier, open-access dissertation version has the same sample and the same findings reported in the abstract. 

That version was one of six reviews that Anna Popova and I synthesized in our paper What Really Works to Improve Learning in Developing Countries? An Analysis of Divergent Findings in Systematic Reviews (open-access version).

Nothing new under the sun — Education edition

Some years ago, I was evaluating an education program in The Gambia (read the evaluation here), soon after the government had outlawed corporal punishment in schools. We included a question about it in the evaluation and learned that there was a gap between legislation and practice, which the government then sought to resolve. 

Of course, controversy over corporal punishment in schools isn’t new, but I was surprised to see it debated in Noli me tangere, the 1887 novel by Filipino writer José Rizal. A frustrated schoolteacher recounts that after reading several books, his views changed: 

Lashings, for example, which since time immemorial had been the province of schools and which before I had seen as the only effective way to make children learn (that is how they have accustomed us to believe), began to seem far removed from contributing to a child’s progress, completely useless. I became convinced that when one keeps the switch or the rod in view reasoning is impossible… I began to think that the best thing I could do for these children was to develop confidence, security, and self-esteem.

So he eliminates corporal punishment. 

Little by little I held back the switch. I took the whips home and replaced them with emulation and belief in oneself.

Like any good experimenter, he evaluated short and medium run impacts.

In the beginning it seemed as though my method was impractical: a lot of them stopped studying altogether. But I pressed on, and I noticed that little by little their spirits rose. More students attended class, and more often. And when one day one was praised in front of everyone, the following day he learned twice as much.

But the local priest and the parents didn’t buy it and demanded he return to the traditional system.

I had to renounce a system that after a great deal of effort had begun to bear fruit.

Poor guy, but I imagine there are a number of reformers today who can feel his pain. 

The quotes are from Harold Augenbraum’s translation of the book.