+ (at the end) one hypothesis for why published randomized trials could disagree more than non-experimental studies
This week’s Lancet has a paper about the effect of treating schoolchildren for malaria in Kenya, by Sian Clarke, Matthew Jukes, and several other people. The study seems to have been very well done. Children in some schools received malaria treatment and children in other schools received placebos: Which were which was “revealed to the investigators only after completion of the statistical analysis.”
Children in treatment schools were healthier and had longer attention spans, but “no effect was shown for inattentive or hyperactive-compulsive behaviours or on educational achievement.”
This study is interesting because the intervention doesn’t just treat kids with malaria: it treats all kids. Apparently (from the article and the associated podcast), children who grow up in areas with lots of malaria develop a degree of immunity to malaria flare-ups (with the fevers and other symptoms with which I am intimately familiar) but still can have malaria in their system which has other negative effects like anemia.
The authors conclude “Effective malaria interventions could be a valuable addition to school health programmes.” In the podcast (July 12), the always charismatic World Bank researcher Don Bundy explains that the study is being replicated in Senegal and in a different part of Kenya.
The lack of effects on school performance stands in contrast to a recent study in Sri Lanka that found significant impacts of malaria treatment on children’s school performance. This reminds me of Worrall’s paper (which I blogged about earlier) which pointed to a greater tendency among randomized trials to see disparate results than among non-experimental trials. One plausible explanation for this is that non-experimental trials are subject to greater publication bias: A non-experimental trial has to meet a higher bar to publish a dissenting result than a randomized trial. This could lead to more variance in published experimental trial results.
I don’t find it particularly surprising that we have lots of variation across randomized trials. Contexts are different. Heartening? No. Surprising? No.