At a recent microfinance conference hosted by Innovations for Poverty Action, the Financial Access Initiative and Yale University, the Philanthropy Action editors sat down with Esther Duflo and Abhijit Banerjee, two of the founders of the Abdul Latif Jameel Poverty Action Lab (JPAL) at MIT and IPA Research Affiliates. Duflo, Banerjee and the other JPAL and IPA economists apply the rigor of randomized controlled trial techniques (the same approach used by the medical industry to determine if a drug or treatment does what it was designed to do) to poverty interventions to identify whether or not a program is effective. Below, they highlight the poverty interventions they view as consistently effective and provide insight into where individual donors can make a true impact.
Philanthropy Action: One thing we always like to know from people like yourselves, who spend so much time in the developing world working directly with the poor, is what you would do if you had a million dollars to put toward any poverty alleviation cause.
Esther Duflo: At this minute I would give it to incentives for immunization. People in very poor countries spend very little on preventive care, maybe for not very good reasons. It is not that they are strongly opposed to immunizations, they just somehow don’t get around to it, or they have other things to do. And what you realize is that spending even a tiny amount of money to get them to invest in preventive care can do a lot. In one project we gave away a kilo of lentils for each immunization given to a child, and it had a huge impact on getting parents to bring their children in for immunizations.
Abhijit Banerjee: Ten times – the impact was a factor of ten.
ED: So there are variants of these programs with various goods. I happen to think that immunization is something we know makes a huge difference in people’s lives so that is my personal favorite at this minute.
AB: I would go with that.
PA: That makes me think of what in technology is referred to as the “last mile” problem. Meaning, the problem so often is not that there isn’t enough money. It is not, for example, that there aren’t enough anti-malaria bed nets, but that there are not enough people using the bed nets, or in your example, taking the immunizations. Are there other interesting things out there aimed at solving the last mile problem?
AB: Within healthcare, one of the projects I think both of us saw some success with has to do with TB (tuberculosis) medicine and getting people to take them properly. This is a problem we have been thinking about for many years. We were asked to set something called the Yunus Challenge [named after the Nobel Prize winner] which challenges students to come up with designs of practicably implementable solutions which can solve a specific problem. The one we picked was how to get TB treatment compliance.
A bunch of students came up with a brilliant solution – at least conceptually brilliant – which is now being subject to a randomized evaluation in Pakistan. The idea is very self-incentivizing: we look for traces of the medicine in a patient’s urine. In the program the patient puts their urine on a test strip, and if they’ve taken the medicine it reveals a number or code which they can put into their cell phone and get extra minutes. In some ways it is a brilliant solution, but it may not work. The need for experimentation is based on the fact that theory fails often. But this is an idea where the concept is very clever and we are looking for the last mile – how do we incentivize people? If it works, it will create a model for more technologically sophisticated solutions for these kinds of problems, which I think it is a very good direction to go in general.
ED: There are other kinds of last mile issues as well to do with the people who are supposed to deliver the service: they need incentives too. I don’t know if you would call that the last mile…
AB: That’s kind of the first mile.
ED: [Laughs.] It’s sort of in between. The immunization project I spoke about suggests that even when you fully fix the supply problem and immunization services are available, you still need to nudge people a bit. You need to provide an incentive for the parent to bring the child and you need to provide an incentive for the service provider to be there as well. So this is also a very important area for research and projects – thinking about how to get the services to the people.
PA: When you read about where most private philanthropic dollars and official aid go, a lot of it goes into health, education and other areas that Oxford economist Paul Collier refers to as ‘photogenic subjects.’ Do you think the money going to these sectors is being used well?
AB: I think Collier’s implied criticism is in the wrong place. If you had to think about things that are unambiguously good, a lot of them are in this area of health and education. Especially in health, I think there are just a lot of dollar bills left on the ground and somebody needs to pick them up. Certainly, other programs might have more impact on economic growth, but if you think about the immediate human consequences it is hard to argue against health and education.
The real issue is often in program design. Our endlessly repeated line is that details matter infinitely. The difference between successful programs and unsuccessful programs is not the sector, it is how you do it. Has the implementer thought completely about the reasons why a program might not work? In my experience, even when you talk to very competent, well-meaning organizations, that is the step where you see the biggest gap. Perhaps because the paradigms are easily available to jump into, the last mile thinking just doesn’t happen.
Details unexpectedly and often in unfortunate ways determine everything. You get the details wrong and you got it all wrong. So we often end up ritualistically talking about big picture research, and how this didn’t work or this did work. But in practice those big picture issues probably do not matter. What matters is that some programs are well designed and some are not. Our goal very much has been to draw attention to the detailing of that.
Take water – within water there are any number of special solutions, and then a couple of things that seem like good ideas. The most valuable work is not in distinguishing water from health, but in separating the many meretricious ideas that get lots and lots of coverage and turning them into practical programs that are not overly complicated or unnecessarily expensive.
There is a tendency to think, “I have a hammer so everything is a nail.” Everyone has their own particular bias and everything has to go through that filter. Ideology is a huge reason why you get bad design.
PA: On that point specifically, what would you say is the biggest wrong belief currently present in development circles?
ED: The biggest wrong belief is that there is a magic bullet and I can find it.
AB: I can find it introspectively.
ED: That is the most dangerous line of thinking, because there are a lot of reasons why a person is poor and a lot of reasons why poor people stay poor. And to say that there is one program or one solution is just a problem.
AB: There seems to be a real axiom out there that says you can put people in charge of their own destiny and walk away. Yet when you think about it, these people are under enormous stress in every respect. Their child is sick, their parents are dying, they have no income. Every day they have to go out and find work. To expect them to be able to have the leisure or perspective to always be able to take charge of their larger destiny – to operate the political systems and find opportunities? I wouldn’t say that is the biggest wrong idea, but it is just a very untested idea.
I suspect that in the aggregate it is wrong. Meaning, that if you picked one or two things that you said people have to be involved in, they would probably do a pretty good job. But if you think of the number of things we are beginning to expect people to take care of themselves it quickly gets overwhelming. We want them to be on the forest maintenance committee, the environment maintenance committee, the water committee, the education committee, the village health committee. They have to take a microcredit loan and pay that down and start a small business, at the same time that they have to make sure the children are getting homework help. If you think of the number of things we impose by default on them, I think that is an enormous challenge.
PA: That speaks to this line of thinking that says, well, you can’t do everything so pick a focus. But if you don’t do everything then it’s just a drop in the bucket and you won’t have an impact.
ED: Not necessarily. There are many things that have a big impact. There are quite well targeted programs and they are designed to do one thing and they do this one thing very well. I think another untested and potentially wrong idea is that you have to do everything at the same time or else. This is a pretty convenient untested belief because if you live in that world, it is almost impossible to evaluate what you do.
Whereas if you say, I am going to press on this button and see whether it provides this result, you might find there are many things that do work surprisingly well with surprising consistency. So it is not that the world is so incredibly complex that every place needs a unique combination of five factors just to produce anything. I don’t know that we would have been able to say the same thing five years ago, but now we are starting to be in the position to say that a number of things, if well designed, just work pretty well in a lot of contexts.
For example, remedial education programs and more generally education programs that are explicitly targeted to deliver a specific set of competencies to children who do not have them. Or, as I mentioned, small subsidies or token incentives to get people to do things (mostly related to health and education) that are good for them but that for some reason do not get done without a nudge. Conditional cash transfer programs or deworming initiatives I would also place in that category.
PA: You’ve said before, Dr. Duflo, that what happens often is that a theory about a program that might work gets developed and implemented in one environment and one group of people and one approach and then evaluated, and if results aren’t evident quickly then people throw up their hands and say, well that doesn’t work and move on.
ED: What I was more saying is that these theories come out and a program is tried, but we never really learn from it. And then somehow it goes out of fashion and we move on to the next program. I think already one experiment is a quantum improvement over ze ro. More is better, but it is not usually what we have. Look at microfinance. There is no evaluation yet of the impact of a microfinance loan – we have the first preliminary results ever of the impact of a plain vanilla, group lending microfinance model. That’s it. It is not as if there have been mixed results before now. The studies don’t exist. And that is microfinance, where there are already a hundred economists studying it.
Microfinance fortunately did not go out of fashion before [randomized evaluation] came into fashion, so we have a chance to have a meeting of minds here. But other things did. Take fertilizer subsidies: at first they were fashionable, now they are unfashionable. In the meantime we have not learned what fertilizer subsidies do. They might be good or bad, at this point I don’t know. They’re coming back in fashion, by the way. All of this without a single piece of evidence about whether a subsidy changes the demand for fertilizer. This is just one example of what to me is the biggest mistake, which is doing the same things over and over and over again without learning from the experience, whether it is fertilizer subsidies or microfinance.
AB: It seems to me part of the issue is that the expectations are also set too high. If the goal of microfinance is to change people’s lives it will have been deemed already to be a failure, despite the fact that results are showing it is doing some reasonably good things. I think this comes from a particular development mindset, also untested, which is that all that is needed is one big thing to happen and things just—
ED: The “big push”…
AB: —explode. I don’t think there is any evidence that things ever happened that way. Look at the US. Growth was very slow over a long time and accelerated slowly over time. It wasn’t seen then as a disaster; the view was generally that the US was doing fine. But suddenly we are in this world of accelerated expectations, so it is easy to fail.
So, if microfinance suddenly doesn’t make all babies do calculus by the age of five, it is deemed a failure. I think if we just accept that in the end a few things work and somehow that gives people the spirit or the enthusiasm or the hope to act a bit more and enable those things to accumulate on their own.
We don’t really have levers to make everything work and I don’t think anybody ever did. I think a few things worked. In the US, public education did some work. It took time, and then things accumulated. I don’t think we really know which things cumulate with which other ones and getting all of them to fit together and start moving along the same trajectory – that just seems real pie in the sky.
ED: Going back, I want to return to that perception that if you don’t do everything at once what you are doing is nice, but will not have any real impact on development. I accept that there are certainly big ideas, and then there are small ideas. But the notion that development and improvement in welfare is not an accumulation of small things is—
AB: —As yet untested.
ED: The small thing often is what there is.
AB: Maybe that is what you can do – a few small things – and hope that somehow people’s innate desires are sufficiently aided by those things. That is not at all an unfair reading. It is untested as well. But there is this presumption that big changes need big levers. There is no evidence that big changes were ever wrought by big levers.
PA: Speaking of big levers, many people talk about education as a big lever. So there has been a lot of attention on increasing attendance rates for children. Certainly improved attendance has shown to be a good first step but it’s not translating into improved performance for those kids. The small, overlooked thing seems to be to improve the instruction, improve infrastructure, improve teacher attendance. What is your latest work showing in this area?
ED: The bottom line on that is that I don’t think it is so hard to get students and teachers to come to school if you give them incentives to come to school. The problem is that a lot of the teachers don’t have those incentives. One thing we are finding is that pedagogy is very important. One problem that seems to be common across a lot of the curricula is that they are just not adapted to the students.
AB: All this in one direction: they are too hard.
ED: Too hard for the students, and therefore I think very depressing for the teachers. Part of the teachers’ lack of intrinsic motivation is a feeling of helplessness: What can they do? They have illiterate children to which they are expected to teach multivariate calculus, and it is just not happening.
So there are a number of things people have done – I’ve done some work on computer assisted teaching and on tracking teacher attendance. I’ve also done some work with Pratham [an Indian NGO] on below-remedial education such as reading programs. Along those lines we are now working on ways to cut the curriculum into pieces that are achievable by people at various levels. Maybe it is not rocket science, but children will be able to learn and teachers will be able to teach if the goal is achievable. In part, a lot of these countries have inherited curricula that are really colonial curricula designed to take the elite among the native children and teach them enough of what they need to become tax officials or something. It’s often completely not relevant.
PA: So we need to save the world by lowering our expectations?
ED: Well…for example take the health service in India. One reason it is a disaster, and I am talking about truly a disaster – 30 percent presence, five percent immunization – is because some bureaucrat in the office dreamed up all these things that the nurses have to do, which nobody can accomplish. They are required to be in five places at once. So if you have so many things to do it is very easy to tell a visitor, I am sorry I had to be somewhere else, and that is why I was not doing my job.
AB: I think, even less cynically, in a system which is totally unrealistic you feel you are a failure in any case, so then you might as well be a total failure.
PA: It is very easy to create cynics in development.
AB: Absolutely. And I think the system is very good at creating cynics, because it creates a set of models which are beautifully sketched out by some ministry or the WHO, people who sit in their offices and tell workers on the ground what steps to take on which issue. Once the worker has done it they will have to write a memo to be evaluated by the ministry. But there are two million of these things, because everyone has their own idea of what you should be doing. And then you get cynical very quickly, because the deal is that implementing these things is not practical and nothing bad happens to you anyway if you don’t do it.
ED: And I don’t think it is a matter of lowering expectations, because the WHO official is so far removed, and the Indian bureaucrat does not have any expectation at all – in a way his are the lowest expectations. So we are trying to raise expectations but make them real.
PA: We see the exact same dynamic happening between donors and NGOs. The donor won’t give unless the NGO sets this unrealistic expectation, but then the donor becomes a cynic because the unrealistic expectation that he forced the NGO to accept fails to have any impact.
ED: That is exactly right and that is exactly this culture that we are trying to change. To say, okay, you are going to see what the impact of this program is and you can agree to make mistakes.
Part of the unrealistic expectation is also the inability to fail. You can not admit that you got it wrong, ever. If you’re a government with the wrong program you won’t get reelected, if you are donor or an NGO you can never admit it either and that is a real problem. Because for a donor, even a big donor, the amount of money given is very small compared to what developing country governments spend. So the best thing to do with that money really is to use it in an experimental way. Take risks. Accept failure. Try things. Because for a government, it is always going to be hard – governments cannot fail, they have to be elected. Donors are allowed to fail. They should act as venture capitalists, try things, let people fail, but at least be accountable to documenting either success or failure and learn from the mistakes.