Rigorous Evaluation: Not an Afterthought

There's a new piece in Foreign Policy magazine which takes a tough look at Jeff Sachs and the Millenium Villages Project--not in regard to results or interventions, but in regard to evaluation. The project was always pitched as a demonstration of a "different" approach to ending poverty that could provide a blueprint for addressing poverty globally. 

As the piece explains--citing FAI's founder Jonathan Morduch, FAI Affiliate Michael Clemens, Ted Miguel from UC-Berkeley and Nancy Birdsall from CGD, among others--that is no longer a realistic option. The project wasn't structured to allow for the kind of rigorous evaluation that would give it credibility as a demonstration or a justification for scale-up. While it seems there is now an effort to do more rigorous evaluation, for most aspects of the project it is simply too late to establish the comparisons and baselines necessary for credible claims of impact. 

The MVP is far from alone in failing to develop rigorous evaluation of new projects. It took almost 30 years for rigorous evaluation of microcredit to be done, for instance. Coincidentally this week, Knowledge@Wharton has released a new ebook for social entrepreneurs (The Social Entrepreneur's Playbook) about how important it is to test their ideas and plans before committing themselves. While the book doesn't really address rigorous impact evaluation, it is a good start for anyone seeking to launch a new project with uncertain outcomes. You can read an interview wiht the authors here. Note that you can download a copy of the ebook for free until July 15th via a link at the bottom of the interview). 

For any who would like to take up the ideas of the Social Entrepreneur's Playbook and start the process of rigorous impact evaluation, a new web tool called RandomiseMe was launched last week. It's a simple start to doing randomized evaluations and worth a look for anyone who doesn't know where or how to start.