Reliability of Self-Reported Data - Diaries and Alternative Methodologies

In last week’s blog post, I suggested that self-reported data should be supplemented with objective sources of information from independent third-party entities. Sometimes, however, independent data sources simply aren’t available and researchers have no choice but to base their analysis on self-reported data. Under these circumstances, some data collection methodologies might be more useful than others in ensuring that self-reported data are reliable. In this post, I discuss several studies of the potential of the diaries methodology and alternative strategies to capture accurate self-reported data.

Klaus Deininger, Calogero Carletto, Sara Savastano and James Muwonge examine the effect of personal diaries on the quality of self-reported agricultural data in their study, “Can Diaries Help in Improving Agricultural Production Statistics? Evidence from Uganda.” In Uganda, a large part of crop output consists of continually harvested crops such as cassava and banana. Since these crops are harvested over long periods of time, farmers who are asked to report harvest data may have trouble recalling events that happened several months earlier . . . 

Read More

Wagner’s Jonathan Morduch on Reimagining Banking for Half the World’s Adults

Half of the adults in the world are “unbanked” -- about 2.5 billion people. That’s the starting point of a new book, Banking the World: Empirical Foundations of Financial Inclusion, published by the MIT Press.

To reach those 2.5 billion people, NYU Wagner professor Jonathan Morduch argues that we need to think about banking in radically different ways. Promising solutions involve using new technologies like mobile phones, as well as re-imagined ideas like self-governing village-based saving groups. Understanding those possibilities is a focus of the The Financial Access Initiative, the NYU center Morduch founded with colleagues at Yale and Harvard. Morduch co-edited Banking the World . . . 

Read More

Reliability of Self-Reported Data: Deliberate Misreporting

Program evaluations and policy proposals are only as good as the data upon which they are based. Although we all know this to be true, discussions about the reliability of data, especially self-reported data, have only recently emerged in the field of development economics. The other week, I highlighted two papers from the Journal of Development Economics’ Symposium on Measurement and Survey Design which discussed how recall bias might undermine the reliability of self-reported data. Even when recall bias is not at play though, self-reported data might be threatened by respondents’ desire to misreport their activities so as to portray their behaviors in a more positive light.

Sarah Baird and Berk Özler explore this phenomenon as it relates to education in their study, “Examining the Reliability of Self-Reported Data on School Participation.” Many Conditional Cash Transfer (CCT) programs are evaluated based on self-reported data about school enrollment and attendance rates. However, the desire to give socially desirable answers or the belief that program funding is linked to evaluation results might lead survey participants to over-report their level of school participation. Baird and Özler test the extent to which self-reported data of school enrollment rates can be considered reliable in CCT evaluations of this nature . . . 

Read More

Week of May 6, 2013

Payments, Cash and Geographic & Economic Mobility

Right now there is a lot of talk about allowing more geographic mobility to enable more economic mobility--in other words, easing immigration restrictions. There is powerful evidence that enabling more migration (internal and external) would be a powerful tool to fight global poverty.

But there is a different kind of geographic and economic mobility that is worth thinking about--the geographic and economic immobility of cash. 

A just-for-fun project to track the movement of specific dollar bills as they move from place to place and person to person has yielded very interesting data on this issue in the United States. Back in 1998 the Where's George project started encouraging people to log the geographic (by zip code) location of their cash before spending it . . . 

Read More

A Must Read from Chris Dunford on Research-Practitioner Relationships

A regular theme in our writing is about the need for the microfinance industry to learn from and adapt to the needs of poor households. A few weeks ago, a new paper appeared based on an interesting attempt to test whether MFIs are interested in generating and using rigorous evidence. The researchers sent emails to 1,419 MFIs inquiring about their interest in "a partnership to randomly evaluate their programs." There were three different emails sent however: 1) a neutral email, 2) an email that emphasized positive findings from other studies of microfinance, and 3) an email that emphasized "null" findings from other studies of microfinance. 

Unsurprisingly, the positive emails had double the response rate of the negative emails. The authors interpret this finding as evidence of confirmation bias among MFIs--they are only intereted in good news that backs up their existing beliefs, and less interested in learning how to improve . . . 

Read More

"How Microfinance Really Works" - Jonathan Morduch in Milken Review

It's an important moment for the microfinance movement. At a time when real progress has been made in making financial services available to the poor, questions abound about the effectiveness of microfinance as a way of helping people escape from poverty. The priveleged position microfinance has enjoyed among poverty interventions and social investment is eroding. Charting the right path forward for microfinance--and effective investments in reducing poverty--requires a closer look at how microfinance really has worked . . . 

Read More

Week of April 22, 2013

What is the Impact of Muhammad Yunus?

Muhammad Yunus spoke to an overflowing crowd at NYU on April 15, an event jointly sponsored by the Wagner School of Public Service, Stern School of Business, and Financial Access Initiative.

Professor Yunus is known for fighting to improve the lives of millions of poor families around the world, the quest that was celebrated by the 2006 Nobel Peace Prize. These days there is a lot of talk about the impact of microcredit. But here was an opportunity to ask: what is the impact of Yunus? Given where we were, more specifically, how has Yunus changed the way we--economists, academics, policy makers and influencers--think about problems?

Read More

Yunus, Entrepreneurs, and Employees

We had the good fortune to host, with NYU Wagner and NYU Stern, a talk by Muhammad Yunus today at FAI. If you couldn't join us in the room or via the livestream, you can read the tweetstream from the talk by searching Twitter for #FAIYunus, and soon we'll post video of the event. 

In the meantime, I wanted to offer some quick thoughts about one of the main topics that Professor Yunus addressed: entrepreneurship. During his talk, Yunus mentioned his advice to the children of Grameen Bank borrowers who have completed school: don't be a job seeker, be a job maker. In other words, be an entrepreneur. During the question and answer session, a member of the audience asked Yunus, "[You] assume everyone is inherently entrepreneurial. What about people who are not?" Yunus' response was that people who are not entrepreneurial have simply not been given a chance to discover and develop that side of themselves . . . 

Read More

Week of April 8, 2013

Measuring (and Missing) Financial Inclusion

The fastest growing part of the financial inclusion movement isn’t a product or even a standard, it’s data and measurement. And if there’s something experts are increasingly agreeing on, it’s that it is illusory to try to define financial inclusion in any precise, universal way. John Gitau says he’s confused, and so am I. How do you measure financial inclusion?

It’s true that you might not be able to measure financial inclusion itself, but you can still measure things that indicate either actual, or the potential for, progress. Such indicatorsare what we can measure, and they are very useful as long we don’t confuse them with actual measurement of financial inclusion.

There are two broad types of indicators which can be applied to fuzzy concepts like our cherished financial inclusion . . . 

Read More

Beyond Business: Rethinking Microfinance

In just 30 years, the microfinance movement has reached 200 million people who had been deemed "unbankable." That's a stunning success. But the narrative that drove this success has implicitly shut the vast majority of the unbanked out of the system. That's why it's time to change the story, and our minds, on how microfinance works, argue FAI's Jonathan Morduch and Timothy Ogden in Foreign Policy. They suggest that the fundamental need of poor households is tools to smooth out volatile and uncertain cash flows, not credit for business investment . . . 

Read More

Financing Seasonal Migration: A New Use for Microcredit?

In many places, agriculture is highly seasonal. That presents difficulties for subsistence farmers who have to stretch incomes year-round. If farmers (or family members) could migrate during the off-season to areas where wage labor is available, they could substantially smooth their annual income and consumption. Indeed, this is what happens in many places. But even where seasonal migration does happen, many people don't migrate even when it seems it would be advantageous to do so. Why?  

In recent work, Bryan, Chowdhury and Mobarak study the role of risk aversion in preventing households from migrating. You can see a presentation of this work here.

While migrating from poorer areas to wealthier ones, particularly from rural to urban locations, can provide access to more and (much, much) better-paying wage labor, it comes at a significant cost . . . 

Read More

Worse than AP: The Damage of a Repayment Crisis in Chiapas

A month ago I wrote a post singling out the Mexican state of Chiapas as a potential site of a coming repayment crisis. No, this is not a follow-up announcing that it has begun, nor am I rooting for one to start. In my next post, I will review the options that the Mexican microfinance sector has to avoid it, and what the global microfinance community can do to help. But for now, let’s dig a bit deeper into what a Chiapas crisis might mean, and why I continue to focus on Mexico, as opposed to the broader issue of excessive credit and over-indebtedness.

Let’s be blunt: not all countries are created equal. Some remember my warning three years ago about the danger of a credit crisis in Andhra Pradesh. Back then I compared a possible crisis in India to the crisis in Bolivia a decade before: "India is no Bolivia – if the bubble bursts there, the entire global microfinance sector will find itself reeling." Well, Mexico is no India . . . 

Read More

Reliability of Self-Reported Data – Recall Bias

In a recent post, Tim Ogden and I discussed the importance of having solid, reliable data on which to base program evaluations and policy decisions. The Journal of Development Economics explored this theme in last year’s Symposium on Measurement and Survey Design which featured more than a dozen papers on improving data quality in development research (Hat tip to Berk Ozler of the World Bank’s Development Impact blog for pointing us to it).

An important discussion at the symposium was the extent to which self-reported data can be considered accurate and reliable. Because study participants are usually asked to report information after significant time has elapsed, self-reported data are often subject to recall bias and can be inaccurate or misleading. This post is the first in a three-part series that will explore the reliability of self-reported data through a discussion of papers featured at the symposium . . . 

Read More

Gender and Mobile Money

The mobile money revolution has been greeted with great excitement in some circles for the potential it holds to increase financial access for the world’s poorest. Women may especially benefit from expanding financial inclusion through mobile financial services (MFS). Women not only handle a lot of cash to provide for household needs in many societies, but they may be explicitly or implicitly discouraged from using bank branches.  A new report (sponsored by Visa and the GSMA mWomenProgramme) by FAI affiliate Daryl Collins explores just this theme. In “Unlocking the Potential: Women and Mobile Financial Services in Emerging Markets,”  Collins and her coauthors explore the untapped potential of woman as a strategic consumer base for MFS providers.

A summary of the report’s conclusions . . . 

Read More

Update: Another paper on microinsurance and why insuring against risk matters

We wrote a post a few months ago about a paper  that looks at how microinsurance affects decision-making. Specifically, the paper analyzed whether insuring farmers in Andhra Pradesh, India, against rainfall-related risks (too much or too little) affected their investment and production decisions. Another recent paper by Karlan, Osei, Osei-Akoto & Udry uses a similar approach and found similar results in northern Ghana.

In this experiment, farmers in northern Ghana were randomly assigned to receive cash grants, subsidized rainfall index insurance or simply offered the option to purchase insurance at actuarially fair rates. As in the Andhra Pradesh study, the authors find strong responses to investment to the subsidized rainfall insurance. When provided with insurance against one of  the main risks they face, farmers find the resources to increase expenditures on their farms. Interestingly, they only found small effects of cash grants.

Karlan, et. al. also discuss the demand for rainfall insurance. They point out that take-up rates on rainfall insurance are low, which seems counterintuitive given that we know risk discourages investment, and investment often has large returns. To understand the demand for rainfall insurance, we have to keep in mind the various factors that drive demand. Of course, price is important but trust and experience matter as well. Demand is sensitive to whether farmers trust that payouts will be made. They found that demand for insurance increased after a farmer or someone in his network received a payout.

The study provides additional evidence that households are managing their finances by avoiding risks. In other words, they are smoothing their income by avoiding investments that could pay off but could go very sour. Reducing this risk-avoiding income smoothing through insurance seems to work well—if households can be convinced that the risk of the insurance (that an unfamiliar and relatively expensive product is reliable and worth the short-term cash cost) is less than the risk of the status quo.