1. Financial Inclusion: I [Sean] organized a session on savings and financial inclusion that looked at the impact of various savings interventions such as commitment devices, opt-out savings plans, and mobile money. Continuing last week’s theme on similarities between developed and developing countries, a savings intervention that has greatly increased savings in the US is opt-out savings plans or “default assignment,” such as being automatically enrolled in a 401(k) plan. In an experiment in Afghanistan, Joshua Blumenstock, Michael Callen, and Tarek Ghani explore why defaults affect behavior: some employees are defaulted into a savings program where 5% of their salaries are automatically deposited in a mobile money savings account, but they can opt out at any time. Those who were defaulted in were 40 percentage points more likely to contribute to the savings account, which is comparable to the effect of the employer matching 50% of employees’ savings contributions
Commitment savings accounts have also been tested in the US and in many other countries. In a study by Emily Breza, Martin Kanz, and Leora Klapper, employees in Bangladesh were offered a commitment savings account, with a twist: depending on the treatment arm, employers sometimes endorsed the product, and employees were sometimes told that their decision would be disclosed to the employer. Only the treatment arm that had both employer endorsement and disclosure of the employee’s choice led to higher take-up, suggesting that workplace signaling motivated employees to save. Another study by Simone Schaner et al. (covered in last week’s faiV) offered employees in Ghana a commitment savings product with the goal of building up enough savings to stop incurring overdraft fees, which are common. Take-up was high, but baseline overdrafters were more likely to draw down their savings before the commitment period ended -- meaning they benefited less from the intervention.
Two important barriers to financial inclusion in the US and around the world are transaction costs and low trust in banks. In a paper I coauthored with Pierre Bachas, Paul Gertler, and Enrique Seira, we study the impact of providing debit cards to government cash transfer recipients who were already receiving their benefits directly deposited into a bank account. Debit cards lower the indirect transaction costs -- such as time and travel costs -- of both accessing money in a bank account and monitoring the bank to build trust. Once they receive debit cards, beneficiaries check their balances frequently, and the number of checks decreases over time as their reported trust in the bank and savings increase"
2. Household Finance: Digital credit is a financial service that is rapidly spreading around the world; it uses non-traditional data (such as mobile phone data) to evaluate creditworthiness and provide instant and remote small loans, often through mobile money accounts. One of the concerns about digital credit is that customers’ credit scores can be negatively impacted, even for the failure to repay a few dollars. In turn, this can leave them financially excluded in the future. Andres Liberman, Daniel Paravisini, and Vikram Pathania find a similar result for “high-cost loans” in the UK (which we would call payday loans in the US). They use a natural experiment and compare applicants who receive loans with similar applicants who do not receive loans to study the impact of the loans on financial outcomes. For the average applicant, taking up a high-cost loan causes an immediate decrease in the credit score, and as a result the applicant has less access to credit in the future.
Read More1. The Economics Production Function: Over the last few years, papers on microenterprises generally shared a couple of remarkable--given the general narrative--findings: microenterprises (on average) didn't grow no matter what you did to try to boost them, and women-owned microenterprises performed worse than male-owned ones. Those findings led to plenty of yowls from practitioners whose work, livelihoods and in some cases core beliefs were based on the opposite. In many conversations I had, I got the impression that people outside the profession believed that economists would publish these findings and then move on. But that perception really misunderstands the motivations of economists and the way the field works. Economists don't leave puzzles alone once they find them--the field pursues them relentlessly.
The best session I attended this weekend was based on the particular puzzle of why female-owned microenterprises are less profitable. Natalia Rigol presented work following up on an earlier studies that documented the profitability gender gap, finding that the source of the gap is mostly due to lower returns from female-owned enterprises where there was another (male-owned) enterprise in the household. Those male-owned enterprises were in more profitable industries (something documented in the original studies), so the households were making quite rational decisions to allocate additional funds to the more profitable business (and making it look as if the female-owned business had 0 or negative returns). In households where there was only a female-owned business there is no gap in returns to capital. Leonardo Iacovone and David McKenzie presented on efforts in Mexico and Togo, respectively, to provide training to help women entrepreneurs improve their businesses with positive results--in both cases seemingly based on personal initiative training rather than business skills. And Gisella Nagy presented results (unfortunately there's nothing yet to point to on this one) that women tailors in Ghana show lower profitability than male tailors because there are more women tailors which drives down prices they can get in the market. This last finding is particularly important because it suggests that part of the way forward for microcredit aimed at building women's businesses is to do a much better job targeting, or as I've called it elsewhere, abandoning the vaccine (everyone gets one!) model of microcredit for an antibiotic (only people who really need it get one!) model.
And all of that is just a very small sample of work being done on the puzzle of heterogeneity of returns to microenterprises and what can be done about it. I'm now sorely tempted to write an overview on all these studies, but dammit I really want to get to "subsistence retail."
2. Causal Inference is Hard: Those two topics aren't orthogonal to each other of course. One way they are joined together is my common theme about how hard causal inference is for the average person, and in particular for the subsistence (or just above) operator of a microentrprise (whether farming or retail). That's what I kept thinking about when reading this new post from David McKenzie on "Statistical Power and the Funnel of Attribution". David is writing for economists trying to write convincing papers, but this point "Failure to see impacts on your ultimate outcome need not mean the program has no effect, just that the funnel of attribution is long and narrows" is equally important for the people being treated. If the funnel of attribution is long and narrows, then its approaching impossible for the individual (not gifted with a large sample size or a deep understanding of statistics) to figure out which of their actions actually matter.
There is a connection to AEA here. As I was perusing the poster displays (also known as "the saddest place on earth") I kept hearing people arguing with Jacob Cosman, the creator of a poster about how the opening of new restaurants in a neighborhood affects the behavior of existing restaurants. The answer: a very precisely estimated no effect at all. (Here's a link to an old version of the paper with somewhat different results.) Economists walking by simply couldn't believe this and were constantly suggesting to the author things he must have done wrong. I was amused. My strong prior is that a person would not open one of these restaurants unless they believed that their restaurant was unique (otherwise, you would believe that your restaurant would quickly fail like the 90+% of other small restaurants and you wouldn't open in the first place). So when another new restaurant arrives, you don't actually see it as a threat that needs a response. You are, after all, different! But even if you did think you needed to respond, how would you possibly know what the right response was? Do prices matter? Menus? Advertising? Item descriptions? Coupons? The funnel of attribution on all of these is so long and imprecise we should assume that individual entrepreneurs have no idea what to do even if they wanted to do something. Which ultimately brings us back to why it's so hard to get microentrepreneurs to change their behavior in a lasting way, and why personal initiative training may work much better than business skills. Personal initiative training teaches you that what you do matters, even if you can't tell, while business skills training teaches you to do something even though you can't tell that it matters.
Read More1. Social Investment: Last week I was at European Microfinance Week. Video of the closing plenary I participated in is here. My contribution was mainly to repeat what seems to me a fairly obvious point but which apparently keeps slipping from view: there are always trade-offs and if social investors don't subsidize quality financial services for poor households, there will be very few quality financial services for poor households.
Paul DiLeo of Grassroots Capital (who moderated the session at eMFP) pointed me to this egregious example of the ongoing attempt to fight basic logic and mathematics from the "no trade-offs" crowd. This sort of thing is particularly baffling to me because of the close connection that impact investing has to investing--a world where everything is about trade-offs: risk vs. return; sector vs. sector; company vs. company; hedge fund manager vs. hedge fund manager. The logic in this particular case, no pun intended, is that a fund to invest in tech start-ups in the US Midwest is an impact investment, even though the founder explicitly says it isn't, because it is "seeking potential return in parts of the economy neglected by biases of mainstream investors." If that's your definition of impact investing you're going to have a tough time keeping the Koch Brothers, Sam Walton and Ray Dalio out of your impact investment Hall of Fame. Sure, part of the argument is that these are investments that could create jobs in areas that haven't had a lot of quality job growth. But by that logic, mining BitCoin is a tremendous impact investment. You see, mining BitCoin and processing transactions is enormously energy intensive. And someone's got to produce that energy, and keep the grid running. Those electrical grid jobs are one of the few high paying, secure mid-skill jobs. Never mind that BitCoin mining is currently increasing its energy use every day by 450 gigawatt-hours, or Haiti's annual electricity consumption. And, y'know, reversing the trend toward more clean energy. Hey anyone remember the good old days of "BitCoin for Africa"?
2. Philanthropy: There are plenty of trade-offs and questions about impact in philanthropy, not just in impact investing, and not just in programs. Here's a piece I wrote with Laura Starita about making the trade-offs of foundations investing in weapons, tobacco and the like more transparent.
I could have put David Roodman's new reassessment of the impact of de(hook)worming in the American South in early 20th century under a lot of headings (for instance, Roodman once again raises the bar on research clarity, transparency and data visualizations; Worm Wars is back!; etc.). The tack I'm going to take, in keeping with the prior item, is the impact of philanthropy. The deworming program was driven by the Rockefeller Sanitary Commission and is frequently cited, not only as evidence for current deworming efforts, but as evidence for the value and impact of large scale philanthropy. Roodman, using much more data than was available when Hoyt Bleakley wrote a paper about it more than 10 years ago, finds that there isn't compelling evidence that the Rockefeller program got the impact it was looking for. Existing (and continuing) trends in schooling and earnings appear unaltered.
Ben Soskis has a good overview of the seminal role hookworm eradication had in the creation of American institutional philanthropy. His post was spurred by an article I linked back in the fall about the return of hookworm in many of the places it was (supposedly?) eradicated from by Rockefeller's philanthropy. We may need to rewrite a lot of philanthropic history to reflect that the widely cited case study in philanthropic impact didn't eradicate hookworm and may not have had much effect. And while we're in the revision process, it may be useful to reassess views on the impact of the Ford Foundation-sponsored Green Revolution: a new paper that argues that there was no measurable impact on national income and the primary effect was keeping people in rural farming communities (as opposed to migrating to urban areas). Given what we now generally know about the value to rural-to-urban migration, that means likely significant negative long-term effects.
If you care about high quality thinking about philanthropy, democracy and charitable giving in general, which I of course think you should, you should also be paying attention to some of Ben Soskis' other current writing. Here he is moderating a written discussion of Americans' giving capacity. And here's a piece about how the Soros conspiracy theories are damaging real debate about the role of large scale philanthropy in democratic societies.
In the spirit of the holidays, I feel like I should wrap up an item on philanthropy with some good news. In the last full edition of the faiV I mentioned the MacArthur Foundation's 100&Change initiative, which is picking one idea to get $100 million to "solve" a problem. For all the problems I have with that, the program is doing something really interesting, thanks to Brad Smith and the Foundation Center. All of the proposals, not just the finalists, are now publicly available for other foundations to review.
Read More1-4. An Experimental Podcast: Every month or so someone asks me if I've considered doing the faiV as a podcast. The answer is not really, because the faiV doesn't lend itself to audio at least when I'm not ranting. Also because I rarely listen to podcasts because I don't commute and realistically I'm never going to sit at my desk and listen to audio for 30 minutes or more.
But because of the Thanksgiving holiday and travel this week to European Microfinance Week I wasn't able to the faiV. So I thought it was a good time to experiment with an addendum to the faiV in podcast form. Thankfully Graham Wright of Microsave agreed to experiment with me. So we recorded a conversation about digital finance, its potential and its pitfalls, inspired by Graham's post, "Can Fintech Really Deliver On Its Promise For Financial Inclusion?
We discuss whether mission matters, barriers to adoption, the tensions in building agent networks and why everyone who says "X is not a silver bullet" is lying. All in just over 30 minutes. Give it a listen and let me know if you'd like to hear more conversations like it.
Read More1. Our Algorithmic Overlords: I saw someone joke on Twitter recently that the best way to do a literature review was to complain on Twitter that "no one is studying..." and just use the incensed replies that come pouring in. It's an interesting form of trolling. This week Cathy O'Neil, author of Weapons of Math Destruction and perhaps better known as mathbabe.org, had an Op-Ed in the New York Times saying, "Academics aren't paying attention to our algorithmic overlords." Of course, I agree with the need to pay attention--hence this regularly featured topic--but it's a curious framing that academics aren't paying attention. In fact, all of the examples she gives of areas where academics need to be paying attention to algorithms and their effects are areas of intense academic work. Say the use of sentencing algorithms. Or teacher assessments. Or dynamic scheduling.
And it's not just the specific instances. There's also work on the big picture of the use of algorithms and big data in policy making. Or simply understanding how companies will approach gathering and using data and algorithms (How could I not link to a paper so excellently titled as "Seeing Like a Market"?). Or how about a whole academic center "examining the social implications of artificial intelligence"? The conspiracy theorist in me couldn't help noting that the center, at NYU, was officially announced the day after O'Neil's op-ed which proposes an academic center, though they have a 2nd annual report on the use of AI and 10 recommendations to guide research and accountability.
I can hardly be opposed to academic research centers, but it seems to me that what's missing is not academics paying attention or research centers devoted to the topic, but a Consumer Algorithm Protection Board. Yes, this is a pipe dream given the dire outlook for the Consumer Financial Protection Board, but it is a pipe dream I'm particularly fond of. Anyone want to help me make the case for it?
2. Household Finance: Before the algorithmic overlords item gets ridiculously long, let's move on to something that could fit either under algorithms, protection boards, or household finance. Entrepreneurial Finance Lab, which uses psychometrics to assess creditworthiness, has a piece on the FICO blog about how their testing for personality traits like impulsiveness and delayed gratification predicts default rates. It's such a good example of why I've been a fan of EFL while being queasy at the same time, it almost felt like I was being trolled. On the positive side it's an operationally relevant way of assessing borrowers who otherwise would be shut out of access to credit. On the queasy side, there's apparently huge variation in different cultures (while the metric remains predictive), and real questions about the immutability of the features they are testing--which cuts both ways. If they're mutable there's a question about what we are measuring; if they're immutable, what do we do about people who lost the "present bias" lottery? It's a good thing to protect people from themselves by not offering them credit they are likely to default on, but it still leaves me queasy nonetheless.
In terms of others being trolled, here's a piece about Refinery29's ongoing series where women share a week-long financial diary, and then readers rip apart their life choices. I'm not entirely sure whether it's the ones sharing or the ones critiquing that are the trolls, perhaps both.
And since we're on the topic of diaries and I've already gone fishing for help on one research interest of mine, here's another: I read this week that more than 100,000 puertoriquenos have migrated to Central Florida since this fall's hurricanes. Wouldn't it be great to do financial diaries of those households? It's a really unique opportunity, wouldn't be very expensive to do, and it breaks my heart to see it go to waste. If you think so too, call me.
Read More1. Appropriate Frictions and End-User Behavior: A key theme of the EPIC conversations on debt from my perspective was the importance of differential frictions in access to various kinds of debt. One example: it's much more time consuming to open a home equity line of credit than a credit card account. There are reasons for that of course: we want people to be careful about borrowing against their home, because we fear the consequences for people if they default. But the cost of unsecured credit is so much higher, and various forms of debt are so interlinked, that households can end up in worse straits precisely because we tried to protect them. The true conundrum of appropriate frictions is that the process of determining the best form of credit for a household is in itself a friction that drives consumers toward those willing to provide credit without a care for its impact on the household--a somewhat obtuse but accurate way of describing predatory lenders.
This is one of the lessons from microcredit. Demand for microcredit in most contexts is actually quite low, and rarely did microcredit have much of an impact on local moneylenders. The reason of course being that taking a microloan usually involves a lot of friction, while borrowing from a moneylender is low friction. Those operating in the US will immediately see the exact overlap with payday/auto-title lending vs. working with a community development credit union.
But it's not just a question of the behavior of consumers. Front-line staff also play a role; they are an under-recognized form of end-user that has to be taken into account. Here's some new work by Beisland, D'Espallier and Mersland on "personal mission drift" among credit officers of Ecuadorian MFIs. Now don't look away because this is about microcredit or Ecuador--it's directly applicable to any kind of financial service offered to any kind of customer anywhere. Beisland et al. find that as credit officers gain experience they tend to serve fewer "vulnerable" clients (e.g. smaller loans, young borrowers, disabled borrowers). Why? Because it takes too much time--there are those frictions again. Figuring out how to offer quality products, especially credit, with appropriate frictions for both the borrowers and the credit officer, is a conundrum everywhere.
For further evidence of this, check out the similarities between this piece from Bindu Ananth about conversations with newly banked customers in Indian cities, and this report on "Generational Money Chatter" in the US from Hope Schau and Ignacio Luri (especially from GenXers and Millennials). The common theme I perceive: lots of questions and uncertainties about products and providers, little faith in the "systems," and confusion about where to turn for trustworthy advice.
2. Frictions, Temptation and Digital Finance: Those of you working in the digital finance world may already be thinking about how digital tools can lower frictions--after all, not only can FinTech tools more quickly and easily gather data from consumers, but they often cut the front-line staff right out of the equation! Take that, friction!
Oh but friction can be useful. This is one of those areas where I'm constantly baffled at the disconnect between the developed and developing worlds. In the developed world, it's generally understood that the goal of payment and digital finance innovation is usually to remove friction specifically for the purpose of getting people to spend more money, more often. Amazon didn't develop and patent one-click ordering out of concern for saving people time (Interesting side note, Amazon's patent on one-click expired last month--exogenous variation klaxon!). The sales pitch that credit card issuers make to merchants has always been that credit cards induce people to spend more.
Here's one of my favorite new pieces of research in a long time: a study of how people in debt management plans handled spending temptation (if that description is too dry to get you to click, try this one: "Target is the Devil!"). The sub-text, and sometimes text, is how hard retailers and some credit providers work to break down the frictions that prevent people from spending.
What's the connection to digital finance, particularly in developing countries. I'll enter there through this piece from Graham Wright based on a debate at the recent MasterCard Foundation Symposium on Financial Inclusion. Graham was asked to make the argument against the hope for digital finance serving poor customers. His list of five reasons why digital finance is "largely irrelevant" in the typical rural village is worth reading at face value. But it's also worth thinking about in terms of how much of digital finance is aimed at removing frictions, how it's failed to remove some of those frictions for poorer customers and what can (or will) happen to poor households when appropriate frictions are removed.
Read More1. The Future of Microfinance: A few weeks ago I linked to a curious piece about the future of Indian microfinance, that seemed to be praising the swallowing of MFIs by traditional banks and justifying the extinction of MFIs that tried to go it alone. Dan Rozas wrote to point out some of the subtext: The pattern in India is similar to other countries, with the largest MFIs turning into banks. And while there are mergers and acquisitions, it is still largely the same organizations serving poor customers, "only now they're called banks." This week Barbara Magnoni tweeted from the Foromic conference that "microfinance is stale" so I asked her what she thought was next. Her response: "[B]ig MFIs win, digitize processes, poor too expensive to reach. Poor go back to cash/informal markets/ and consumer loans.YAY?:("
Between the two comments, I feel like the future of microfinance is already here, right here in the USA! Per Dan's note, the transition in India and elsewhere sounds a lot like the history of banking in the United States, right up through credit unions. And per Barbara's note, the next step is pulling back from poorer customers because they are more expensive to serve. So you end up with a system where even an institutional form whose original reason for being was to serve the excluded and put "clients at the center" (to borrow a phrase) has, aside from exceptional organizations, left the poorest behind. The global microfinance movement, I think, needs to spend a lot more time looking at the financial services landscape in the United States, because that is where, absent some major investment, are headed: nearly ubiquitous financial services, but very little quality available to lower-income customers, with plenty of predatory or just indifferent-to-the-effects-on-poor-customers actors ready to fill the gaps. I guess you could say that's the negative way of making the "Case for Social Investment in Microcredit".
To keep things from going too dark right off the bat, here's the story of how BRAC's MFIs in Liberia and Sierra Leone managed the Ebola crisis and it's aftermath (blog summary). And a shout-out to the Global Delivery Initiative for writing up stories like this in sufficient detail to be operationally useful.
OK, that's enough optimism for me. You might be skeptical of my take on where microfinance is headed. So let me present this piece from Matthew Soursourian over at CGAP on what can happen when we push consumers toward digital merchant payments, drawing parallels to the US experience.
2. The Future of Digital Finance (and of us all): That last piece could just as easily have fit here, so to encourage you to read it, let me just say again: the future of digital finance is already here, and contrary to popular opinion, it's in operation in the United States.
Still not buying it? Here's another CGAP piece drawing on the US experience: "How Developing Countries Can Prevent Their Own Equifax Breach." The encouraging thing is that this possibility is being considered; the discouraging thing is that David Medine is probably wrong: developing countries can't prevent their own breaches. At least there is no evidence so far that institutions are learning from examples like this, given how pedestrian the causes of such breaches are.
At a more macro level, Tyler Cowen and Matt Levine (a dreamteam if there ever was one) discuss where technology is taking finance. Pay special attention to the section in the middle where they discuss whether technology ultimately increases or decreases access to credit. While they don't mention it specifically, this is the big question mark about Lenddo and EFL, and their like, that I mentioned: while the algorithms will rescue lots of people who are good credit risks but can't prove it conventionally, they will also likely simultaneously lock bad credit risks out of the system permanently.
Speaking of being locked out, here are Charles Kenny and Cordelia Kenny reporting on a forum at CGD on women being locked out of FinTech companies and how that affects what services are being developed and who is served.
Read More1. The Search for Truth, Part II: Last week's opening theme was about how hard social science is. I often find there's an unspoken wistfulness in social science research for the clear questions and clear answers of the "hard sciences."
But cheer up! It's just as bad on the other side of the fence. When you're frustrated that there doesn't seem to be a biological mechanism that explains the long-term positive outcomes of deworming, remember that we have no idea--literally, no idea--what causes "side stitch," that shooting pain we've all had in our abdomen during exercise. And when you're down in the dumps that so many development interventions don't seem to show much effect, remember that the universe shouldn't exist, and we don't know why it didn't explode nanoseconds after coming into being.
On the other hand, Ioannidis, Stanley and Doucouliagos' paper on how vastly underpowered most economics papers are has finally been published (it's been circulating for awhile). If that's not enough to send you back into despair, the fact that economists need to be reminded of basic good practice in presenting their ideas--per this slide deck from Rachael Meager--might do the trick. Don't get me wrong, it's good advice. But I was reminded of the time I attended a conference for PR "professionals" where the advice included such gems as, "Make sure the reporter you're pitching actually covers the topic" and "Read the last few articles the reporter wrote." Last year I was joking with Jessica Goldberg about starting a side-business editing the introductions and slide decks of job market papers. Perhaps I shouldn't have been joking.
2. The Mess that is US Higher Education (or Labor Markets are Broken All Over): Studying labor market inefficiencies is a common topic in development economics (yes, this is clickbait for David McKenzie). But as in so many domains, the problems we study in developing economies also exist in developed ones, just wearing a Halloween mask. Here's a new study on "credentialism" in the US labor market, the demand for college degrees for jobs that have no reason to require a college degree (as demonstrated by the fact that the vast majority of people currently in those jobs don't have one). That's bad for employers who pay some of the cost of the self-imposed mismatch in the labor market, but it's much, much worse for potential employees who are shut out of well-paying, stable jobs for no good reason. Unless, of course, they spend large amounts of money to get a credential. The large, and growing, lifetime earnings gap between those with a credential and those without has justified the incredible growth in student debt to finance these credentials. But if the credential is just an artifact of herd behavior among employers...
And why are those credentials so expensive? One reason is that the universities providing those credentials are spending, and borrowing, huge amounts themselves in order to attract the students who have to get the credential to apply for a job. So the students borrow, and borrow some more. And then they get shut out of programs for loan forgiveness that they are should be eligible for, because the system is a mess. But don't worry, if their debt gets too out of hand, they can discharge those loans in bankruptcy. Oh wait, we changed the bankruptcy law so they can't ever discharge those loans. Don't forget too that large numbers of the people we've pushed into needing a credential are entering universities, taking loans, but never getting the credential (e.g. 70% of single mothers who enroll).
And the advanced degree market may be worse. A few weeks ago I featured some work on English football academies juxtaposed with a paper about the Clark Medal. Perhaps my comparison was too oblique--so here's a piece from Nature making the connection explicit. The chances that a Ph.D. student will land a permanent academic job in the US or UK is well under 10%. The reason it's plausible to offer job market paper editorial consulting is that the premium for a well-written paper is so large. And it's large because there is massive over-supply.
For those newly minted Ph.D.'s taking adjunct teaching jobs just so they can stay marginally attached to academia and perhaps make enough to supplement their food stamps, I have bad news. Current students (bachelor's and master's students that is) teach just as well as adjuncts, suggesting that "student instructors can serve as an effective tool for universities to reduce their costs." Oh right, I was trying to avoid a novella.
Read More1. The Search for Truth: The New York Times Magazine has a long piece about Amy Cuddy, the social psychologist of "power posing" fame, and the messy process by which her research has been popularized and then discredited. The piece suggests that Cuddy (though it by no means holds her out as blameless) has been uniquely and personally targeted as the face of unreplicable and bad social science in an era of changing research practices and expectations, perhaps because she is a woman. More broadly it ponders whether the process and social conventions of communication around challenging social science research may do more harm than good. It points specifically to Uri Simonsohn, Joseph Simmons and Andrew Gelman and their roles in both calling out bad social science and in specifically highlighting Cuddy's power posing paper as an example.
It's well worth the long read, careful consideration but also some critical evaluation. The piece comes at a very interesting time, with the Weinstein saga, #MeToo, and more specifically the push back about Econ Job Market Rumors and bad behavior in economics. It's important to read the piece in the context of such things as EJMR and this anecdote from Rohini Pande (in an interview with David McKenzie this week) relating how a "senior male World Bank economist wrote to our senior male colleagues at MIT and Yale asking that they review our work and correct our mistakes" in one of her early papers (with Esther Duflo; see question 4 in the link, but read the whole thing, it's very good on a lot of topics).
But on reflection, I don't think the idea that Cuddy was uniquely targeted or treated more harshly than others holds water. It only appears so to a New York Times reporter because Cuddy's works is the kind that gets broad attention. Remember when Ben Goldacre kicked off "Worm Wars" with an amazingly condescending piece asking people not to point and laugh at Miguel and Kremer for the supposed "errors" in their Worms paper because they shared their data? Or the language and dudgeon around Reinhart and Rogoff's Excel error? Or the intemperate words flowing around the failure to replicate John Bargh's priming work? From another field, here's some pointed language challenging a recent result on gene editing alleging some pretty basic errors.
Of course, the commonality of bad behavior in academic circles doesn't excuse it. But that cuts both ways. Cuddy has also been using this faulty logic in her own defense. As far as I can tell, her main defense has always been "everyone was engaging in bad research practices, so it's not my fault", and that's definitely the implication that the NYT article gives. I don't see much distance between that and people excusing sexual harassment because they were "raised in the '60s and '70s."
Could the practice of social science be better? There's no question, but it's also not clear exactly how, other than the obvious avoidance of misogyny, ad hominem and personal attacks. But that line is difficult to see sometimes because the nature of social science research requires a great deal of personal investment. It's hard not to feel attacked when one's research, quite literally one's life's work, is criticized.
To me, the most thought-provoking part of the NYT piece is when Simmons, reviewing an email he sent to Cuddy about follow-up work on whether the power posing research was reliable, says "that email was too polite" given how serious he thought the problems were. And there is a lot of bad science that needs to be called out. This week, there's yet another update to the Brian Wansink saga--several papers flat out misrepresent who the study participants were (e.g. a paper claiming participants were 8-11 when they were 4-5). Not calling bad science out, I think, is a real contributor to real world problems, like Chief Justice John Roberts being able to call good political science research "sociological gobbledygook."
Here's a Chris Blattman thread on his reactions. Here's Andrew Gelman's response to the NYT piece and for the sake of this topic it is one of the few posts anywhere on the internet where you should read the comments. Someone in one of the Twitter threads wondered about the responsibility of Gelman and other bloggers like Tyler Cowen to police their comments. I'm sympathetic to this idea, but I'm old enough to remember policing comments on my own blog. It's an incredibly time-consuming and soul sucking affair with lots of trade-offs. The "business model" of blogging just doesn't allow it. In fact, in some ways it was the business model required to police commentary, also known as paid journalism, that led to blogging: the gatekeepers of commentary shut out too many voices who should be heard. Science, and the pursuit of truth, is hard.
Read More1. Evidence-Based Policy: Yesterday I was at a workshop hosted at Yale SOM and funded by the Hewlett Foundation on how to better connect evidence to policy. The workshop was part of a bigger project and a series of reports are coming that I will share when they are available. There was a lot of good discussion, but I thought I would share two thoughts that I find to be missing appropriate weight in evidence-based policy discussions. First, there is often discussion of a mismatch in the time horizons of researchers, implementers and policy makers. While this is no doubt true, the mismatch between those groups is trivial in comparison to the mismatch all those groups have with the amount of time it takes for change that people can feel to occur. Deworming's important effects--on earnings, not school attendance--are only felt decades after treatment. Moving to Opportunity similarly has a decade-scale effect. Few if any of the researchers, implementers or policymakers are still going to be around when the world really is undeniably different because of them.
Which brings me to the second point. The enterprise of evidence-based policy is grounded in marginal improvements across large groups of people--and that's a good thing! I'm a big believer in the value of marginal improvements (QED). But people have a really, really hard time noticing or caring about marginal improvements. Human beings prefer stories about big changes for a few people with unclear causality a lot more than they do about marginal gains with sound causal inference. I'm more and more convinced (because of evidence!) that hope is a key ingredient for even marginal impact, but hope comes from Queen of Katwe, not from 1/10 a standard deviation improvement in average test scores. So the unanswered question for me in this conversation is, "How do we manage the tension between the policies that are good for people and the policies that people want?"
In other evidence-based policy news, here's a rumination on the difficulty of applying research to practice in democratization (specifically Myanmar). And here's Andrew Gelman on not waiting for peer review, particularly in Economics, to start putting evidence into practice.
2. Evidence-Based Operations: OK, so there's one more thought: the gap between policy and research, and operations. But rather than a long discussion on that topic, here's a very good new piece on the operational choices of front-line social workers and the gap between policy (whether evidence-based or not) and practice. The challenge in the spotlight is not the Marxist-style view of workers dissociated from their work by rules but workers dissociated because of having too many morally-fraught choices. More light-heartedly, here's a piece that illustrates how hard it is to go from evidence to operational choices, as reflected through the failure of the US men's soccer team (I told you it would return). There is growing attention to front-line staff and the "product" as actually experienced by the beneficiary in impact evaluations, but much more is needed as far as I'm concerned.
3. Our Algorithmic Overlords: Speaking of operations, one of the areas where more attention is needed is the way that operations are being instantiated into algorithms that are opaque or entirely invisible. Ruben Mancha and Haslina Ali argue that that the unexamined algorithm is not worth using. Of course, they are arguing from ethics, not from business profits, where it's abundantly clear that unexamined algorithms are worth using.
Here's a piece about technology-related predictions from Gartner, a tech industry research and advisory company. Skip the first three to see some striking predictions about AI-generated false information, such as that people in "mature economies will consume more false information than true information." There's a threat to advancing evidence-based policy that definitely wasn't on the agenda yesterday. I started my career at Gartner way back in 1995 and I remember one of the first things we were given to read was an an article in Scientific American about the coming age of fake photography and video. Apparently that future has finally arrived.
Read More
