1. Evidence-Based Policy: Yesterday I was at a workshop hosted at Yale SOM and funded by the Hewlett Foundation on how to better connect evidence to policy. The workshop was part of a bigger project and a series of reports are coming that I will share when they are available. There was a lot of good discussion, but I thought I would share two thoughts that I find to be missing appropriate weight in evidence-based policy discussions. First, there is often discussion of a mismatch in the time horizons of researchers, implementers and policy makers. While this is no doubt true, the mismatch between those groups is trivial in comparison to the mismatch all those groups have with the amount of time it takes for change that people can feel to occur. Deworming's important effects--on earnings, not school attendance--are only felt decades after treatment. Moving to Opportunity similarly has a decade-scale effect. Few if any of the researchers, implementers or policymakers are still going to be around when the world really is undeniably different because of them. Which brings me to the second point. The enterprise of evidence-based policy is grounded in marginal improvements across large groups of people--and that's a good thing! I'm a big believer in the value of marginal improvements (QED). But people have a really, really hard time noticing or caring about marginal improvements. Human beings prefer stories about big changes for a few people with unclear causality a lot more than they do about marginal gains with sound causal inference. I'm more and more convinced (because of evidence!) that hope is a key ingredient for even marginal impact, but hope comes from Queen of Katwe, not from 1/10 a standard deviation improvement in average test scores. So the unanswered question for me in this conversation is, "How do we manage the tension between the policies that are good for people and the policies that people want?" In other evidence-based policy news, here's a rumination on the difficulty of applying research to practice in democratization (specifically Myanmar). And here's Andrew Gelman on not waiting for peer review, particularly in Economics, to start putting evidence into practice.
2. Evidence-Based Operations: OK, so there's one more thought: the gap between policy and research, and operations. But rather than a long discussion on that topic, here's a very good new piece on the operational choices of front-line social workers and the gap between policy (whether evidence-based or not) and practice. The challenge in the spotlight is not the Marxist-style view of workers dissociated from their work by rules but workers dissociated because of having too many morally-fraught choices. More light-heartedly, here's a piece that illustrates how hard it is to go from evidence to operational choices, as reflected through the failure of the US men's soccer team (I told you it would return). There is growing attention to front-line staff and the "product" as actually experienced by the beneficiary in impact evaluations, but much more is needed as far as I'm concerned.
3. Our Algorithmic Overlords: Speaking of operations, one of the areas where more attention is needed is the way that operations are being instantiated into algorithms that are opaque or entirely invisible. Ruben Mancha and Haslina Ali argue that that the unexamined algorithm is not worth using. Of course, they are arguing from ethics, not from business profits, where it's abundantly clear that unexamined algorithms are worth using. Here's a piece about technology-related predictions from Gartner, a tech industry research and advisory company. Skip the first three to see some striking predictions about AI-generated false information, such as that people in "mature economies will consume more false information than true information." There's a threat to advancing evidence-based policy that definitely wasn't on the agenda yesterday. I started my career at Gartner way back in 1995 and I remember one of the first things we were given to read was an an article in Scientific American about the coming age of fake photography and video. Apparently that future has finally arrived.