The causal effects of neighborhood quality (aka Heckman's favorite paper of 2018)

There is extensive sociological research by William Julius Wilson and others that strives to show that neighborhood quality matters for individual outcomes. The Moving to Opportunity (MTO) voucher program, started in 1994, aims to examine this hypothesis by randomly giving financial incentives to families who live in high-poverty neighborhoods to relocate to neighborhoods with less poverty. There is a robust literature on MTO that shows that so-called “voucher effects” - the impact on income/unemployment/welfare for those awarded an MTO voucher - are not significant.

So who is right? Do the sociologists have a nice theory that is not borne out by the data? Have the economists failed to capture the true effect of moving to a new neighborhood? Or is the housing voucher program not designed well enough to target those people who might actually benefit from it?

The Problem:

There are three groups in the MTO program: control, section 8, and experimental. Those receiving an experimental voucher are given a rent-subsidizing voucher to move from a high-poverty neighborhood to a low-poverty neighborhood. Those receiving a section 8 voucher can similarly move to either a low-poverty or middle-poverty neighborhood. The fundamental problem in evaluating the causal effect of moving from a high-poverty neighborhood to a middle- or low-poverty neighborhood is noncompliance.

Nearly 50% of families receiving a voucher did not use it to relocate. In addition, 21% of control families relocated to lower poverty neighborhoods despite not receiving a voucher. Across a broad range of covariates, those who use the voucher to relocate differ significantly from those who do not. This means that, while assignment to any of the three groups is random, there is a selection effect within any group. Due to this selection problem, prior research on has focused only on the causal effect of receiving a voucher for a given family (an intent-to-treat analysis). It should not be surprising, however, that this literature has found that voucher effects are insignificant: any estimates of voucher effects will include the null effects of those 50% who receive vouchers but do not relocate.

The Solution:

In Noncompliance as rational choice (2018), Rodrigo Pinto uses a brilliant identification strategy to show how noncompliance, which heretofore has been seen as a thorn in the side of researchers estimating causal treatment effects, can be used to nonparametrically identify the effect of treatment on the treated. Here’s how the identification strategy works.

Each household faces a multinomial choice problem. For any treatment group in which they may be placed (control, section 8, or experimental), the household can choose to stay in a high-poverty neighborhood, move to a middle-poverty neighborhood, or move to a low-poverty neighborhood. Because of the existence of a substantial degree of noncompliance, we can redefine this problem as identifying the unique types in the data.

Each type has a unique profile of counterfactual choices they would make depending on the treatment they  received. One type, for example, would choose to stay in the high-poverty neighborhood if they were in the control group, move to a medium-poverty neighborhood if they were in the section 8 group, and move to a low-poverty neighborhood if they were in the experimental group. Another type would stay in the high-poverty neighborhood regardless of the treatment group they are placed in. This is an extension to a 3 X 3 setting of Angrist, Imbens, and Rubin’s (1996) way of dividing the data into Always-Takers, Never-Takers, Compliers, and Defiers. Here, there are 27 (3^3) unique types.

The treatment effect on these 27 types is not identified because we do not observe the counterfactual outcomes. Only 9 total choices are observed in the data, three for each treatment group. One of the key innovations in this paper is to use a revealed preference approach to reduce the number of possible types from 27 to just 7 economically justifiable response types. How does the revealed preference approach work?

Consider an agent who is in the control group. If she chooses to move to a medium-poverty neighborhood, then revealed preference rules out any type where she gets a section 8 housing voucher and stays in a high-poverty neighborhood. This is because the section 8 voucher would augment her budget set if she moved to the middle-poverty neighborhood relative to when she is in the control group, and if she has preferences that are strictly monotonic in at least one good (an innocuous assumption), she would strictly prefer this augmented budget set. Therefore, three types are eliminated: (medium, high, high), (medium, high, low), and (medium, high, medium).*

We cannot, however, assume that she will move to a low-poverty neighborhood if she receives the experimental voucher. Remember that the experimental voucher subsidizes movement only to low-poverty neighborhoods. Since we only observe her choice when she is in the control group, we do not know which of the following relations capture her preferences: medium > high > low or low > medium > high.

The revealed preference approach reduces the 27 potential types to the following seven types, which are presented in the table below, sourced from the original paper:

The author shows that the elimination of 20 types embeds 9 monotonicity conditions whose validity he then tests using propensity scores. The estimated propensity scores satisfy each one of the 9 monotonicity conditions. As there are 336 total feasible combinations for the propensity score inequalities, the data strongly supports the use of the revealed preference approach.

Pinto is able to estimate the proportion of each type in the data, and, using some nifty matrix algebra, counterfactual outcomes for all seven economically justifiable types. He finds that two types are most common in the data, s1 and s4. He is also able to predict an individual’s type based on her covariates. For example, someone who has no teenagers, has moved in the past to seek schools, and indicates being a prospective mover, is more likely to be type s4 than the average recipient of the vouchers. Voucher recipients who do not relocate are more likely to have disabled household members and to have lived in their current neighborhood a long time. This suggests that we can rewrite the eligibility criteria for the MTO program to target those people who are mostly likely to make use of the voucher if they receive it.

At the beginning of this post, I asked whether the sociologists or the economists were right in their assessment of neighborhood effects. Pinto finds that moving from a high-poverty neighborhood to a low-poverty neighborhood yields a 14% increase in income, a 20% decrease in the likelihood of being unemployed, and a 38% increase in the chance of breaking out of poverty. All of these effects are statistically significant. This paper provides convincing evidence in support of the sociological view that neighborhood characteristics are important in determining individual outcomes and suggests that the prior emphasis on an intent-to-treat analysis does not do a good job of capturing these neighborhood effects.

The paper is available here.

*The first element of the ordered triple is the choice of the agent if placed in the control group, the second is the choice if placed in the section 8 group, and the third is the choice if placed in the experimental group.