Are the Results Sensitive to Controls?
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101
102 103 104 105 106 107 108 109 110 111 112 113 114 115
The final two rows of Table 6-5 present two sets of results obtained by
the committee when estimating models identical to those of Model 6.1, but
excluding socioeconomic and demographic controls. We include only the
FIGURE 6-2 Year-by-year estimates of the percentage change in aggregate crime
(normalized to adoption date of right-to-carry law, year 0).
Estimate, o bottom of 95% confidence interval (CI), | Top of 95% CI
-50
-40
-30
-20
-10
0
10
20
-10 -5 0 5 10
Years Relative to Law Passage
Percentage change
Murder
-40
-30
-20
-10
0
10
-10 -5 0 5 10
Years Relative to Law Passage
Percentage Change
Rape
right-to-carry variable, year dummies, and county fixed effects. These estimates
tell us how crime has changed in states that have adopted the rightto-
carry laws before and after the law change, relative to national time
patterns in crime. It is important to stress that the committee is not arguing
that excluding all socioeconomic and demographic covariates is an appropriate
method of identifying the effects of right-to-carry laws. Rather, we
are simply assessing whether such laws are associated with a decline in the
level of crime. If not, then detecting the effect, if any, of right-to-carry laws
FIGURE 6-3 Year-by-year estimates of the percentage change in disaggregate violent
crimes (normalized to adoption date of right-to-carry law, year 0).
Estimate, o bottom of 95% confidence interval (CI), | Top of 95% CI
-30
-20
-10
0
10
-10 -5 0 5 10
Years Relative to Law Passage
Percentage Change
Aggravated Assault
-50
-40
-30
-20
-10
0
10
20
-10 -5 0 5 10
Years Relative to Law Passage
Percentage Change
Robbery
requires controlling for appropriate confounding variables and thereby reliance
on a model such as those used by Lott and others.
The results without controls are quite different. Using the earlier sample
period and the new data, one finds three negative coefficients, only one of
them statistically significant. When the sample is extended to 2000, only
one of nine coefficients is negative, and it is insignificantly different from
zero. For example, the violent crime coefficient with controls is 4.1 percent,
while it is 12.9 percent without controls. These results show that states that
-30
-20
-10
0
10
20
-10 -5 0 5 10
Years Relative to Law Passage
Percentage Change
Auto Theft
Burglary
-40
-30
-20
-10
0
10
-10 -5 0 5 10
Years Relative to Law Passage
Peercentage Change
Larceny
-30
-20
-10
0
10
20
30
-10 -5 0 5 10
Years Relative to Law Passage
Percentage Crime FIGURE 6-4
Year-by-year estimates of the percentage change in disaggregate property
crimes (normalized to adoption date of right-to-carry law, year 0).
Estimate, o bottom of 95% confidence interval (CI), | Top of 95% CI
passed right-to-carry laws did not on average experience statistically significant
crime declines relative to states that did not pass such laws.
There are two points to make about the no-controls results. First, the
no-controls results provide a characterization of the data that shows that, if
there is any effect, it is not obvious in the dummy variable model. What do
estimates from that model mean? The model says that crime rates differ
across counties and, moreover, that they change from one year to the next
in the same proportionate way across all counties in the United States. Over
and above this variation, there is a one-time change in the mean level of
crime as states adopt right-to-carry laws. So these estimates indicate that,
for the period 1977-1992, states adopting right-to-carry laws saw roughly
no change in their violent crime rates and 8.5 percent increases in their
property crime rates, relative to national time patterns. Estimating the model
using data to 2000 shows that states adopting right-to-carry laws saw 12.9
percent increases in violent crime—and 21.2 percent increases in property
crime—relative to national time patterns. The first-blush evidence provided
by these no-controls models is thus not supportive of the theory that rightto-
carry laws reduce crime.
A final lesson to draw from the no-controls dummy variable results is
that the results are sensitive to the inclusion of controls. That is, whether
one concludes that right-to-carry laws increase or decrease crime based on
models of this sort depends on which control variables are included. Such
laws have no obvious effect in the model without controls (and therefore no
clear level effect in the raw data). Moreover, as demonstrated above, seemingly
minor changes to the set of control variables substantially alter the
estimated effects. Given that researchers might reasonably argue about
which controls belong in the model and that the results are sensitive to the
set of covariates, the committee is not sanguine about the prospects for
measuring the effect of right-to-carry laws on crime. Note that this is distinct
from whether such laws affect crime. Rather, in our view, any effect
they have on crime is not likely to be detected in a convincing and robust
fashion.
Estimates from the trend model are less sensitive to the inclusion of
controls. While the no-control point estimates displayed in the third and
fourth rows of Table 6-6 are smaller than in the model with controls, most
of these estimates are negative and statistically significant. The trend model
without controls shows reductions in violent and property crime trends
following the passage of right-to-carry laws for both sample endpoints. For
murder, however, the results are positive when using the 2000 endpoint,
negative when using the 1992 endpoint, and statistically insignificant in
both cases.