Are the Results Sensitive to Controls?

К оглавлению
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 
102 103 104 105 106 107 108 109 110 111 112 113 114 115 

The final two rows of Table 6-5 present two sets of results obtained by

the committee when estimating models identical to those of Model 6.1, but

excluding socioeconomic and demographic controls. We include only the

FIGURE 6-2 Year-by-year estimates of the percentage change in aggregate crime

(normalized to adoption date of right-to-carry law, year 0).

Estimate, o bottom of 95% confidence interval (CI), | Top of 95% CI

-50

-40

-30

-20

-10

0

10

20

-10 -5 0 5 10

Years Relative to Law Passage

Percentage change

Murder

-40

-30

-20

-10

0

10

-10 -5 0 5 10

Years Relative to Law Passage

Percentage Change

Rape

right-to-carry variable, year dummies, and county fixed effects. These estimates

tell us how crime has changed in states that have adopted the rightto-

carry laws before and after the law change, relative to national time

patterns in crime. It is important to stress that the committee is not arguing

that excluding all socioeconomic and demographic covariates is an appropriate

method of identifying the effects of right-to-carry laws. Rather, we

are simply assessing whether such laws are associated with a decline in the

level of crime. If not, then detecting the effect, if any, of right-to-carry laws

FIGURE 6-3 Year-by-year estimates of the percentage change in disaggregate violent

crimes (normalized to adoption date of right-to-carry law, year 0).

Estimate, o bottom of 95% confidence interval (CI), | Top of 95% CI

-30

-20

-10

0

10

-10 -5 0 5 10

Years Relative to Law Passage

Percentage Change

Aggravated Assault

-50

-40

-30

-20

-10

0

10

20

-10 -5 0 5 10

Years Relative to Law Passage

Percentage Change

Robbery

requires controlling for appropriate confounding variables and thereby reliance

on a model such as those used by Lott and others.

The results without controls are quite different. Using the earlier sample

period and the new data, one finds three negative coefficients, only one of

them statistically significant. When the sample is extended to 2000, only

one of nine coefficients is negative, and it is insignificantly different from

zero. For example, the violent crime coefficient with controls is 4.1 percent,

while it is 12.9 percent without controls. These results show that states that

-30

-20

-10

0

10

20

-10 -5 0 5 10

Years Relative to Law Passage

Percentage Change

Auto Theft

Burglary

-40

-30

-20

-10

0

10

-10 -5 0 5 10

Years Relative to Law Passage

Peercentage Change

Larceny

-30

-20

-10

0

10

20

30

-10 -5 0 5 10

Years Relative to Law Passage

Percentage Crime FIGURE 6-4

Year-by-year estimates of the percentage change in disaggregate property

crimes (normalized to adoption date of right-to-carry law, year 0).

Estimate, o bottom of 95% confidence interval (CI), | Top of 95% CI

passed right-to-carry laws did not on average experience statistically significant

crime declines relative to states that did not pass such laws.

There are two points to make about the no-controls results. First, the

no-controls results provide a characterization of the data that shows that, if

there is any effect, it is not obvious in the dummy variable model. What do

estimates from that model mean? The model says that crime rates differ

across counties and, moreover, that they change from one year to the next

in the same proportionate way across all counties in the United States. Over

and above this variation, there is a one-time change in the mean level of

crime as states adopt right-to-carry laws. So these estimates indicate that,

for the period 1977-1992, states adopting right-to-carry laws saw roughly

no change in their violent crime rates and 8.5 percent increases in their

property crime rates, relative to national time patterns. Estimating the model

using data to 2000 shows that states adopting right-to-carry laws saw 12.9

percent increases in violent crime—and 21.2 percent increases in property

crime—relative to national time patterns. The first-blush evidence provided

by these no-controls models is thus not supportive of the theory that rightto-

carry laws reduce crime.

A final lesson to draw from the no-controls dummy variable results is

that the results are sensitive to the inclusion of controls. That is, whether

one concludes that right-to-carry laws increase or decrease crime based on

models of this sort depends on which control variables are included. Such

laws have no obvious effect in the model without controls (and therefore no

clear level effect in the raw data). Moreover, as demonstrated above, seemingly

minor changes to the set of control variables substantially alter the

estimated effects. Given that researchers might reasonably argue about

which controls belong in the model and that the results are sensitive to the

set of covariates, the committee is not sanguine about the prospects for

measuring the effect of right-to-carry laws on crime. Note that this is distinct

from whether such laws affect crime. Rather, in our view, any effect

they have on crime is not likely to be detected in a convincing and robust

fashion.

Estimates from the trend model are less sensitive to the inclusion of

controls. While the no-control point estimates displayed in the third and

fourth rows of Table 6-6 are smaller than in the model with controls, most

of these estimates are negative and statistically significant. The trend model

without controls shows reductions in violent and property crime trends

following the passage of right-to-carry laws for both sample endpoints. For

murder, however, the results are positive when using the 2000 endpoint,

negative when using the 1992 endpoint, and statistically insignificant in

both cases.