Appendix B
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101
102 103 104 105 106 107 108 109 110 111 112 113 114 115
Committee Response
to Wilson’s Dissent
This response addresses Professor Wilson’s dissent from one aspect of the
committee report. It is important to stress at the outset that his dissent
focuses on one part of one chapter of the report. Except for the effects of
right-to-carry laws on homicide, the entire committee is in agreement on
the material in Chapter 6 and the report overall. In particular, the committee,
including Wilson, found that “it is impossible to draw strong conclusions
from the existing literature on the causal impact” of right-to-carry
laws on violent and property crime in general and rape, aggravated assault,
auto theft, burglary, and larceny in particular.
The only substantive issue on which the committee differed is whether
the existing research supports the conclusion that right-to-carry laws substantially
reduce murder. The report suggests that the scientific evidence is
inconclusive. Wilson disagreed, arguing that virtually every estimate shows
a substantial and statistically significant negative effect of right-to-carry
laws on murder.
While it is true that most of the reported estimates are negative, several
are positive and many are statistically insignificant. In addition, when we
use Lott’s trend model but restrict the out years to five years or less (Table
6-7), the trends for murder become positive and those for other crimes
remain negative. Therefore, the key question is how to reconcile the contrary
findings or, conversely, how to explain why these particular positive,
or negative, findings should be dismissed. Three sets of results discussed
more fully in Chapter 6 provide support for the committee’s conclusion:
Published studies, the committee’s analysis of control variables, and the
committee’s analysis extending the time period.
COMMITTEE RESPONSE TO WILSON’S DISSENT 273
1. Published studies. There is no question that the empirical results on
the effects of right-to-carry laws on murder (and other crimes) are sensitive
to seemingly small variations in data and specification. Indeed, Wilson
agrees that a few studies find positive effects of right-to-carry laws on
murder. We cite four studies in Tables 6-3 and 6-4: Ayres and Donohue,
Black and Nagin, Moody, and Plassmann and Tideman (cited in Chapter
6). There are almost certainly others not reported in these tables.
The rest of the committee and Wilson agree that fragility does not
prove that the results of any specific paper are incorrect. However, some
of the published results must be incorrect because they are inconsistent
with one another. The important question, therefore, is whether the correct
results can be identified. The rest of the committee thinks that they
cannot. Contrary to Wilson’s claim, the committee did assess the existing
body of empirical literature on right-to-carry laws (see the section beginning
on page 127 and Tables 6-3 and 6-4). As described in the report, all of
the empirical research on right-to-carry laws relies on the same conceptual
and methodological ideas (page 121). Relative to the basic models estimated
by Lott, some researchers used data from more counties and some
from fewer; some used hybrid linear models while others used nonlinear
specifications; some provide state-specific estimates while most provide a
single national estimate; some added control variables while others used
relatively parsimonious specifications; and so forth. All of the studies
described in the literature review made plausible cases for their choices of
models and data. Wilson seems to argue that a careful evaluation of the
literature would reveal which paper or papers obtained correct results, but
he does not suggest the evaluation criteria. The rest of the committee does
not think that application of any scientific criteria to existing papers would
identify the effects of right-to-carry laws on crime.
2. Committee control variable analysis. Chapter 6 shows that when
the trend and dummy variable models do not include demographic and
socioeconomic covariates (but do include year and county dummy variables)
the estimates are relatively small, positive in one case (Table 6-6,
Row 3), and statistically insignificant in all cases. Contrary to Wilson’s
assertion, the chapter does not claim that this or any other specification is
correct. Rather, this finding simply reveals that “detecting the effect, if any,
of right-to-carry laws requires controlling for appropriate confounding variables.”
In light of the fragility revealed in the literature, the fundamental
issue is which set of covariates is sufficient to identify the effects of right-tocarry
laws on homicide and other crimes. The importance of controlling for
the correct set of covariates is well known. In fact, much of the debate
between Lott and his statistically oriented critics focuses on determining the
correct set of control variables. Everyone (including Wilson and the rest of
the committee) agrees that control variables matter, but there is disagree274
ment on the correct set. Thus, the facts that there is no way to statistically
test for the correct specification and that researchers using reasonable specifications
find different answers are highly relevant. Given the existing data
and methods, the rest of the committee sees little hope of resolving this
fundamental statistical problem.
Furthermore, the example of the relationship between crime rates and
policing in the dissent raises another problem. The usual way one proceeds
in research is to estimate the relationship between two variables and if a
significant relationship is found controls are introduced to test the relationship.
As the dissent notes, these controls are selected based on reasonable
theories and research. In this case, the bivariate relationship (between right
to carry laws and crime) is small, positive in one case, and insignificant in
all. This is not like the hypothesized conflicting bivariate findings in Wilson’s
police example. Thus the selection of controls in the analysis of right-tocarry
laws is as difficult as the committee contends
3. Committee trend model analysis. Wilson states that the trend model
analysis in Table 6-7 estimates the effects of right-to-carry laws on a yearly
basis, rather than a single trend.1 This is incorrect. The estimates reported
in Table 6-7 are found using Lott’s trend model with restrictions on the
number of postadoption years used in the analysis. If the model is correctly
specified, this restriction should be inconsequential. However, we find substantial
differences, especially for murder. In fact, when we restrict the
number of postadoption years to five or fewer, the estimates switch from
negative to positive. Thus, Model 6.2 appears to be misspecified. Moreover,
despite Wilson’s assertion, these types of sensitivity test are commonly
used in peer-reviewed journals and are suggested by Rosenbaum (2001) as
a way to assess the robustness of an empirical model. Of course, results like
those reported in Chapter 6 might often lead a paper to be rejected from a
peer-reviewed journal.
Wilson further suggests that Lott’s findings may depend on the crime
rate trends that changed dramatically over the course of the 1990s. All of
the studies in this literature, however, attempt to control for trends in
crime, and thus purport to reveal a time invariant effect of right-to-carry
laws. If the effects vary by time, all of the existing models are misspecified.
In sum, we are encouraged that Professor Wilson agrees with the rest of
the committee except for the specific conclusion regarding the effects of
right-to-carry laws on murder. On this point, we find his arguments to be
unconvincing and his summary of some parts of the chapter inaccurate. In
our view the evidence on homicide is not noticeably different from that on
other crimes evaluated in this literature and cannot be easily separated. If
1Contrary to Wilson’s claim, the results in Table 6-7 all rely on models with covariates.
COMMITTEE RESPONSE TO WILSON’S DISSENT 275
the effects of right-to-carry laws on violent and property crimes are ambiguous,
as argued in Chapter 6, we see no reason why the same is not true
of homicide. Professor Wilson may be correct on this matter—it is theoretically
possible—but we maintain that the scientific evidence does not support
his position.
REFERENCE
Rosenbaum, P.R.
2001 Replicating effects and biases. American Statistician 55(3):223-227.
276