Appendix B

Committee Response

to Wilson’s Dissent

This response addresses Professor Wilson’s dissent from one aspect of the

committee report. It is important to stress at the outset that his dissent

focuses on one part of one chapter of the report. Except for the effects of

right-to-carry laws on homicide, the entire committee is in agreement on

the material in Chapter 6 and the report overall. In particular, the committee,

including Wilson, found that “it is impossible to draw strong conclusions

from the existing literature on the causal impact” of right-to-carry

laws on violent and property crime in general and rape, aggravated assault,

auto theft, burglary, and larceny in particular.

The only substantive issue on which the committee differed is whether

the existing research supports the conclusion that right-to-carry laws substantially

reduce murder. The report suggests that the scientific evidence is

inconclusive. Wilson disagreed, arguing that virtually every estimate shows

a substantial and statistically significant negative effect of right-to-carry

laws on murder.

While it is true that most of the reported estimates are negative, several

are positive and many are statistically insignificant. In addition, when we

use Lott’s trend model but restrict the out years to five years or less (Table

6-7), the trends for murder become positive and those for other crimes

remain negative. Therefore, the key question is how to reconcile the contrary

findings or, conversely, how to explain why these particular positive,

or negative, findings should be dismissed. Three sets of results discussed

more fully in Chapter 6 provide support for the committee’s conclusion:

Published studies, the committee’s analysis of control variables, and the

committee’s analysis extending the time period.

COMMITTEE RESPONSE TO WILSON’S DISSENT 273

1. Published studies. There is no question that the empirical results on

the effects of right-to-carry laws on murder (and other crimes) are sensitive

to seemingly small variations in data and specification. Indeed, Wilson

agrees that a few studies find positive effects of right-to-carry laws on

murder. We cite four studies in Tables 6-3 and 6-4: Ayres and Donohue,

Black and Nagin, Moody, and Plassmann and Tideman (cited in Chapter

6). There are almost certainly others not reported in these tables.

The rest of the committee and Wilson agree that fragility does not

prove that the results of any specific paper are incorrect. However, some

of the published results must be incorrect because they are inconsistent

with one another. The important question, therefore, is whether the correct

results can be identified. The rest of the committee thinks that they

cannot. Contrary to Wilson’s claim, the committee did assess the existing

body of empirical literature on right-to-carry laws (see the section beginning

on page 127 and Tables 6-3 and 6-4). As described in the report, all of

the empirical research on right-to-carry laws relies on the same conceptual

and methodological ideas (page 121). Relative to the basic models estimated

by Lott, some researchers used data from more counties and some

from fewer; some used hybrid linear models while others used nonlinear

specifications; some provide state-specific estimates while most provide a

single national estimate; some added control variables while others used

relatively parsimonious specifications; and so forth. All of the studies

described in the literature review made plausible cases for their choices of

models and data. Wilson seems to argue that a careful evaluation of the

literature would reveal which paper or papers obtained correct results, but

he does not suggest the evaluation criteria. The rest of the committee does

not think that application of any scientific criteria to existing papers would

identify the effects of right-to-carry laws on crime.

2. Committee control variable analysis. Chapter 6 shows that when

the trend and dummy variable models do not include demographic and

socioeconomic covariates (but do include year and county dummy variables)

the estimates are relatively small, positive in one case (Table 6-6,

Row 3), and statistically insignificant in all cases. Contrary to Wilson’s

assertion, the chapter does not claim that this or any other specification is

correct. Rather, this finding simply reveals that “detecting the effect, if any,

of right-to-carry laws requires controlling for appropriate confounding variables.”

In light of the fragility revealed in the literature, the fundamental

issue is which set of covariates is sufficient to identify the effects of right-tocarry

laws on homicide and other crimes. The importance of controlling for

the correct set of covariates is well known. In fact, much of the debate

between Lott and his statistically oriented critics focuses on determining the

correct set of control variables. Everyone (including Wilson and the rest of

the committee) agrees that control variables matter, but there is disagree274

ment on the correct set. Thus, the facts that there is no way to statistically

test for the correct specification and that researchers using reasonable specifications

find different answers are highly relevant. Given the existing data

and methods, the rest of the committee sees little hope of resolving this

fundamental statistical problem.

Furthermore, the example of the relationship between crime rates and

policing in the dissent raises another problem. The usual way one proceeds

in research is to estimate the relationship between two variables and if a

significant relationship is found controls are introduced to test the relationship.

As the dissent notes, these controls are selected based on reasonable

theories and research. In this case, the bivariate relationship (between right

to carry laws and crime) is small, positive in one case, and insignificant in

all. This is not like the hypothesized conflicting bivariate findings in Wilson’s

police example. Thus the selection of controls in the analysis of right-tocarry

laws is as difficult as the committee contends

3. Committee trend model analysis. Wilson states that the trend model

analysis in Table 6-7 estimates the effects of right-to-carry laws on a yearly

basis, rather than a single trend.1 This is incorrect. The estimates reported

in Table 6-7 are found using Lott’s trend model with restrictions on the

number of postadoption years used in the analysis. If the model is correctly

specified, this restriction should be inconsequential. However, we find substantial

differences, especially for murder. In fact, when we restrict the

number of postadoption years to five or fewer, the estimates switch from

negative to positive. Thus, Model 6.2 appears to be misspecified. Moreover,

despite Wilson’s assertion, these types of sensitivity test are commonly

used in peer-reviewed journals and are suggested by Rosenbaum (2001) as

a way to assess the robustness of an empirical model. Of course, results like

those reported in Chapter 6 might often lead a paper to be rejected from a

peer-reviewed journal.

Wilson further suggests that Lott’s findings may depend on the crime

rate trends that changed dramatically over the course of the 1990s. All of

the studies in this literature, however, attempt to control for trends in

crime, and thus purport to reveal a time invariant effect of right-to-carry

laws. If the effects vary by time, all of the existing models are misspecified.

In sum, we are encouraged that Professor Wilson agrees with the rest of

the committee except for the specific conclusion regarding the effects of

right-to-carry laws on murder. On this point, we find his arguments to be

unconvincing and his summary of some parts of the chapter inaccurate. In

our view the evidence on homicide is not noticeably different from that on

other crimes evaluated in this literature and cannot be easily separated. If

1Contrary to Wilson’s claim, the results in Table 6-7 all rely on models with covariates.

COMMITTEE RESPONSE TO WILSON’S DISSENT 275

the effects of right-to-carry laws on violent and property crimes are ambiguous,

as argued in Chapter 6, we see no reason why the same is not true

of homicide. Professor Wilson may be correct on this matter—it is theoretically

possible—but we maintain that the scientific evidence does not support

his position.

REFERENCE

Rosenbaum, P.R.

2001 Replicating effects and biases. American Statistician 55(3):223-227.

276