Lott’s Results
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101
102 103 104 105 106 107 108 109 110 111 112 113 114 115
Table 6-1 (first row) displays Lott’s estimates from Model 6.1. Lott
finds that where they have been adopted, right-to-carry laws have reduced
homicide by about 8 percent, rapes by about 5 percent, and aggravated
assaults by about 7 percent (Lott, 2000:51). Lott also finds that adoption of
right-to-carry laws may increase the rates of nonviolent property crimes
(burglary, larceny, auto theft). In theory, this is possible, as criminals substitute
away from crimes that involve contact with victims toward crimes
that do not involve encounters with victims.
Rows 2 and 3 of Table 6-1 report the results of the committee’s replication
of these estimates. In row 2, we use the revised original data set and
Lott’s computer programs. The committee was unable to replicate Lott’s
estimate of the reduction in the murder rate, although the estimates are
close and consistent with the conclusion that right-to-carry laws reduce the
incidence of murder. Through communication with Lott, the committee
learned that the data used to construct Table 4.1 of Lott (2000) were lost
and that the data supplied to the committee are a reconstruction and not
necessarily identical to the original data.
Row 3 displays estimates using the revised new data set restricted to
period 1977-1992. The estimates from these revised data are substantially
different from those originally reported by Lott (2000). In the dummy
variable model, the magnitude of the estimated reduction in the rates of
violent crime and aggravated assault was reduced, the estimated reduction
in the murder rate increased, and the sign of the estimated effects of rightto-
carry laws on robbery reversed. Moreover, the effects of right-to-carry
laws on violent crime are no longer statistically significantly different from
zero at the 5 percent significance level. Finally, the estimated increase in the
rates of all property crimes increased substantially.
Table 6-2 presents estimates of the trend model. The first row displays
Lott’s estimates. Lott finds the passage of right-to-carry laws to be associated
with changes in the crime trend. He finds a 0.9 percent reduction in the
annual rate of growth of violent crime overall, and a 0.6 percent reduction
in the rate of growth of property crimes. Row 2 of Table 6-2 shows the
committee’s attempt to replicate Lott’s results using the revised original
data set. The committee was unable to replicate most of the results in Lott’s
Table 4.8. Through communication with Lott, the committee learned that
TABLE 6-1 Dummy Variable Model with Common Time Pattern,
Original and Revised Dataa
Violent
Sample Years Crime Murder Rape
1. Lott (2000) Original 1992 1992 –4.9% –7.7% –5.3%
2. Committee
replication Revised 1992b 1992 –4.91 –7.30 –5.27
SE (0.98)** (1.57)** (1.22)**
3. Committee
replication Revised 2000c 1992 –1.76 –9.01 –5.38
SE (1.07) (1.70)** (1.33)**
aThe regressions use the covariates and specification from the original Lott and Mustard
(1997) models that do not control for state poverty, unemployment, death penalty execution
rates, or regional time trends. The controls include the arrest rate for the crime category in
question (AOVIOICP), population density in the county, real per capita income variables
(RPCPI RPCUI RPCIM RPCRPO), county population (POPC), and variables for the percentage
of the population that is in each of many race x age x gender categories (e.g., PBM1019 is
the percentage of the population that is black, male, and between ages 10 and 19). The “no
this is because there are many misprints in Table 4.8. Nonetheless, Lott’s
and the committee’s results have the same signs for all crimes except aggravated
assault. Row 3 displays estimates using the revised new data set
restricted to the period 1977-1992. These new results tend to show larger
reductions in the violent crime trends than those found using the revised
original data.
Other Statistical Evaluations of Right-to-Carry Laws
Researchers have estimated the effects of right-to-carry laws using Lott’s
or related data and models. Many of these studies have found that the use
of plausible alternative data, control variables, specifications, or methods
of computing standard errors, weakens or reverses the results. Tables 6-3
and 6-4 display estimates from selected studies that illustrate variability in
the findings about the effects of right-to-carry laws. The committee does
not endorse particular findings or consider them to provide better estimates
of the effects of right-to-carry laws than do Lott’s results. Moreover, the
committee recognizes that several independent investigators have used alternative
models or data to obtain results that are consistent with Lott’s.
These investigators include Bartley and Cohen (1998) and Moody (2001).
We focus on the conflicting results in this section because they illustrate a
variability of the findings that is central to the committee’s evaluation of
their credibility.
Aggravated Property
Assault Robbery Crimes Auto Theft Burglary Larceny
–7.0% –2.2% 2.7% 7.1% 0.05% 3.3%
–7.01 –2.21 2.69 7.14 0.05 3.34
(1.14)** (1.33) (0.72)** (1.14)** (0.76) (0.89)**
–5.60 1.17 5.84 10.28 4.12 6.82
(1.25)** (1.45) (0.76)** (1.24)** (0.83)** (0.82)**
controls” specification” includes county fixed effects, year dummies, and the dummy for
whether the state has a right-to-carry law.
bUsing Lott’s reconstruction of his original 1977-1992 data.
cUsing the revised new data set, which contains observations, 1977-2000, even though the
estimates in this row use data only through 1992.
NOTE: All samples start in 1977. SE = standard error. Standard errors are in parentheses,
where * = significant at 5% and ** = significant at 1%.
TABLE 6-2 Trend Model with Common Time Pattern, 1977-1992a
Violent
Sample Years Crime Murder Rape
1. Lott (2000) Original 1992 1992 –0.9% –3.0% –1.4%
2. Committee
replication Revised 1992b 1992 –0.50 –4.25 –1.37
SE (0.41) (0.65)** (0.51)**
3. Committee
replication Revised 2000c 1992 –2.15 –3.41 –3.37
SE (0.39)** (0.62)** (0.48)**
aThe regressions use the covariates and specification from the original Lott and Mustard
(1997) models that do not control for state poverty, unemployment, death penalty execution
rates, or regional time trends. The controls include the arrest rate for the crime category in
question (AOVIOICP), population density in the county, real per capita income variables
(RPCPI RPCUI RPCIM RPCRPO), county population (POPC), and variables for the percentage
of the population that is in each of many race age gender categories (e.g., PBM1019 is
the percentage of the population that is black, male, and between ages 10 and 19).
Control Variables and Specification
The most common modifications to Lott’s original analyses of right-tocarry
laws has been to assess the sensitivity of the findings to variation in the
control variables or the specification of the model. Lott’s basic model relies
on dozens of controls, but concerns have been raised that some controls may
be missing, others may be unnecessary, and still others may be endogenous
(that is, related to the unobserved determinates of county crime rates).
Duggan (2001), for example, raises concerns that county-level control
variables may not be precisely measured on an annual basis and that the
arrest rate control variable, which includes the crime rate in the denominator,
may bias the estimates. In response to these concerns, Duggan estimated
a simple dummy variable model that controls only for year and
county fixed effects.7 Duggan drops all other covariates from the model.
When estimated on all county-year observations with nonmissing crime
7Duggan also changed the coding of the dates of adoption of right-to-carry laws, although
this had only a minimal effect on the estimates. According to Duggan (2001) and others (see, for
example, Ayres and Donohue, 2003a), there is an inconsistency in the coding used by Lott and
Mustard. Duggan finds that in 8 of the 10 right-to-carry states, the adoption date is defined as
the year the law was passed, but in 2 states, Florida and Georgia, the adoption date is set to the
calendar year after the law was passed. Lott, in personal communications, maintains that the
dates are coded correctly. The committee does not take a stand on which coding is correct.
Aggravated Property
Assault Robbery Crimes Auto Theft Burglary Larceny
–0.5% –2.7% –0.6% –0.1% –0.3% –1.5%
0.46 –2.72 –0.69 –0.31 –1.58 –0.11
(0.48) (0.56)** (0.30)* (0.48) (0.32)** (0.37)
–2.63 –3.02 –1.13 0.25 –1.80 –0.84
(0.45)** (0.53)** (0.27)** (0.45) (0.30)** (0.30)**
bUsing Lott’s reconstruction of his original 1977-1992 data.
cUsing the revised new data set, which contains observations, 1977-2000, even though the
estimates in this row use data only through 1992.
NOTE: All samples start in 1977. SE = standard error. Standard errors are in parentheses,
where * = significant at 5% and ** = significant at 1%.
data, this reduced the magnitude of the estimated reduction in the rates of
murder and aggravated assault, and it reversed the signs of the estimated
effects of right-to-carry laws on rape, robbery, and all violent crime. That
is, according to Duggan’s estimates, adoption of right-to-carry laws increases
the frequencies of rape, robbery, and violent crime as a whole.
Moreover, Duggan found there is no statistically significant effect of rightto-
carry laws on violent crimes (at the 5 percent significance level).
Other researchers have varied the specification of the model, allowing
for the effects of right-to-carry laws to be more heterogeneous. Black and
Nagin (1998), for example, estimated a dummy variable model in which
the effects of right-to-carry laws are allowed to vary among states (that is,
the coefficient d is allowed to take different values for different states).
Plassmann and Tideman (2001) estimate a nonlinear Poisson regression
model with a restricted set of covariates, but otherwise similar to Model
6.1. Ayres and Donohue (2003a) combined Models 6.1 and 6.2, thereby
obtaining a hybrid model in which adoption of right-to-carry laws can
affect both the level and the trend of crime. The results from these analyses,
which vary the way in which right-to-carry laws can effect crime, are highly
variable, with some suggesting that the laws increase crime, others suggesting
that they decrease crime, and many being statistically insignificant.
In Black and Nagin (1998), for example, only Florida has a statistically
significant decrease in the murder rate following adoption of a right-tocarry
law, and only West Virginia has a statistically significant increase in
TABLE 6-3 Summary of Selected Studies: Dummy Variable Model
(percentage) (shaded cells indicate a positive coefficient)
Violent
Source Modification Crime Murder Rape
Lott (2000) Original specification and data –5* –8* –5*
Moody Unweighted –6* –4* –5*
State-level analysis –11 15 –22*
Duggana County and time effects only –1 –6 3
All counties 0 –1 6
Black and Nagin Large counties –9* –4
Exclude Florida –1 1
Florida –27.7* –17*
Georgia –5.2 –5
Idaho –21 –10
Maine 7.2 4
Mississippi 5.4 32*
Montana –36.7 –97*
Oregon –5.9 4
Pennsylvania –8.9 4
Virginia 3.9 –8
West Virginia 72* –29*
Plassmann and No control for arrest rate –7* –6*
Tideman All counties –2 –5
Count model (Poisson) –11* –4*
Florida –24* –16*
Georgia –8* –16*
Idaho –6 10*
Maine 1 –2
Mississippi 5 11*
Montana –7 –4
Oregon –10* –2
Pennsylvania –5 14*
Virginia 8* –3
West Virginia 5 –1
Ayres and State trends 0 –9* –2
Donohue (2003a) 1977-1997 data 2 0 3
State level analysis
State and time effects only –3 –8 –1
1977-1999 data 9* –2 6*
Plassmann and Regional trend + others
Whitleya,b 1977-2000 data –3 –6* –7*
Ayres and Regional trends + other controls
Donohue (2003b)a,b 1977-2000 corrected data 0 –4 –5
continued
Aggravated Property Auto
Assault Robbery Crimes Theft Burglary Larceny
–7* –2 3* 7* 0 3*
–9* –1 3* 3 1 4*
–18* –10 1 –9 4 3
–6 4 6* 9* 8* 5
–5 10 7* 11* 10* 5
–7* –3
–6* –5
–7 7
–4 8
–31* –64*
–52* –33*
–45* 10
–71* –14
–17* –4
7* –5
–16* –12
–3 9
–1
2
6*
–3*
1
–41*
–22*
25*
–27*
–48*
–14*
–5*
–9*
3 –8 –1* –1* –4* 1
7* 0 –1 4 1 4
–10 –5 7* 9* 9* 7*
4* 16* 16* 23* 14* 16*
–2 –5 4 9* 0 6
1 –3 6* 11* 2 8*
aUses clustered sampling standard errors.
bAdded covariates for state poverty, unemployment, death penalty execution rates, and
regional time trends.
TABLE 6-3 Continued
Violent
Source Modification Crime Murder Rape
Standard errors
Lott (2000) Unadjusted standard errors 0.98 1.57 1.22
Duggan State clustered standard errors 2.31 2.95 2.32
Helland and Placebo standard errors 4.9 6.4 5.6
Tabarrok
its murder rate. The estimated changes in the murder rates of other states
that adopted right-to-carry laws are sometimes positive (three cases) and
sometimes negative (five cases) and are not statistically significantly different
from zero. Black and Nagin also report variations in the directions and
statistical significance of changes in the rates of rape and aggravated assault.
They report no statistically significant increases in robberies, but only
2 of the 10 states that adopted right-to-carry laws had statistically signifi-
TABLE 6-4 Summary of Selected Studies: Trend and Hybrid Variable
Model (shaded cells indicate a positive coefficient)
Violent
Source Modification Crime Murder Rape
Lott (2000) Original specification and data 2* –3* –1*
Lott (2000)a 1977-1996 –2* –2* –3*
Ayres and Hybrid model: Level 7* 3 7*
Donohue (2003a) Trend –2* –5* –3*
1977-1997 data: Level 0 7* 6*
Trend –2* –4* –3*
Plassmann and Regional trend + others
Whitleya,b 1977-2000 data –1 –2 –3*
Ayres and Regional trends + other controls
Donohue (2003b)a,b 1977-2000 corrected data 0 –2 –2
aAdded covariates for state poverty, unemployment, death penalty execution rates, and
regional time trends.
bStandard errors adjusted for state clustering.
NOTES: Shaded cells indicate a positive coefficient estimate and * indicates the estimate is
statistically significant at the 5% significance level. Unless otherwise noted, the standard
errors are not adjusted for state-level clustering. Exceptions: Duggan, Plassmann and Tideman,
Ayres and Donohue.
Aggravated Property Auto
Assault Robbery Crimes Theft Burglary Larceny
1.14 1.33 0.72 1.14 0.76 0.89
2.77 3.34 1.89 2.59 2.29 2.27
6.6 7.5 5.1 6.5 5.7 5.7
Aggravated Property Auto
Assault Robbery Crimes Theft Burglary Larceny
–1* –3* –1* 0* –2* 0
–3* –3* –2* –3* –1* –2*
10* –3 0 0 –3 0
–2 –1 0 0 0 1
6* 4 –1 9* 4* 5*
–3* –4* 0 –2* –3* –2*
–2 –3* 0 0 –2 –1
–1 –2 0 0 –1 0
NOTES: Shaded cells indicate a positive coefficient estimate and * indicates the estimate is
statistically significant at the 5% significance level. Unless otherwise noted, the standard
errors are not adjusted for state-level clustering. Exceptions: Duggan, Plassmann and Tideman,
Ayres and Donohue.
cant decreases. In summary, according to Black and Nagin, adoption of a
right-to-carry law may increase, decrease, or have no discernible effect on
the crime rate depending on the crime and the state that are involved.8
8To avoid selection problems associated with using counties with positive crime rates,
Black and Nagin also restricted their analysis to counties with populations of 100,000 or
more. This was done to mitigate a possible bias arising from Lott’s use of the arrest rate as an
explanatory variable. The arrest rate is the number of arrests divided by the number of crimes
Plassmann and Tideman (2001) document similar variability in the
estimates. To account for the fact that county-level crime data include a
large number of observations for which the outcome variable equals zero,
Plassmann and Tideman estimate a nonlinear count data model. Using data
from all counties with reported crime figures, the resulting estimates on
murder and rape are consistent with Lott’s findings, but the sign of the
estimated effect of right-to carry laws on robbery is reversed. Furthermore,
when the effects of right-to-carry laws are allowed to vary among states,
Plassmann and Tideman found that adoption of a right-to-carry law may
increase, decrease, or have no effect on the crime rate depending on the
crime and state that are involved. Consider, for example, murder. Right-tocarry
laws are estimated to have a statistically significant decrease in the
murder rate in Florida, Georgia, and Oregon following adoption of a rightto-
carry law. Virginia has a statistically significant increase in its murder
rate. The changes in the murder rates of other states that adopted right-tocarry
laws are not statistically significantly different from zero. Plassmann
and Tideman conclude by noting the fragility in the estimated effects of
right-to-carry laws: “While this ambiguous result is somewhat discouraging,
it is not very surprising. Whenever the theoretically possible and in
practice plausible effects of public policy are ambiguous, it can be expected
that the effects of such a policy will differ across localities that are clearly
different from each other” (p. 797).
Finally, the added flexibility of the hybrid model estimated by Ayres
and Donohue (2003a) produces estimation results that are different from
Lott’s.9 The results found when using the revised original data (1977-
and is undefined in counties that report no crimes of the types analyzed. Therefore, these
counties are not included in Lott’s analysis. Because the denominator of the arrest rate variable
contains the dependent variable in Lott’s models, it is possible that dropping no-crime
counties biases the results of his analysis. Nearly all of the low-crime counties have populations
below 100,000. Therefore, use of only counties with larger populations largely overcomes
the problem of missing arrest rate data without creating a bias.
Lott (1999:8-9; 2000:142-143), however, has argued that Black’s and Nagin’s results are
unreliable because they eliminated 85 percent of the counties in the nation (all the counties with
populations of less than 100,000). In particular, they used only one county in West Virginia.
Lott (2000: Table 4.9) presents his own estimation results according to which his findings are
largely unaffected by disaggregating the right-to-carry effect by state. However, Lott does not
report the details of his analysis or the statistical significance levels of his estimates. Moreover,
his response does not explain why Black and Nagin found statistically significant increases in
some crime rates for some states following passage of right-to-carry laws.
9The committee takes no position on whether the hybrid model provides a correct description
of crime levels or the effects of right-to-carry laws. The important feature of the hybrid
model is that it nests Models 6.1 and 6.2.
1992) are illustrated in Figure 6-1, which shows the “relative trend” in
the logarithm of the violent crime rate obtained from the Ayres and
Donohue model for a hypothetical county in which a right-to-carry law is
adopted in year 8. The relative trend is the difference between the crime
trend in the adopting county and the trend in a nonadopting county with
the same values of the explanatory variables X. According to the figure,
adoption of the law increased the level of violent crime but accelerated a
decreasing (relative) trend. Ayres and Donohue obtained similar results
for rape and aggravated assault. For murder, the shift in the level is not
statistically significant, but there is a statistically significant downward
shift in the trend. There is no statistically significant effect on either the
level or the trend for robbery and property crimes. Ayres and Donohue
also report estimates from an expanded data set that includes the years
1977-1999. The results found using these data, which are reported in
Table 6-4, are similar.
Updated Sample Endpoint
Several researchers, including Lott, have assessed whether the basic
findings from Models 6.1 and 6.2 continue to hold when using more recent
data. In the epilogue to the second edition of his book, Lott (2000: Table
9.1) analyzes data covering the period 1993-1996. Plassmann and Whitley
(2003) use data through 2000. In addition to updating the data, these Trend of Log(Crime Rate)
Year
0 5 10 15
-0.2
-0.15
-0.1
-0.05
0
FIGURE 6-1 Trend in the logarithm of the violent crime rate.
researchers also change the model specification. In particular, these analyses
include additional covariates (i.e., state poverty, unemployment and
death penalty execution rates) and allow for region-interacted time patterns,
as opposed to a common time trend used in the original Lott models
(Lott 2000:170).
With these new models and the updated sample endpoints, Lott found
that the basic conclusions from the trend model are robust to the additional
years of data covering the periods 1977-1996. Likewise, Plassmann and
Whitley (2003) found that when the data are updated to cover the period
1977-2000, the trend model estimates of the effects of right-to-carry laws
on crime continue to be negative, but only the estimates for rape and
robbery are statistically significant. In the dummy variable model, Plassmann
and Whitley found negative coefficient estimates for the right-tocarry
coefficient for each violent crime category and positive coefficients for
each of the property categories.
Ayres and Donohue (2003b), however, document a number of errors in
the data used by Plassmann and Whitley, and Lott’s revised new data
correct these errors. Plassmann, in communications with the committee,
has agreed that the changes to these data are appropriate. Using the revised
new data, the committee exactly replicated the results reported by Ayres
and Donohue (2003b).
In particular, Ayres and Donohue (2003b) found that rerunning the
dummy variable model regressions using the corrected data reduced the
magnitude of the estimated reduction in the rates of violent crime, murder,
rape, and robbery, and it reversed the sign of the estimated effects of rightto-
carry laws on aggravated assault. Moreover, none of the negative estimates
is statistically significant, while effects for larceny, auto theft, and
property crime overall are positive and significant. Likewise, the changes in
the crime trends are generally small in absolute value, and none of the
changes is significantly different from zero (see Table 6-4).10
Maltz and Targonski (2002) do not update the data but instead assess
the quality of the county crime data used in the empirical research on rightto-
carry laws. In particular, they note that not all police jurisdictions report
their crime levels to the FBI and argue that there is systematic underreporting
in the UCR. Maltz and Targonski (2002:298) conclude that “county-level
crime data, as they are currently constituted, should not be used, especially
in policy studies.” However, Maltz and Targonski do not estimate the
magnitude of the effects of underreporting on the results obtained by Lott
and others. Thus, it is not known whether correcting for underreporting, if
it were possible, would change any of the results.
10Both Ayres and Donohue (2003b) and Plassmann and Whitley (2003) use standard errors
that account for state clustering.
Lott and Whitley (2002: Figure 5) report estimates of the effects of
right-to-carry laws that are obtained by dropping from the data counties
with large fractions of missing UCR reports. Lott’s and Whitley’s figure
shows estimated trends in crime levels before and after adoption of right-tocarry
laws, and they claim that these trends support the conclusion that
adoption of right-to-carry laws reduces crime. The committee disagrees.
According to Figure 5b of Lott and Whitley (2002), the murder rate peaks
and begins to decrease at an accelerating rate approximately 5 years before
the adoption of right-to-carry laws. Aggravated assault decreases prior to
adoption and then increases for approximately 3 years following adoption
before starting to decrease again (Figure 5e). Adoption has no effect on rape
(Figure 5c). The rate of violent crimes as a whole decreases up to the time of
adoption and then remains unchanged until approximately 3 years after
adoption before beginning a steeper decline (Figure 5a). Among violent
crimes, only robbery displays a decrease immediately following adoption
(Figure 5d). However, this followed a period during which the robbery rate
first increased and then remained constant for approximately 5 years. In
summary, the committee concludes that it is at least possible that errors in
the UCR data may account for some of Lott’s results.
Standard Errors
A final point that has been argued in the literature is that conventional
standard errors reported by Lott and others are not appropriate. The statistical
analyses of dummy variable and trend models are conducted using a
county-year pair as the unit of analysis. Right-to-carry laws, however, almost
always vary only at the state level. Consequently, some investigators
believe that treating the county-level observations as if they are statistically
independent may lead to estimates of the standard errors that underestimate
their true magnitude. These investigators make adjustments for statelevel
clustering that inflate their standard errors. For example, the standard
error for the dummy variable model estimate of the effect of right-to-carry
laws on violent crime increases from 0.98 when reporting the unadjusted
standard error, to 2.31 when estimating clustered sampling standard errors
(Duggan, 2001), to 4.9 when using the methods advocated by Helland and
Tabarrok (2004) (see Table 6-3). The fact that the adjustments in most
cases greatly increase the standard errors is a reason for concern. Once the
standard errors have been adjusted for clustering, very few of the point
estimates, in any of the models, using any of the data sets, are statistically
different from zero.
However, investigators reporting cluster-adjusted standard errors do
not formally explain the need for these adjustments. These adjustments, in
fact, are not supported in the basic models specified in Equations 6.1 and
6.2. Instead, those who argue for presenting clustered standard errors often
cite Moulton (1990) as the source of their belief that adjustments are needed.
Moulton considered a model in which there is an additive source of variation
(or additive effect) that is the same for all observations in the same
cluster. He showed that ignoring this source of variation leads to standard
errors that are too low. Investigators who make clustering corrections usually
consider the counties in a state to constitute one of Moulton’s clusters
and appear to believe that the absence of state-level additive effects in their
models causes standard errors to be too low. The models estimated in this
literature, including those of Lott and his critics, typically contain countylevel
fixed effects (the constants gi in equations 6.1 and 6.2). Every county
is always in the same state, so, any state-level additive effect simply adds a
constant to the gi’s of the counties in that state. The constant may vary
among states but is the same for all counties in the same state. The combined
county- and state-level effects are indistinguishable from what would
happen if there were no state-level effects but each gi for the counties in the
same state were shifted by the same amount. Therefore, state-level effects
are indistinguishable from county-level effects. Any state-level effects are
automatically included in the gi’s. There is no need for adjustments for
state-level clustering.
Other observationally equivalent but different models can support the
use of adjusted standard errors. If, for example, the effects of right-to-carry
laws (or other explanatory variables) vary across states, then the assumption
of independence across counties would be incorrect. Adjustments to
the standard errors can allow for uncertainty arising from the possibility
that the coefficients of variables in the model that are not allowed to vary
across states, in fact, vary randomly across states. The adjustments made by
Duggan and Plassmann and Whitley, for example, can be used to correct
estimated standard errors for this possibility (see Wooldridge, 2003).
These alternative models have not been discussed in the literature or by
the committee. Thus, it is not clear whether the models that would support
using clustered-sampling-adjusted standard errors are appropriate to evaluate
the effects of right-to-carry laws. At the most basic level, researchers
need to assess whether models that support clustering are of interest.11 If,
for example, coefficients can vary randomly among states, Models 6.1 and
6.2 reveal the mean coefficients. In other words, if different states have
different coefficients, then researchers estimate an average over states. It is
11There are also important technical issues to consider. For example, a commonly used
method for making these corrections is reliable only when the number of “clusters” (here
states) is large, and there is reason to think that the 50 states do not constitute a large enough
set of clusters to make these methods reliable.
not clear why anyone should care about this average, which is not related in
any obvious way to (for example) nationwide benefits of right-to-carry
laws. If coefficients vary among states, then it may be much more useful to
estimate the coefficients for each state. It is entirely possible that the effects
of right-to-carry laws vary among states, even after controlling everything
else that is in the model. If they do, it may be much more useful to know
which states have which coefficients, to see the magnitude of the variation,
and to have a chance of finding out whether it is related to anything else
that is observable. Of course, a number of the studies summarized above
have varied Lott’s model by allowing the effect of right-to-carry laws to
differ by states (see, for example, Black and Nagin, 1998, and Plassmann
and Tideman, 2001). A model in which coefficients are estimated separately
for each state does not require adjustment of standard errors.
In summary, whether adjustment of standard errors is needed depends
on the details of the effects that are being estimated and the model that is
used to estimate them. These issues have not been investigated in studies of
right-to-carry laws to date. Adjusted standard errors are not needed for
Models 6.1 and 6.2. The precision of estimates from these models should be
evaluated using unadjusted standard errors.