Proxy Measures of Ownership
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101
102 103 104 105 106 107 108 109 110 111 112 113 114 115
Research linking firearms to suicide (and violence more generally) is
limited by the lack of detailed information on firearms ownership (see Chapter
2). The existing surveys cannot be used to link ownership to outcomes of
interest and, for that matter, cannot generally be used to draw inferences
about ownership in more precise geographic areas (e.g., counties) that are
often of interest in ecological studies. The GSS, which collects individual and
household information on firearms ownership over time, is representative of
the nine census regions and the nation as whole. Other surveys—the Behavioral
Risk Factor Surveillance System (BRFSS) and the Harvard Injury Control
Research Center Survey (HICRC)—collect information on gun ownership
prevalence rates representative of individual states in certain years.2
2The BRFSS included firearm ownership questions in the 1992-1995 surveys conducted in 21
states. The HICRC can be used to draw inferences on ownership by states in 1996 and 1999.
As a result of these limitations, many ecological studies evaluating the
relationship between firearms and suicide (and homicide) rely on proxies of
ownership, rather than direct measures. Proxies have included the fraction
of homicides committed with a firearm (FH/H), the fraction of suicides
committed with a firearm (FS/S), subscription rates to Guns & Ammo
(G&A), and other similar measures.3
The primary advantage of these proxies, as opposed to survey information,
is that they can be readily computed at state, county, and other finer
geographic levels. The disadvantage is that the proxy is not the variable of
interest; ownership is. Thus, except in very particular circumstances, proxy
measures result in biased estimates of the relationships of interest.
Several studies have explicitly evaluated different proxy measures of
ownership. These assessments generally involve computing a correlation
coefficient between the proxy and self-reported ownership measures from
the GSS or other surveys.4 Azrael et al. (2004), for example, systematically
assess a number of commonly used proxies. Their basic results using the
GSS and other ownership surveys are displayed in Table 7-2. The fraction
of suicides committed with a firearm has the highest correlation among all
of the measures considered, ranging from 0.81 in the state level data to 0.93
when using ownership data from the nine census regions. The fraction of
homicides committed with a firearm has the lowest correlations, and G&A
subscription rates lie between the two.
3See Azrael et al. (2001) for a summary of the different proxy measures used in the
literature.
4These correlations are computed using both geographic and time-series variation in the
ownership and proxy measures. Duggan (2003), in addition to comparing the G&A proxy to
the GSS data, also uses other indicators that are thought to be highly correlated with ownership,
such as the location of gun shows and community characteristics thought to be associated
with ownership.
TABLE 7-2 Correlation Coefficient Between a Proxy and Gun
Ownership Rates
GSS BRFSS HICRC
Proxy N = 9 regions N = 21 states N = 48 states
FS/S 0.93 0.90 0.81
FH/H 0.52 0.19 0.02
Guns & Ammo 0.75 0.67 0.51
NOTE: GSS = Generalized Social Survey; BRFSS = Behavioral Risk Factor Surveillance Survey;
HICRC = Harvard Injury Control Research Center.
SOURCE: Azrael et al. (2004: Table 3). Used with kind permission of Springer Science and
Business Media.
Given this evidence, Azrael et al. conclude that “FS/S is a superior
proxy measure for cross-section analysis, easily computed from available
data for state and large local jurisdictions and valid against survey based
estimates” (p. 50). They also find, using similar methods, that FS/S is a
useful proxy for measuring intertemporal variation in ownership. This finding
appears to share some consensus. Many other researchers have also
accepted FS/S as the best and in fact a nearly ideal proxy for studying the
cross-sectional relationship between firearms and violence. One notable
exception is Duggan (2003), who argues that the FS/S is a poor proxy for
studying suicide, even in cross-sectional analyses.
After reviewing the existing evidence, the committee urges more caution
in using FS/S as a proxy for gun ownership. As Duggan points out, the
most obvious statistical problems concern the circularity of using FS/S as a
proxy in a study of suicide, but the properties of FS/S in other kinds of
studies (e.g., homicide) have also not yet been well described.
There are three basic problems with the existing analysis of proxies of
firearms access. First, there is the problem of the accuracy of self-reported
measures of firearm access, the standard against which the proxies are
being compared. The effects of nonresponse and erroneous response in the
surveys of firearms ownership, and random sampling errors more generally,
have not been investigated. Certainly, response errors alone—as described
both in Chapters 2 and 5—may result in biased estimates of the true
prevalence of gun ownership. Moreover, if persons who are at risk for
attempting suicide are less likely to participate in a household survey than
other persons, then household surveys may not reflect the true relationship
between gun ownership and method choice among persons who are actually
at risk of attempting suicide. Existing research does not yet shed much
light on these possible biases.
Second, there is the problem of aggregation bias in the correlation
analysis. The primary reason for using a proxy is that more direct gun
ownership data may not be available at the appropriate level of aggregation.
But even if the proxy is highly correlated with observed ownership
rates at one geographic level, it need not be correlated with gun ownership
in smaller areas or in subgroups of the population. To explore this possibility,
the committee reexamined the correlation between FS/S and gun ownership
levels using the individual GSS survey responses aggregated to the 100
primary sampling units rather than the 9 census regions. In this case, we
estimated the correlation between the percentage of suicides committed
with a firearm and ownership levels to be 0.646 for firearms of any type
and 0.639 for handguns, substantially less than the correlations reported by
Azrael et al. (2004).
A similar problem is presented in Figure 7-1, which displays the relationship
between FS/S and household gun ownership by age and gender.
This figure shows that the relationship between FS/S and household gun
ownership (as reported in the GSS) varies by age and gender and appears
to have changed between 1980 and 2000; for example, the difference in
patterns of association between males and females has diminished substantially.
Such changes suggest that the relationship between FS/S and
other measures of gun ownership may be influenced by a number of
social, political, and cultural factors that are not yet understood.
Third, even if the estimated correlation coefficients are valid, it is not
clear how this confirms (or refutes) the utility of such a proxy as a measure
of gun ownership. To the contrary, except in very specific circumstances,
regressions with proxies result in biased estimators.5 Under the best cir-
0
0.5
1
1.5
Age Groups
Ratio FS/S to GSS Ratio FS/S to GSS
Ratio FS/S to GSS
1980 1990
0
0.5
1
1.5
2
Age Groups
15-24
35-44
55-64
75-84
2000
0
0.5
1
1.5
2
2.5
Age Groups
Male
Female
15-24
35-44
55-64
75-84
15-24
35-44
55-64
75-84
FIGURE 7-1 Changing relationship of fraction of suicides using a firearm (FS/S) to
household gun ownership (GSS) in the US by age and sex.
5Maddala (1992) and Wooldridge (2000) illustrate the biases created by proxies measures
in linear mean regression models.
cumstances, proxies reveal the sign but not the magnitude of the relationship
of interest (Krasker and Pratt, 1986; Maddala, 1992). Azrael et al.
(2004) attempt to provide some insight into this scale problem by running
a simple linear regression of the form:
PREV = 0 + 1FS/S + U,
where PREV is the true ownership rate, FS/S is the observed proxy, 0 and
1 are unknown coefficients, and U is a mean zero unobserved random
variable, conditional on FS/S. The estimated slope coefficient is near unity,
suggesting that a one-unit increase in FS/S implies a one-unit increase in the
expected prevalence rate. The authors take this result, coupled with the
strong cross-sectional correlation coefficients, as evidence supporting the
idea that the FS/S proxy leads to (nearly) unbiased estimators of both the
sign and the magnitude of the relationships of interest.
This logic, however, could be misleading. In the classical omitted variable
model described by Wooldridge (2000:284-286), a unit coefficient on
1 is sufficient. In other models, however, unbiased estimators may not
exist. It is difficult to assess whether these conditions result in an unbiased
estimator since Azrael et al. (2004) do not clearly describe the model they
have in mind.6 This problem becomes particularly important when FS/S is
being used as a proxy in the study of suicide, and it seems to be an important
source of misunderstanding. For example, Miller et al. (2002a, 2002c)
assess the potential biases created by the FS/S proxy in the study of suicide,
using statistical simulations. These authors claim to demonstrate that FS/S
is not, by construction, correlated with the overall suicide rate, so that FS/
S may be appropriately used as a measure of gun ownership in such a study.
However, they do not explicitly describe their statistical model, and their
description of the Monte Carlo simulation does not provide enough information
to understand much about what was done. Furthermore, it is not
6No one, as far as we can tell, has investigated the actual linear or nonlinear shape of the
relation between FS/S and gun ownership. Furthermore, Azrael et al. do not consider issues
associated with the statistical error of the model. Suppose instead that we consider another
linear model, in which the gun suicide rate is a function of the gun ownership prevalence:
FS/S = g0 + g1PREV + V, with V being a mean zero unobserved random variable, conditional
on PREV (see, for example, Duggan, 2003). Indeed, this model may be more plausible if one
believes that gun ownership is a causal factor in firearm-related suicides. And, if this were
correct, then in models of the relation between suicide and the FS/S proxy, the explanatory
variable (FS/S) would be correlated with the regression “error,” a well-known cause of bias in
regression analysis. In any case, the two models are not the same and do not have the same
implications for the effects of using FS/S as a proxy. In the first model, the measurement
errors are mean-independent of the proxy but not of the variable of interest, prevalence. In
the second model, the measurement errors are independent of prevalence but not of the
proxy.
BOX 7-2