Proxy Measures of Ownership

Research linking firearms to suicide (and violence more generally) is

limited by the lack of detailed information on firearms ownership (see Chapter

2). The existing surveys cannot be used to link ownership to outcomes of

interest and, for that matter, cannot generally be used to draw inferences

about ownership in more precise geographic areas (e.g., counties) that are

often of interest in ecological studies. The GSS, which collects individual and

household information on firearms ownership over time, is representative of

the nine census regions and the nation as whole. Other surveys—the Behavioral

Risk Factor Surveillance System (BRFSS) and the Harvard Injury Control

Research Center Survey (HICRC)—collect information on gun ownership

prevalence rates representative of individual states in certain years.2

2The BRFSS included firearm ownership questions in the 1992-1995 surveys conducted in 21

states. The HICRC can be used to draw inferences on ownership by states in 1996 and 1999.

As a result of these limitations, many ecological studies evaluating the

relationship between firearms and suicide (and homicide) rely on proxies of

ownership, rather than direct measures. Proxies have included the fraction

of homicides committed with a firearm (FH/H), the fraction of suicides

committed with a firearm (FS/S), subscription rates to Guns & Ammo

(G&A), and other similar measures.3

The primary advantage of these proxies, as opposed to survey information,

is that they can be readily computed at state, county, and other finer

geographic levels. The disadvantage is that the proxy is not the variable of

interest; ownership is. Thus, except in very particular circumstances, proxy

measures result in biased estimates of the relationships of interest.

Several studies have explicitly evaluated different proxy measures of

ownership. These assessments generally involve computing a correlation

coefficient between the proxy and self-reported ownership measures from

the GSS or other surveys.4 Azrael et al. (2004), for example, systematically

assess a number of commonly used proxies. Their basic results using the

GSS and other ownership surveys are displayed in Table 7-2. The fraction

of suicides committed with a firearm has the highest correlation among all

of the measures considered, ranging from 0.81 in the state level data to 0.93

when using ownership data from the nine census regions. The fraction of

homicides committed with a firearm has the lowest correlations, and G&A

subscription rates lie between the two.

3See Azrael et al. (2001) for a summary of the different proxy measures used in the

literature.

4These correlations are computed using both geographic and time-series variation in the

ownership and proxy measures. Duggan (2003), in addition to comparing the G&A proxy to

the GSS data, also uses other indicators that are thought to be highly correlated with ownership,

such as the location of gun shows and community characteristics thought to be associated

with ownership.

TABLE 7-2 Correlation Coefficient Between a Proxy and Gun

Ownership Rates

GSS BRFSS HICRC

Proxy N = 9 regions N = 21 states N = 48 states

FS/S 0.93 0.90 0.81

FH/H 0.52 0.19 0.02

Guns & Ammo 0.75 0.67 0.51

NOTE: GSS = Generalized Social Survey; BRFSS = Behavioral Risk Factor Surveillance Survey;

HICRC = Harvard Injury Control Research Center.

SOURCE: Azrael et al. (2004: Table 3). Used with kind permission of Springer Science and

Business Media.

Given this evidence, Azrael et al. conclude that “FS/S is a superior

proxy measure for cross-section analysis, easily computed from available

data for state and large local jurisdictions and valid against survey based

estimates” (p. 50). They also find, using similar methods, that FS/S is a

useful proxy for measuring intertemporal variation in ownership. This finding

appears to share some consensus. Many other researchers have also

accepted FS/S as the best and in fact a nearly ideal proxy for studying the

cross-sectional relationship between firearms and violence. One notable

exception is Duggan (2003), who argues that the FS/S is a poor proxy for

studying suicide, even in cross-sectional analyses.

After reviewing the existing evidence, the committee urges more caution

in using FS/S as a proxy for gun ownership. As Duggan points out, the

most obvious statistical problems concern the circularity of using FS/S as a

proxy in a study of suicide, but the properties of FS/S in other kinds of

studies (e.g., homicide) have also not yet been well described.

There are three basic problems with the existing analysis of proxies of

firearms access. First, there is the problem of the accuracy of self-reported

measures of firearm access, the standard against which the proxies are

being compared. The effects of nonresponse and erroneous response in the

surveys of firearms ownership, and random sampling errors more generally,

have not been investigated. Certainly, response errors alone—as described

both in Chapters 2 and 5—may result in biased estimates of the true

prevalence of gun ownership. Moreover, if persons who are at risk for

attempting suicide are less likely to participate in a household survey than

other persons, then household surveys may not reflect the true relationship

between gun ownership and method choice among persons who are actually

at risk of attempting suicide. Existing research does not yet shed much

light on these possible biases.

Second, there is the problem of aggregation bias in the correlation

analysis. The primary reason for using a proxy is that more direct gun

ownership data may not be available at the appropriate level of aggregation.

But even if the proxy is highly correlated with observed ownership

rates at one geographic level, it need not be correlated with gun ownership

in smaller areas or in subgroups of the population. To explore this possibility,

the committee reexamined the correlation between FS/S and gun ownership

levels using the individual GSS survey responses aggregated to the 100

primary sampling units rather than the 9 census regions. In this case, we

estimated the correlation between the percentage of suicides committed

with a firearm and ownership levels to be 0.646 for firearms of any type

and 0.639 for handguns, substantially less than the correlations reported by

Azrael et al. (2004).

A similar problem is presented in Figure 7-1, which displays the relationship

between FS/S and household gun ownership by age and gender.

This figure shows that the relationship between FS/S and household gun

ownership (as reported in the GSS) varies by age and gender and appears

to have changed between 1980 and 2000; for example, the difference in

patterns of association between males and females has diminished substantially.

Such changes suggest that the relationship between FS/S and

other measures of gun ownership may be influenced by a number of

social, political, and cultural factors that are not yet understood.

Third, even if the estimated correlation coefficients are valid, it is not

clear how this confirms (or refutes) the utility of such a proxy as a measure

of gun ownership. To the contrary, except in very specific circumstances,

regressions with proxies result in biased estimators.5 Under the best cir-

0

0.5

1

1.5

Age Groups

Ratio FS/S to GSS Ratio FS/S to GSS

Ratio FS/S to GSS

1980 1990

0

0.5

1

1.5

2

Age Groups

15-24

35-44

55-64

75-84

2000

0

0.5

1

1.5

2

2.5

Age Groups

Male

Female

15-24

35-44

55-64

75-84

15-24

35-44

55-64

75-84

FIGURE 7-1 Changing relationship of fraction of suicides using a firearm (FS/S) to

household gun ownership (GSS) in the US by age and sex.

5Maddala (1992) and Wooldridge (2000) illustrate the biases created by proxies measures

in linear mean regression models.

cumstances, proxies reveal the sign but not the magnitude of the relationship

of interest (Krasker and Pratt, 1986; Maddala, 1992). Azrael et al.

(2004) attempt to provide some insight into this scale problem by running

a simple linear regression of the form:

PREV = 0 + 1FS/S + U,

where PREV is the true ownership rate, FS/S is the observed proxy, 0 and

1 are unknown coefficients, and U is a mean zero unobserved random

variable, conditional on FS/S. The estimated slope coefficient is near unity,

suggesting that a one-unit increase in FS/S implies a one-unit increase in the

expected prevalence rate. The authors take this result, coupled with the

strong cross-sectional correlation coefficients, as evidence supporting the

idea that the FS/S proxy leads to (nearly) unbiased estimators of both the

sign and the magnitude of the relationships of interest.

This logic, however, could be misleading. In the classical omitted variable

model described by Wooldridge (2000:284-286), a unit coefficient on

1 is sufficient. In other models, however, unbiased estimators may not

exist. It is difficult to assess whether these conditions result in an unbiased

estimator since Azrael et al. (2004) do not clearly describe the model they

have in mind.6 This problem becomes particularly important when FS/S is

being used as a proxy in the study of suicide, and it seems to be an important

source of misunderstanding. For example, Miller et al. (2002a, 2002c)

assess the potential biases created by the FS/S proxy in the study of suicide,

using statistical simulations. These authors claim to demonstrate that FS/S

is not, by construction, correlated with the overall suicide rate, so that FS/

S may be appropriately used as a measure of gun ownership in such a study.

However, they do not explicitly describe their statistical model, and their

description of the Monte Carlo simulation does not provide enough information

to understand much about what was done. Furthermore, it is not

6No one, as far as we can tell, has investigated the actual linear or nonlinear shape of the

relation between FS/S and gun ownership. Furthermore, Azrael et al. do not consider issues

associated with the statistical error of the model. Suppose instead that we consider another

linear model, in which the gun suicide rate is a function of the gun ownership prevalence:

FS/S = g0 + g1PREV + V, with V being a mean zero unobserved random variable, conditional

on PREV (see, for example, Duggan, 2003). Indeed, this model may be more plausible if one

believes that gun ownership is a causal factor in firearm-related suicides. And, if this were

correct, then in models of the relation between suicide and the FS/S proxy, the explanatory

variable (FS/S) would be correlated with the regression “error,” a well-known cause of bias in

regression analysis. In any case, the two models are not the same and do not have the same

implications for the effects of using FS/S as a proxy. In the first model, the measurement

errors are mean-independent of the proxy but not of the variable of interest, prevalence. In

the second model, the measurement errors are independent of prevalence but not of the

proxy.

BOX 7-2