Monte Carlo Experiment

There is not enough information available from the published Monte Carlo design

(Miller et al., 2002a, 2002b) to enable someone to replicate it. However, the

committee did a Monte Carlo experiment that implied quite different results. The

Monte Carlo simulates a study of the relation between the suicide rate and FS/S as

a proxy for gun ownership. Let Z1, Z2, and Z3 denote unobserved independent

standard normal variables, and let

FS = 10 + Z1;

NFS = 6 + Z2;

FS/S = FS/(FS + NFS);

POP = 50 + Z3; and

RATE = (FS +NFS)/POP,

where FS is the number of firearm suicides, NFS is the number of nonfirearm

suicides, POP is the population size, and RATE is the total suicide rate for the

population. With 1,000 replications, this design gave a mean value of FS/S in the

neighborhood of 0.6 (similar to the fraction of suicides currently committed with a

firearm in the United States). The correlation coefficient of FS/S and RATE was

–0.29. The linear regression of RATE on FS/S gave a slope coefficient of –0.18

with a t-statistic of 9.6. So, according to this simulation, there is a negative association

between the suicide rate and FS/S. In other words, if FS/S is a good proxy for

ownership, gun owners are less likely than nonowners to commit suicide.

obvious why the simulation is at all relevant: the basic finding that proxies

create biases is an analytical result that cannot be resolved by a simulation.

It is very easy to create other plausible simulations that lead to substantial

correlations between FS/S and suicide and, more importantly, substantial

biases in the estimated relations of interest.

In Box 7-2, for example, we present the results of a simulation conducted

by the committee. In this Monte Carlo simulation, we study the relation

between the suicide rate and FS/S as a proxy for gun ownership, but we derive

very different results than those reported by Miller et al. (2002a, 2002c). In

particular, we find a negative association between the suicide rate and FS/S:

in this simulation, if FS/S is a good proxy for ownership, gun owners are less

likely than nonowners to commit suicide.

This exercise illustrates at least two things: (1) the design of the Monte

Carlo simulation matters and (2) having suicide-related variables on both

sides of the regression can produce perverse results. In the end, the biases

created by proxy measures are application specific. Duggan (2003), for example,

highlights the potential problems caused by using FS/S as an explanatory

variable in a model whose dependent variable is also suicide-related. As

demonstrated in the simulation above, unobserved factors associated with

the measure of gun and nongun suicide (e.g., measurement error) may lead to

purely spurious correlations between suicide and FS/S. Since suicide, S, is on

both sides of the estimated equation, the implicit model is often a complicated,

nonlinear relation between S and FS, not the linear model that is

assumed in the literature. These issues may or may not be problematic when

using FS/S to estimate the relationship between gun ownership and homicide.

Another important issue is how the proxy affects inference from specific

models that may include other explanatory variables. This depends,

among other things, on how true firearms prevalence and FS/S are related

to the other observed and unobserved explanatory variables. These issues

are complicated, and most of them have not been recognized, much less

investigated, in the suicide and firearms literature.