Monte Carlo Experiment
17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33
34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67
68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84
85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101
102 103 104 105 106 107 108 109 110 111 112 113 114 115
There is not enough information available from the published Monte Carlo design
(Miller et al., 2002a, 2002b) to enable someone to replicate it. However, the
committee did a Monte Carlo experiment that implied quite different results. The
Monte Carlo simulates a study of the relation between the suicide rate and FS/S as
a proxy for gun ownership. Let Z1, Z2, and Z3 denote unobserved independent
standard normal variables, and let
FS = 10 + Z1;
NFS = 6 + Z2;
FS/S = FS/(FS + NFS);
POP = 50 + Z3; and
RATE = (FS +NFS)/POP,
where FS is the number of firearm suicides, NFS is the number of nonfirearm
suicides, POP is the population size, and RATE is the total suicide rate for the
population. With 1,000 replications, this design gave a mean value of FS/S in the
neighborhood of 0.6 (similar to the fraction of suicides currently committed with a
firearm in the United States). The correlation coefficient of FS/S and RATE was
–0.29. The linear regression of RATE on FS/S gave a slope coefficient of –0.18
with a t-statistic of 9.6. So, according to this simulation, there is a negative association
between the suicide rate and FS/S. In other words, if FS/S is a good proxy for
ownership, gun owners are less likely than nonowners to commit suicide.
obvious why the simulation is at all relevant: the basic finding that proxies
create biases is an analytical result that cannot be resolved by a simulation.
It is very easy to create other plausible simulations that lead to substantial
correlations between FS/S and suicide and, more importantly, substantial
biases in the estimated relations of interest.
In Box 7-2, for example, we present the results of a simulation conducted
by the committee. In this Monte Carlo simulation, we study the relation
between the suicide rate and FS/S as a proxy for gun ownership, but we derive
very different results than those reported by Miller et al. (2002a, 2002c). In
particular, we find a negative association between the suicide rate and FS/S:
in this simulation, if FS/S is a good proxy for ownership, gun owners are less
likely than nonowners to commit suicide.
This exercise illustrates at least two things: (1) the design of the Monte
Carlo simulation matters and (2) having suicide-related variables on both
sides of the regression can produce perverse results. In the end, the biases
created by proxy measures are application specific. Duggan (2003), for example,
highlights the potential problems caused by using FS/S as an explanatory
variable in a model whose dependent variable is also suicide-related. As
demonstrated in the simulation above, unobserved factors associated with
the measure of gun and nongun suicide (e.g., measurement error) may lead to
purely spurious correlations between suicide and FS/S. Since suicide, S, is on
both sides of the estimated equation, the implicit model is often a complicated,
nonlinear relation between S and FS, not the linear model that is
assumed in the literature. These issues may or may not be problematic when
using FS/S to estimate the relationship between gun ownership and homicide.
Another important issue is how the proxy affects inference from specific
models that may include other explanatory variables. This depends,
among other things, on how true firearms prevalence and FS/S are related
to the other observed and unobserved explanatory variables. These issues
are complicated, and most of them have not been recognized, much less
investigated, in the suicide and firearms literature.