I was inspired to write this up after reading some of the debate in the
Guns don't kill people; People kill people node.
ccunning rightfully criticizes the use of the Australian data series to infer that decreased gun ownership results in increased violent crime based on the lack of a
control group. The data suggest that violent crime increased dramatically after
gun control was instituted in 1996. However, given that there is no control group,
ccunning claims that it is impossible to separate the effects of gun control from other legal and social factors. This is not strictly the case.
Randomized intervention analysis (RIA) is a statistical procedure developed by an ecologist to deal with the lack of a control group. Often, in ecological studies, an impact or disturbance occurs and researchers are obliged to examine the impacts without the benefit of a control group. In 1989, Stephen Carpenter introduced RIA1 as a potential solution to this problem. Provided the researchers have several observations before and after the disturbance, the test works in the following manner:
- Calculate the mean values of the variables of interest for the pre- and post-disturbance periods. Call these values Dpre and Dpost.
- Calculate |Dpre-Dpost|. This value is the test statistic.
- Permute the data. This means, randomly assign values of the variables of interest to the pre- and post-disturbance periods.
- Recalculate |Dpre-Dpost| for the permuted data.
- Repeat steps 3 and 4 several thousand times.
- Compare the original test statistic against the values produced by permutation. If the test statistic is extreme when compared with these values (say, in the upper 5%), then we can conclude to a relative certainty that the disturbance had an impact.
RIA can easily be extended to control for
confounding factors.
RIA could easily be applied to data concerning things other than environmental data. For example, if we want to know whether gun control has resulted in increased crime rates in Australia, RIA may be a suitable statistical test. However, the test's power is determined by the number of observations. As such, we will have to wait for a number of years before applying this test to the Australian data. However, data series from England or Canada could be treated in this way and more scientifically justifiable results will follow.
1 Carpenter, S. R., T. M. Frost, D. Heisey and T. K. Kratz. Randomized intervention analysis and the interpretation of whole-ecosystem experiments. Ecology, 70(4): 1142-1152.