Time series robustness checks to test the effects of the 1996 Australian firearm law on cause-specific mortality

Published online on June 01, 2018

Abstract

Objectives

Many studies utilize time series methods to identify causal effects without accounting for an underlying time trend. We show that accounting for trends changes the conclusions in the study of Chapman et al. (JAMA, 316(3), 291–299, 2016), who evaluated the impact of the Australian firearm law in 1996. We also introduce a new empirical method that tests whether their empirical strategy can actually identify a causal effect that is also useful for panel analyses.

Methods

We use national data from the Australian Bureau of Statistics, assembled in annual counts of: total firearm deaths, firearm suicides, and firearm homicides. These data are used in an independent re-analysis of the impact of the 1996 Australian firearm law that accounts for underlying stochastic trends. We then estimate a series of artificially created interruptions using interrupted times series analysis in a time frame before 1996, to test for changes in the slope of mortality across several years prior to the actual regulatory changes. This tests whether the empirical model produces effects in years other than the year of the intervention, thereby testing if the results can simply be replicated at random using other interruption years.

Results

Controlling for stochastic trends produces less statistical evidence of the impact of the firearm law on firearm mortality than previously reported by Chapman et al. (JAMA, 316(3), 291–299, 2016). Introducing artificial interruptions in 1990 through 1995 produces statistically significant decreases in all firearm-related mortality measures well above the expected type 1 error. Overall, 19 out of the 36 artificial interruption models we tested were found to be statistically significant, suggesting that the empirical model can be implemented in multiple non-intervention years with results similar to the true 1996 interruption year.

Conclusions

Current evidence showing decreases in firearm mortality after the 1996 Australian national firearm law relies on an empirical model that may have limited ability to identify the true effects of the law.