Kwon, Scott, Safranski, and Bae (1997, p. 41) use a cross-sectional research design to assess the effectiveness of gun control laws on firearm related deaths. According to their assessment: "The multivariate statistical regression model suggests that the existence of gun control laws indeed have a deterrent effect on firearm deaths" (p. 41). Although some gun control laws indeed might have a deterrent effect as claimed by Kwon et al., their evidence does not support this conclusion. In fact, the Kwon et al. study suffers from many methodological flaws and this renders any substantive conclusion impossible. Most of what is reported by Kwon et al. can be attributed to methodological artifacts. Furthermore, the authors omit the mention of a large number of published books and articles about the relation between gun control laws and violence.
The authors in this study are simply wrong when they state, "In spite of charged emotional debates and passage of numerous laws and regulations, no empirical studies have been done to evaluate the effectiveness of gun control laws in this country. The debate on the Brady Bill could have been better informed by scientific research" (p. 41). At the time this article was published in the AJES, there were at least forty-five empirical studies that evaluated the impact of gun laws on violent crime, suicide, and gun accidents, the bulk of which suggest that gun laws have no impact on rates of violence (Kleck 1991, pp. 251-255, 302-303, 390-392; Kleck, 1995).
More specifically, Kleck and Patterson (1993) completed a Study that evaluated the effectiveness of nineteen gun laws at both the state and city level; they found that most gun control restrictions have no net effect on violence rates. Criminologists on both sides of the gun control debate have cited the Kleck and Patterson study widely, demonstrating a technically more sophisticated research methodology.
Kwon et al. compare their results to a study conducted by Mauser and Holmes (1992). They then erroneously inform their readers that the Mauser and Holmes study found a deterrent effect for the 1977 Canadian gun law on homicide rates. In fact, Mauser and Holmes (1992) stated in their conclusion, "The results are consistent with the findings of previous studies that the 1977 Canadian firearms legislation did not have a significant effect on homicide" (p. 613).
Which Gun Laws were Evaluated?
The authors claim to examine the effectiveness of gun control laws prior to the passage of the Brady Bill in 1992. Unfortunately, they do not make clear which laws they attempt to evaluate. Instead the authors state, "The purpose of this study is to investigate the effectiveness of laws and regulations prior to the passage of the Brady Bill in 1992. A multivariate statistical technique is proposed to establish the relationship between the number of gun related deaths by states and sets of determinants including state laws and regulations on firearm use" (p. 42).
This passage suggests that all current state gun law statutes prior to the passage of the Brady Bill in 1992 were to be evaluated. As of 1990 there were at least nineteen different types of gun laws in the United States ranging from gun registration requirements to assault weapon and handgun bans (Kleck and Patterson 1993, p. 262). However, Kwon et al. focus on only three types of gun laws: those requiring (1) background checks, (2) licensing requirements, and (3) mandatory waiting periods.
Tests of Statistical Significance
The most serious problem with this work is the authors' disregard for tests of statistical significance. Kwon et al. state, "According to the model, states with gun control laws had almost 3 fewer deaths per 100,000 than states without any such laws. The relationship, however, is not statistically significant" (p. 46). Kwon et al. may have misunderstood what significance tests mean. They found no statistically significant negative relationship between gun laws and firearm-related deaths, but they continually refer to their findings as if they did. In fact, the relationship was not even significant at the more generous .10 level or even the .15 level for that matter. However, Kwon et al. simply ignore these tests and rely on the negative beta coefficient that their regression results produced. They do not seem to appreciate that by chance alone they had a 10 percent chance of obtaining a negative coefficient, given that the coefficient had to be positive or negative, even if the true coefficient was exactly zero. Thus, the notion that states with restrictive gun laws had almost three fewer deaths per 100,000 is completely unfounded and can be attributed mostly to noise.
Unit of Analysis
Kwon et al. use U.S. states as their unit of analysis. This is problematic for two reasons. First, states are arbitrary statistical aggregations, including vastly divergent ecological units (such as large cities and rural fanning areas). Thus; states are more heterogeneous with respect to levels of violence and variables that affect violence rates. Second, the most restrictive gun laws in the United States are at the local level. Thus, residents of a particular state might be subject to very strong gun laws at the city level, but they might face little or no state regulation (Kleck 1993, p. 253). As Kleck (1991) noted, "A state could have one area with high violence rates but little local gun regulation, while the rest of its component areas have moderately low violence rates and severe local gun regulation.... Yet when areas are lumped together in the entire state unit, the high violence areas could dominate the violence measure so much that the state showed a higher-than-average violence rate despite generally severe local gun restrictions. The aggregation would thus conceal a causal effect of gun laws evident at lower levels of aggregation" (Ch. 10, p. 389). Thus, Kwon et al. are unable to measure the degree to which the majority of state residents are subject to restrictive gun control.
Kwon et al. Rely on an article published in Time on December 10, 1993, for information on state gun control laws. The author of the article provided a map of the United States regarding the current state of handgun control. States were divided into three categories: (1) states with no restrictions, (2) states with required waiting periods or background checks but without licensing requirements, and (3) states with both required waiting periods and licensing requirements. Kwon et al. use this map as their sole source of information for the coding of state gun laws.
My experience is that studies that rely on a single source of information are usually vulnerable to serious measurement error. This has been especially true of studies which have evaluated multiple gun laws (Kleck 1993, p. 255). By relying on a popular news periodical like Time, Kwon et al. overlooked the majority of restrictive state gun laws. Instead, Kwon et al. claim that "twenty-four states as of 1990 did not have any gun control laws and regulations while 26 states had some type of regulation" (p. 45). Had the authors searched for a more appropriate source for information on gun laws, such as the State Laws and Published Ordinances - 1990(1994) published by the Bureau of Alcohol, Tobacco and Firearms, they would have discovered that all fifty states in the United States now have some form of gun law. Such laws date back to before 1990.
Problems with Gun Law Indexes
The authors combine all three types of gun laws (waiting periods, background checks, and licensing requirements) into a single gun law index. Therefore, states that are coded I on the gun law variable are subject to at least one of the three gun law restrictions. However, lumping together heterogeneous gun laws without including separate measures of the impact of each specific gun law makes determining the independent effects for each type of gun law impossible. Thus, we are unable to determine which of the three gun laws is effective in reducing violence. It may be that only one of the gun laws was effective in reducing violence while the other two had little or no impact. Other combinations are possible as well.
The authors make a similar mistake when they lump together all forms of gun-related deaths into a single variable, which includes unintentional deaths, homicide, suicide, and those of unknown intent. Combining heterogeneous firearm-related deaths into a single category is problematic because each of these categories are completely distinct phenomena. As with the gun law index, combining firearm-related deaths makes determining the effectiveness of gun laws on homicide, suicide, or gun accidents difficult. Because variables affecting one type of violence might not affect the other types, and some could even have opposing effects on different types, each category requires unique model specification. However, the authors fail to provide these specifications.
By examining firearm-related deaths, Kwon et al. are unable to account for any substitution affects that might have occurred due to gun laws. Thus, Kwon et al. are unable to determine whether the gun laws were able to reduce rates of homicides or suicides. For example, if gun suicides declined as a result of a new gun law, but overall suicide rates did not, then one would conclude that the gun law had no overall impact on reducing suicides. All that has been found is that those who intended to commit suicide might have substituted guns for other lethal methods, such as hanging or carbon monoxide poisoning. Thus, gun laws might affect only the method chosen and not the fatal outcome. Indeed, most of the research supports this notion (see Kleck 1995, p. 26 for a review of the suicide literature). Consequently, by examining firearm-related deaths only, Kwon et al. are unable to assess these important substitution effects.
In summary, the work by Kwon et al. suffers from fundamental methodological drawbacks and no reliable conclusions regarding the impact of gun laws can be drawn from these results. Their research is a step backward from existing research in every respect, which is disappointing because their errors could have been avoided easily had they been more informed about the prior research on this subject.
Kleck, Gary. Point Blank: Guns and Violence in America. New York: Aldine, 1991.
Kleck, Gary, and E. Britt Patterson. "The impact of gun control and gun ownership levels on violence rates." Journal of Quantitative Criminology 9, 1993:249-288.
Kleck, Gary. "Guns and Violence: An Interpretive Review of the Field." Social Pathology 1, 1995:12-45.
Kwon, I-W., B. Scott, S. R. Safranski, and M. Bae. "The Effectiveness of Gun Control Laws: Multivariate Statistical Analysis." American Journal of Economics and Sociology, 56(1), 1997:41-50.
Mauser, Gary, and R. A. Holmes. "An Evaluation of the 1977 Canadian Firearms Legislation." Evaluation Review 16(6), 1992: 603-617.
Time. "Beyond the Brady." Dec. 20, pp. 28-30, 1993.
U.S. Bureau of Alcohol, Tobacco and Firearms. 1994. State Laws and Published ordinances, Firearms - 1990. Washington, DC: U.S. Bureau of Alcohol, Tobacco and Firearms.
[Professor Tomislav V. Kovandzic teaches at the School of Criminology and Criminal Justice at Florida State University, Tallahassee, Florida 32306-2025.]…