An Examination of Statistical Software Packages for Categorical Data Analysis Using Exact Methods. (Statistical Computing Software Reviews)
Oster, Robert A., The American Statistician
StatXact 5. Available from Cytel Software Corporation, 675 Massachusetts Avenue, Cambridge, MA 02139; phone: (617) 661-2011; fax: (617) 661-4405; E-mail: firstname.lastname@example.org; Web page: http://www.cytel.com. Released 2001. Academic price: $1,195; Commercial price: $1,495.
LogXact 4.1. Available from Cytel Software Corporation, 675 Massachusetts Avenue, Cambridge, MA 02139; phone: (617) 661-2011; fax: (617) 661-4405; E-mail: email@example.com; Web page: http://www.cytel.com. Released 2000. Academic price: $795; Commercial price: $995.
Stata 7. Available from Stata Corporation, 4905 Lakeway Drive, College Station, TX 77845; phone: (800) 782-8272; fax (979) 696-4601; E-mail: firstname.lastname@example.org; Web page: http: //www.stata.com. Released 2001. Academic price: $499; Commercial price: $995.
The use of categorical data analysis and nonparametric statistical methods has become increasingly important in most research fields during recent years. These methods are often used to analyze data from clinical, public health, environmental health, and epidemiology studies. In particular, data collected using surveys, questionnaires, and case record forms must often be analyzed with these methods. Several recent textbooks have been devoted to such statistical techniques (Agresti 1984, 1990, 1996; Hollander and Wolfe 1999; Hosmer and Lemeshow 2000; Lehmann 1998; Sprent and Smeeton 2001).
Most statistical software packages use approximations to perform statistical hypothesis testing for categorical and nonparametric statistical analysis. Unfortunately, conventional approximations do not always work well, especially if the dataset is small, the data are sparse (among different categories), or the data are unbalanced. These approximations usually assume that the test statistic follows a normal or a chi-square distribution. For logistic regression performed using a conventional approach, model results depend on asymptotic maximum likelihood inference. However, software packages using this type of inference may provide incorrect results, or may fail to provide any results at all, particularly when there are too many independent variables relative to the final sample size of the study, or when two or more independent variables are closely related.
Due to large increases in computing power and in capabilities of personal computers during recent years, exact statistical tests and methods can now be used for statistical hypothesis testing and for examining statistical models. In general, an exact statistical test is performed as follows. First, the data are permuted in all possible ways under the null hypothesis that is being tested. Second, the value of the test statistic for each permutation is computed. Finally, the observed value of the test statistic is compared to the permuted distribution of the test statistic; the associated p value tells one how extreme the observed value is when compared to the permuted distribution. An extreme observed value will yield a small p value, leading one to conclude that there is a statistically significant result.
When assumptions on the test statistic are met (e.g., when the test statistic follows a normal or chi-square distribution), the exact test and the corresponding approximate test will provide similar results, and should lead one to make the same conclusions. However, when assumptions on the test statistic are not met, the exact test should be used since the approximate test may provide invalid results.
In recent years, a few statistical software packages have been developed primarily for the purpose of performing exact statistical tests. In addition, several general-purpose statistical packages now include some exact statistical tests. This article reviews the following statistical software packages: StatXact 5, LogXact 4.1, and selected portions of Stata 7. …