Loglinear Models and Student Course Evaluations

Research in Economic Education

In this section, the Journal of Economic Education publishes original theoretical and empirical studies on economic education dealing with the analysis and evaluation of teaching methods, learning, attitudes and interests, material, or processes.

WILLIAM E. BECKER, JR., Section Editor

Mostafa Mehdizadeh

Student assessment of professors and courses and the interpretations of their statistical analyses have long interested many economics educators. Multiple studies on possible factors influencing student evaluation of teaching effectiveness ( Mirus 1973; Dilts 1980; and Seiver 1983) and student evaluation of courses ( Kelley 1972) constitute a rather extensive literature often centering on the common introductory classes. In most of the statistical analyses, the researchers used a single-equation regression to link some aspect of student evaluation to the characteristic variables. However, recent researchers have noted the limitations of these previous works; to overcome methodological deficiencies, they have attempted alternative methods such as simultaneous equation systems ( Seiver 1983) and simultaneous two-stage least squares regressions ( Charkins, O'Tool, Wetzel 1985). A major problem with these previous works is the application of a technique--regression--to ordinal data. Additionally, the normality of the distribution of the dependent variable in these studies is never established. The inconsistency of the regression results has been discussed in detail by Spector and Mazzeo ( 1980 ), Kerr and Park ( 1985 ), and DeCanio ( 1986 ); that is, Spector and Mazzeo show that the use of ordinary least squares (OLS), when the de-

-7-

pendent variable is ordinal, may lead to overestimation of the significance level and incorrect conclusions.

In this article, I would like to introduce the usefulness of loglinear models in the field of economic education. I will first describe loglinear models in nonrigorous, nonmathematical terms and will then apply the loglinear technique to a data set related to the field of economic education--student evaluations--to select the model that best fits the data.

Loglinear analysis of categorical data is by no means a new technique; however, it has not been widely used until recently, especially in the field of economics, because of the unavailability of software. In addition, the number of hierarchical models that can be fitted grows as the number of independent variables increases. Thus, the identification of a well-fitting unsaturated model is not an easy task (an unsaturated model contains fewer than all possible interactions of the independent variables). Any researcher who uses loglinear probability models should consider selection strategies to limit the number of models to be evaluated. Although a best selection strategy does not exist, a priori knowledge about the correlations among the variables would help the researcher.

The most recent studies in economic education have been conscious of the deficiencies of OLS regression and have employed logit or its variations (i.e., probit and multinominal logit). In comparison with loglinear models, logit models are simpler to formulate, but they do not incorporate the most general interaction terms among independent variables. In a sense, logit models disregard the complex structural relationships that might exist among the independent variables. Although it is possible to derive a logit model from the corresponding loglinear model, the reverse is not true (see Agresti 1984 , chap. 6, for more detail). Loglinear models also differ from logit models in another aspect: in loglinear models, there is no need to distinguish between response variables and independent variables.

The concepts of loglinear models may be introduced step by step. The simplest model is a two-dimensional contingency table of y by x ₁.¹ In a log linear model of y by x ₁, the number of cases in each cell of the contingency table can be expressed as functions of y, x ₁, and the interaction of the two, y ∘ x ₁. To obtain a loglinear model, the natural logs of the estimated cell frequencies--rather than the actual counts--are used. In general, using analysis of variance notation, one can express the model for the log of the frequencies in the ith row and jth column as is done here:²

The procedure HILOGLINEAR of the computer package SPSS^Xnot ...