The following discussion is based on table I above and ( Gopher and Donchin 1986; O'Donnell and Eggemeier 1986; and Wierwille and Eggemeier 1993). A striking feature of the table is the massive sensitivity of the subjective measures to discriminate significant variations in the mental workload as compared to the other measures. Indeed the subjective measures seem to capture a valid, general aspect of mental workload. The physiological measures on the other hand appear to have high diagnosticity, i.e. they discriminate mental workload imposed on different resources. There are, however, many more facets to it:
The subjective measures excel in face validity as they depend directly on the subjects' actual experience. An advantage is that they are easy to obtain. However, the subjective measures shown in the table do not identify particular constituents of the mental workload (low diagnosticity), be they cognitive or physiological.
Secondary task measures are relatively easy to obtain. In contrast to the subjective measures, the mental workload can be measured concurrently with the primary task. A drawback is their potential intrusion in the primary task. Selecting and interpreting a secondary task measure can be difficult, as the interplay between primary task resources and secondary task resources is subtle.
The physiological measures appear to have high diagnosticity. As seen in the table, heart rate mean discriminates between levels of a psychomotor task, but not any of the other tasks. However, physiological measures are in general sensitive to combined mental workload and motor demands. It is therefore important to keep the physical workload constant, if physiological measures are to reflect only the mental component. An advantage of physiological measures is that they provide for a continuous record of the workload, which allows for identification of local variations.
All types of measures (subjective, secondary task and physiological) appear to reflect mental workload. They all have advantages and drawbacks. It is therefore recommended to include all three types of measures for evaluation of mental workload in IT-work -- and of course also the primary task measures that have been excluded from the present description as they do not compare across tasks.
Casali, J. G. & Wierwille, W. W. ( 1983): A comparison of rating scale, secondary-task, physiological, and primary-task workload estimation techniques