Real-Time Assessment of Mental Workload Using Psychophysiological Measures and Artificial Neural Networks

Article excerpt


Modern complex systems can place very high cognitive demands upon their operators. The rate of information flow, the complex nature of this information, and the number and rate of required decisions can overwhelm the human operator. At the other end of the continuum, automation of tasks can lead to operator complacency and errors of inattention (Billings, 1997). However, current systems are capable of modifying themselves to meet the momentary needs of the operator. This includes assuming some task functions until the operator's mental load is reduced. In other cases, systems can adjust to improve the operator's awareness to relieve boredom or inattention. Adaptive aiding based on the current functional state of the operator can be most beneficial when supplied at the appropriate time and with the consent of the operator (Rouse, 1988). Further, accurate assessment of operator functional state is required in the test and evaluation of new and modified systems (Charlton & O'Brien, 2002). In these situations the critical factor is the accurate and reliable assessment of the operator's functional state. The functional state of an operator is defined as his of her ability to carry out the job at that moment in time.

One method of monitoring operator functional state is by examining the operator's physiology. The various physiological measures provide unique information about several aspects of operator state. Eye blink rate contains valuable information with regard to the visual demands of tasks. Heart rate is useful to determine the operator's global response to task demands (Wilson & Eggemeier, 1991). The electroencephalogram (EEG) provides useful information about both high workload and inattention (Gundel & Wilson, 1992; Kramer, 1991; Sterman & Mann, 1995; Wilson & Eggemeier, 1991). EEG measures have been used to classify patients with regard to types of neuropathy and psychiatric disorders using linear statistical techniques (John, Pricep, Fridman, & Easton, 1988) and artificial neural networks (ANNs; Kloppel, 1994). EEG has also been used to classify drug effects and to detect alcohol intoxification and fatigue (Gevins & Smith, 1999; Herrmann, 1982). Physiological signals are always present and can be unobtrusively collected and, thereby, are able to provide uninterrupted information about operator state (Wilson, 2001, 2002).

Several studies have used psychophysiological measures to classify operator state with regard to mental workload. Most of these studies have employed EEG, cardiac, and eye data. Several of these studies used either simple, single-task paradigms (Gevins et al., 1998; Gevins & Smith, 1999; Nikolaev, Ivanitskii, & Ivanitskii, 1998; Wilson & Fisher, 1995) of relatively few peripheral nervous system variables in the context of complex task performance (Wilson & Fisher, 1991). Others have used complex tasks with skilled operators (Russell & Wilson, 1998; Russell, Wilson, & Monett, 1996; Wilson & Russell, 2003). These papers report overall successful task classification in the 80% to 90% correct range. The success rate of correctly classifying high mental workload or altered operator state is very encouraging. This suggests that these methods could be used to provide accurate and reliable operator functional state assessment during test and evaluation and to implement adaptive aiding systems. Hilburn, Jorna, Byrne, and Parasuraman (1997) used psychophysiological measures to show that adaptive aiding controlled by the task demands of their air traffic control task reduced mental workload. This demonstrates that psychophysiological measures of operator functional state change to show reduced mental workload when adaptive aiding is applied.

Psychophysiological measures have also been used to implement adaptive aiding in laboratory situations designed to detect lowered operator engagement in the task being performed (Freeman, Mikulka, Prinzel, & Scerbo, 1999; Freeman, Mikulka, Scerbo, Prinzel & Clouatre, 2000; Pope, Bogart, & Bartolome, 1995; Prinzel, Scerbo, Freeman, & Mikulka, 1995). …