The employee assistance program (CAP) at Federal Occupational HeaLth (FOH) gathered outcomes data from almost 60,000 clients during the three-year period 1999 2002. Measurement of outcomes at pre- and post-EAP use was incorporated into fine standard clinical process for all clients. Outcomes included (1) work productivity as affected by the client's emotional problems, (2) productivity as affected by the client's physical health, (3) the interference of physical or emotional issues on work and social relationships, (4) perceived health status, (5) job attendance/tardiness, and (6) global assessment of functioning (GAF).
Results found statistically significant improvement from pre- to post-EAP intervention for all six measures. Unplanned job absence and tardiness in the previous 30 days decreased by an average of 1.5 days pet case, and line average GAF rose by 10 percent. Outcomes improvements were evident to a similar degree across clients with different kinds of assessed problems.
Every sector of the healthcare profession must assess and evaluate whether it is providing the best care possible and producing optimal client outcomes, client satisfaction, and return on investment. The healthcare community continues to focus on outcomes measures as a means of ensuring quality of service and demonstrating value. For example, hospital and medical surgery clinic "report cards" are becoming more available to the general public; many are on independent Web sites that compare similar procedures.
In contrast, the behavioral healthcare field generally does not collect large-scale outcomes data or make such data available. Perhaps this is due to the multiple variables and difficulties in the collection, study, and reporting of outcomes information in a naturalistic environment such as an employee assistance program (EAP). These limitations, however, should not deter EA professionals from sharing outcomes evaluation methodology and data.
Federal Occupational Health (FOH), a service unit within the U.S. Department of Health and Human Services' Program Support Center, has almost 60 years of occupational health experience. Its mission is to work in partnership with its federal agency customers to deliver comprehensive occupational health services to improve the health, safety, and productivity of the federal and military workforces. Although a federal agency, FOH competes in the market for federal agency business.
FOH's EAP provides services to more than 3.3 million federal employees and family members. In an effort to evaluate and improve services and outcomes, the FOH EAP routinely assesses client health status and outcomes before and alter the use of the service.
FOH uses health status measures recommended by the Health Outcomes Institute and InterStudy (1992), which advocate assessing, tracking, and analyzing outcomes as a result of health treatment interventions. Review of client outcomes and client satisfaction responses are two of several data-driven quality monitoring processes used by FOH to expand its understanding of program strengths and weaknesses.
In evaluating client outcomes information, FOH's goals are to learn the extent to which clients show improvement, are satisfied with services received, and report increased productivity after using the EAP as well as to gain a better understanding of what activities lead to improved outcomes. The data also allow FOH'S EAP counselors to know each client's health status, available social supports, and individual situation so they can help provide better, more focused care.
This study updates and replicates an earlier FOH study that discussed aggregated health status and outcomes for clients of FOH's EAP (Selvik and Bingaman 1998). The earlier study used the same measures as the current study; but on a smaller sample and over a shorter period of time. This is a large-scale study representing a diverse range of EAP clients from across the nation over a three-year period.
The goal of the current study was to explore the extent of EAP client improvement in major outcome areas relevant to workplace performance and overall health and functioning. Data were collected over a three-year period using a standardized procedure that involved validated self report instruments and counselor-assessed measures.' The study is of single-group longitudinal design, with data collected before and after EAP use There was no control group of employees with personal/work problems who did not use the EAP.
The FOH EAP serves employees and family members of 400-plus federal agencies representing a variety of occupations During the three-year period from July 1, 1999, to June 30, 2002, the FOH EAP closed a total of 116,197 cases. The results presented here are from the 59,685 cases (51 percent) with complete data on both the pre- and post-intervention assessments.
The sample sizes for specific analyses differ because counselors determined some questions were inappropriate or because clients declined or were not available to answer certain questions. When these analyses were compared with the full sample, they were found to have similar demographic profiles.
Approximately 85 percent of FOH EAP clients during the study period were federal or military employees, while 15 percent were family members. The typical client was between 45 and 54 years of age (35 percent), non-veteran (70 percent), and married (50 percent). Two to three were Caucasian, 20 percent were African American, and 10 percent were Hispanic. Nearly 40 percent of employee clients had worked for their current employer between 6 and 15 years; 8 percent of employee clients were supervisors.
Approximately three in five clients who contacted the EAP did so on their own initiative, while roughly 15 percent were referred by management and 2 percent by unions. Family members, supervisors, co-workers, and health unit staff also encouraged employees to contact the EAP. About 3 percent of clients who contacted the EAP in an emergency situation needed immediate assistance. Clients were nearly equally split between males and females.
Nearly half of the clients reported that the problem(s) that brought them to the EAP affected their work performance. Roughly 15 percent reported work relationship problems with co-workers or supervisors, 10 percent reported diminished work performance; 10 percent reported absenteeism/tardiness problems at work, and 5 percent reported misconduct/disruptive behavioral problems. Some clients reported an occupational health problem or said a safety violation or accident had occurred. One in three clients cited job-related concerns as their presenting problem(s), while another third cited marriage/relationship/family issues.
EAP counselors also assessed each client's primary problem(s). Counselors reported that about 60 percent of clients had mental health problems (such as depression or anxiety), 15 percent had family/relationship issues, 15 percent had job issues, and about 10 percent had alcohol or drug problems.
Resolution and Referral
The typical EAP client held between three and four meetings with the EAP counselor. Four of five clients were able to resolve their problem(s) within the EAP; the remainder were referred to a variety of other providers, such as formal treatment programs, community social services agencies, medical care services, or self-help resources, for ongoing care. The EAP tracks referred clients for up to a year or as clinically relevant.
At the end of the final EAP session, each client receives a client satisfaction survey to complete and mail to the main EAP office for tabulation. Clients consistently rate themselves "very satisfied" or "satisfied" with the EAP at least 90 percent of the time. In addition, 98 percent of survey respondents report they would recommend the EAP to other employees.
During the three-year period, clients called a toll-free telephone number to access the EAP and were automatically routed to a counselor in their location. Clients met with EAP staff counselors and/or network affiliate counselors. The majority of clients met with counselors in face-to-face sessions, although a small percentage received telephone counseling under a structured protocol (Stephenson et al. 2004). EAP counselors were responsible for collecting outcomes data from clients. The typical time period between case opening and closing was 45 to 60 days.
Of the six outcomes measures, two gauged work productivity, one gauged relationships, one gauged absenteeism, one gauged perceived health status, and one gauged global functioning. The first four items are from the HSQ-12 measure (Health Outcomes Institute and InterStudy, 1992).
Productivity as Affected by Mental Problems. This item asked, "During the past four weeks, to what extent have you accomplished less than you would like in your work or other daily activities as a result of emotional problems (such as feeling depressed or anxious)?" The response options were (1) None at all, (2) Slightly, (3) Moderately, (4) Quite a bit, and (5) Extremely.
Productivity as Affected by Physical Health. This item asked, "During the past four weeks, how much difficulty did yon have doing your work or other regular daily activities as a result of your physical health?" The response options were (1) None at all, (2) A little bit, (3) Moderately, (4) Quite a bit, and (5) Could not do daily work.
Social Relationships. This item asked, "During the past four weeks, to what extent has your physical or emotional problem interfered with your normal social activities with family, friends, neighbors, or groups?" The response options were (1) None at all, (2) Slightly, (3) Moderately, (4) Quite a bit, and (5) Extremely.
Health Status. This item asked, "In general, would you say your health is ...". The response options were (1) Excellent, (2) Very good, (3) Good, (4) Fair, and (5) Poor.
Absenteeism/Tardiness. This item was developed by FOH staff and is similar to items used in other workplace studies. It asked, "How many days have you been unexpectedly absent or tardy in the past 30 days?"
Global Assessed Functioning. The EAP counselors assessed clients' level of functioning using the Global Assessment of Functioning (GAF) scale from the American Psychiatric Association. GAF scores can range from I to 100, with higher scores indicating better functioning.
The study results are organized into three sections. The first section addresses the change in pre- to post EAP use measures in the total sample; the second section examines the extent of improvement in each outcome area; the third section explores the extent of outcome improvement by clients with different types of problems.
Change in Outcomes from Case Opening to Case Closing
Univarite paired t-tests were run on the six outcome measures. Results indicated significant (p-.001) pre- to post- differences for all measures. Thus, each difference was in the expected direction and showed improvements for EAP clients on all measures. Table 1 shows the mean scorns for pre- and post measures and t-test statistics for the four HSQ items with Likert-type rating scales.
Outcome Improvement in Total Sample
Productivity as Affected by Emotional Problems. Figure 1 shows the percentage of clients in each rating level tot this item at case opening and case closing. At case opening, 30 percent of clients reported that in the four weeks before they contacted the EAR they experienced "quite a bit" or "extreme" difficulty in performing their work or other regular daily activities as a result of their emotional problems. At case closing, only 8 percent of clients reported this level of difficult--a 73 percent reduction in the number of cases with low productivity due to mental health issues. The modal response to the question at case opening was 'quite a bit," whereas at case closing it was "none at all."
Productivity as Affected by Physical Health. Figure 2 shows the percentage of clients in each rating level for this item at case opening and case closing. At case opening, 15 percent of clients reported that in the four weeks before they contacted the EAR they experienced "quite a bit" of difficulty or "could not do their daily work" as a result of their physical health. At case closing, only 5 percent of clients reported this level of difficulty, a 66 percent drop in the number of such cases.
Relationships. Figure 3 shows the percentage of clients in each rating level for this item at case opening and case closing. At case opening, 31 percent of clients reported that in the four weeks before they contacted the EAP, their physical or emotional problems interfered "Quite a bit" or "Extremely" with their normal interactions with co-workers, family, or friends. At case closing, only 8 percent of clients reported this level of difficulty, a 74 percent reduction.
Health Status. Figure 4 shows the percentage of clients in each rating level for this item at case opening and case closing. At case opening, 16 percent of clients reported they considered their overall health to be "fair" or "poor"; at case closing, 11 percent of clients reported likewise. This represents a 31 percent decrease in cases with low health status.
Absenteeism/Tardiness. Clients reported at case opening that they had been unexpectedly absent or tardy an average of 2.37 days in the 30 days prior to using the EAP. At case closing, clients reported being unexpectedly absent or tardy an average of 0.91 days in the previous 30 days. This represents a 62 percent drop in average lost time away from work (t = 70.810; df = 59,684) and corresponds to 87,140 avoided whole or partial days missed from work over three years by the 59,685 clients. These results indicate considerable savings by the federal agencies for which the employees work.
Global Assessed Functioning. Average client GAF scores improved from 64.11 to 70.38 from case opening to closing (t = -175.143; df = 47,663). This change reflects an average improvement of 10 percent and equates to moving from a range of mild symptoms and difficulty in functioning to transient, slight symptoms and impairment levels.
Summary. Table 2 summarizes the improvements in outcomes for the entire, population. The number of clients with low or deficit levels of workplace performance and health decreased significantly after using the EAP In addition, the amount of time away from work due to absence or tardiness fell by almost two-thirds. Perceived health status and global functioning also improved, but not as dramatically as suggested by the outcomes in workplace performance areas.
Outcome Improvement by Assessed Problem Type
Table 3 summarizes the previous outcomes variables, showing percentage of improvement by clients with different assessed problems. All client types tend to have a similar outcome improvement profile. Although statistical tests were not performed to provide a direct comparison of the magnitude of the group differences, it is interesting to examine the pattern of difference between client types. EAP clients assessed to have emotional problems (such as stress, anxiety, and depression) tended to have the highest level of outcome improvement, while those with financial or legal problems improved to a slightly lesser extent than those with other problems.
The findings of this investigation compare positively with the findings of our earlier study (Selvik and Bingaman 1998) and with other studies of EAP outcomes. This study recorded an average 1.46 pre-/post-EAP reduction in workdays with unscheduled absenteeism/tardiness across all clients. The 1998 FOH study, by comparison, found a 1.18 pre-/post EAP reduction in workdays with unscheduled absenteeism/tardiness.
Several studies show a positive relationship between EAP counseling and reduced work absenteeism. Attridge (2001) found a pre-/post-EAP absenteeism difference of 2.0 days for the 62 percent of cases with avoided work loss after EAP intervention. In a similar study, Attridge (2001) found a pre-/post-EAP absenteeism difference of 2.1 days for the 60 percent of cases with post-EAP avoided work loss.
Recent health and productivity management literature approaches the issue of mental health-related workplace absenteeism from another perspective. The American Journal of Psychiatry (as reported in the Employee Assistance Program Management Letter of February 2004) reports that workplace absenteeism is twice as high for depressed workers compared with those without depressive symptoms.
Harris et al. (2002) gathered pre-/post-EAP outcomes data from 83 clients over a 10-month period. As with the EAP counselors in this study, the researchers in the Harris investigation utilized questions from the Health Status Questionnaire to obtain clients' self-report scores on their general and emotional health status. Using a five-point Likert-type scale, the researchers found average pre-/post-EAP improvements as follows: a rise of 0.17 points in general health perception; a decrease of 0.53 points in impact of emotional problems on social activities; and a decrease of 0.66 points in impact of emotional problems on daily activities. On a 10-point scale, they found a positive pre-/post-EAP difference of 0.34 on health perception.
Although there are concerns about the validity of self-report data, research by Kessler et al. (2003) supports the validity and usefulness of self-report data when compared against archival work performance measures, payroll records, and experience sample method evaluations of moment-to-moment work experience (validation of real-time activities). Also, information relative to the internal states of people and to their perceptions of their own functioning and productivity can only be gleaned from the individuals themselves.
As Figure 4 shows, 16 percent of FOH's EAP clients were in the two lowest health status categories (fair or poor) at case opening. At case closing, this percentage had dropped to 11 percent. Ware (1997) reports that the incidence of health-related job loss is about 10 times higher for people in the bottom one-fourth of tire health status scale than for those in the top one-fourth. Extrapolating this information to FOH's findings, FOH clients experienced a 31 percent reduction in possible health-related job loss costs.
Connecting health status outcomes with increased productivity and decreased job loss may offer work organizations evidence of the benefits of EAPs, both to clients and the financial bottom line. By collecting health and productivity outcome measures, FOH's EAP demonstrates the value and quality of EAP services. We routinely report our findings in utilization reports to our customer agencies.
As mentioned above, the development of outcome and client satisfaction measures in healthcare is being prompted by practical questions from employees, family members, employers, and purchasers of healthcare services. To answer these questions, the FOH EAP is also evaluating health status results against client satisfaction results, clinical assessment information, demographic data, and interventions used. Straightforward, ongoing measurement of clients' health status before and after treatment, along with assessment and intervention information, will help identify the value and benefits of various EAP counseling interventions.
Notwithstanding differences in clients and counselors, it is of interest that the health and productivity outcome data reported here are consistent with those reported by FOH in 1998. Given the large number of clients involved in this outcome analysis, the consistency of findings over the years, and the routine gathering of pre- and post-EAP status data of all clients in the EAP care process, the information reported here adds significantly to the body of literature supporting the health and productivity benefits of EAPs.
Table 1 Pre- and Post-EAP Mean Scores on Outcome Measures Outcome Type Pre-EAP mean Post-EAP t-test * Description ** (SD) mean (SD) Productivity: 2.72 (1.18) 1.88 (.95) 181.69 Lower scores Emotional mean fewer Problems problems Productivity: 2.01 (1.17) 1.63 (.94) 90.13 Lower scores Physical mean fewer Health problems Work and 2.75 (1.18) 1.89 (.97) 182.85 Lower scores Social mean fewer Relationships problems General 2.55 (.99) 2.37 (.94) 56.55 Lower scores Health Status mean better health status perception Note: N= 59,684 * All paired t-test significant at p < .001 ** All measures have score range from 1 to 5. Table 2 Summary of EAP Client Improvement Outcome Area Definition of Before After Raw Percent Deficit Level EAP EAP Change Improve- ment Productivity Rating of 30% 8% 22 73% Affected by "extreme Mental Health difficulty" or "quite a bit of difficulty" in performing work Productivity Rating of 15% 5% 10 66% Affected by "could not do Physical Health work" or "quite a bit of difficulty" in performing work Work and Rating of 31% 8% 23 74% Social "extreme Relationships interference" or "quite a bit of interference" with normal interactions General Health Rating of 16% 11% 5 31% Status "poor" or "fair" Absent from Mean number of 2.37 0.91 1.46 62% Work or Tardy days in past 30 (5.09) (3.63) days Global Mean score on 64.11 70.38 6.27 10% Assessed 1-100 scale (9.02) (9.74) Functioning Note: N = 59,685 for all measures except GAF (N = 47,664). Numbers in parentheses for Days Absent and GAF are standard deviation scores. Table 3 Improved Outcomes by Assessed Problem Outcome Focus Job Marriage/ Alcohol/ Family Emo- Legal/ Percent Showing Relation- Drug tional Finan- Improvement ship cial Productivity: 68% 66% 66% 65% 70% 64% Mental Health Productivity: 55% 57% 58% 56% 61% 52% Physical Health Work and Social 67% 67% 67% 64% 71% 55% Relationships Health Status 23% 25% 29% 24% 29% 22% Attendance 79% 83% 80% 85% 83% 79% Global 67% 73% 70% 68% 78% 61% Assessment of Functioning Clients who indicated no problems at case opening (and thus had no room for improvement) are excluded. Total N varies with each category. Figure 1 Productivity as Affected by Emotional Problems Difficulty in Performing Difficulty in Performing Work Before Using EAP Work After USing EAP None at all 19% 42% Slightly 26% 37% Moderately 25% 13% Quite a bit 25% 7% Extremely 5% 1% Note: Table made from bar graph. Figure 2 Productivity as Affected by Physical Health Difficulty in Performing Difficulty in Performing Work Before Using EAP Work After USing EAP None at all 48% 60% A little bit 20% 23% Some 17% 11% Quite a bit 12% 4% Could not perform 3% 1% daily work Note: Table made from bar graph. Figure 3 Work and Social Relationships Effect of Physical or Effect of Physical or Emotional Problems or Emotional Problems on Normal Interactions Normal Interactions Before Using EAP After Using EAP None at all 18% 43% Slightly 25% 36% Moderately 25% 13% Quite a bit 26% 7% Extremely 5% 1% Note: Table made from bar graph. Figure 4 Health Status Self-reported Health Status Self-reported Health Status Before EAP Use After EAP Use Excellent 18% 43% Very good 25% 36% Goad 25% 13% Fair 26% 7% Poor 5% 1% Note: Table made from bar graph.
(1) Kessler, R. 2002. The Harvard Health and Work Performance Initiative. See www.hcp.med.harvard.edu/hpq for information regarding the use of self-report measures in health and productivity studies.
American Psychiatric Association. 2000. Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition. Washington, D.C.: American Psychiatric Association.
Attridge, M. 2001. "Personal and Work Outcomes of Employee Assistance Services." Presented at the American Psychological Association Annual Meeting, San Francisco, Calif.
Attridge, M. 2001. "Outcomes of Telephonic Employee Assistance Services in a National Sample: A Replication Study:" Presented at the American Psychological Society Annual Meeting, Toronto, Ontario, Canada.
Employee Assistance Program Management Letter. 2004. "Absenteeism Twice as High, Decreased Work Performance for Depressed Workers." Employee Assistance Program Management letter XVII, 12 (3).
Harris, S., M. Adams, L. Hill, M. Morgan, and C. Soliz. 2002. "Beyond Customer Satisfaction: A Randomized EAP Outcome Study." Employee Assistance Quarterly 17 (4):53-61.
Health Outcomes Institute and InterStudy. 1992. Outcomes Management Systems.
Kessler, R., C. Barber, A. Beck, R Berglund, P. Cleary, D. McKenas, N. Prink, G. Simon, P. Stang, T. Ustun, and P. Wang. 2003. "The World Health Organization Health and Work Performance Questionnaire (HPQ). Journal of Occupational and Environmental Medicine, 45:156-174.
Selvik, R. and D. Bingaman. 1998. "EAP Outcomes: From the Client's Point of View." EAP Digest. 21-23.
Stephenson, D., D. Bingaman, C. Plaza, R. Selvik, B. Sugden, and C. Ross. 2004. "Implementation and Evaluation of a Formal Telephone Counseling Protocol in an Employee Assistance Program. Work Place Behavioral Health 2 (in process).
Ware, J. 1997. "Health Care Outcomes from the Patient's Point of View." In E. Mullen and J. Magnabosco, eds., Outcomes Measurement in the Human Services: Cross-Cutting Issues and Methods. Washington, D.C.: NASW Press, pp. 44-67.
Rick Selvik is an EAP national consultant for Federal Occupational Health and has 21 years' experience in healthcare. He provides direct consultation to customers on organizational issues, develops continuous quality improvement measures to improve operations, and oversees the production of customer/management reports.
Diane Stephenson is an EAP manager with FOH and a licensed psychologist with expertise in direct clinical practice. She has more than 20 years of experience in management, efficiency and quality review, and organizational analysis.
Chris Plaza has worked for FOH since 1997 as a senior applications systems analyst/programmer responsible for reviewing, analyzing, and maintaining the PAP information system.
Brian Sugden is an EAP national consultant for FOH and licensed clinical psychologist.
The authors would like to thank the nearly 300 PAP counselors who collected the data for this study over a three-year reporting period and recognize them for the tremendous work they do in improving the outcomes of the clients they serve. The authors ask that direct correspondence about this study be sent by mail to Federal Occupational Health, Chicago IL 606015519, by phone to (312) 886-4215, or by e-mail at email@example.com. For further information, visit www.foh.dhhs.gov/outcomes.…