Measuring the Outcomes of Leadership Development Programs

Beginning of article

The lack of research evaluating the outcomes of leadership development programs and the lack of a suitable evaluation instrument are evident in the literature. This study represents the first attempt at providing a comprehensive method to evaluate and measure leadership development programs on a post-program level. Social learning theory, adult learning theory, and the EvaluLEAD framework influenced the theoretical model developed in this research. The EvaluLEAD principles provide a basis for the conceptual model and results in the development of a program evaluation instrtkment named the Leadership Program Outcomes Measure. Finally, the application of this measure to one statewide leadership development program is presented.

Keywords: leadership program evaluation; leadership; outcomes; instrument; evaluation; adult learning

The foundation of leadership development programs began in 1983 with the vision of the W. K. Kellogg Foundation (WKKF). The WKKF (2001) funded the creation of the first organized statewide leadership development programs to exist in the United States. These programs were geared toward participants primarily from rural areas. They presently extend to 32 states and three countries, lay claim to thousands of alumni, and receive an immense amount of stakeholder support. However, the lack of research evaluating the outcomes of leadership development programs and the lack of a suitable evaluation instrument are evident in the literature.

This dearth of evaluation mechanisms has not gone unnoticed in the research community. Pointing to this serious shortage of rigorous, systematic evidence in program evaluation, Carman (2007) and the WKKF (2001) have called for an increase in leadership development program evaluation.

This study, Measuring the Outcomes of Leadership Development Programs, sought to assist in leadership program evaluation through the creation of an instrument that measured program outcomes on the individual, organizational, and community levels. We used the early WKKF model to measure the outcomes of one statewide leadership development program created in 1985, and we built on an evaluation framework called EvaluLEAD proposed by Grove, Kibel, and Haas (2005). The results of this study provide the first examination of the effect of a leadership development program at the post-program evaluation level. The study employed a comprehensive instrument called the Leadership Program Outcomes Measure (LPOM). The LPOM was developed by Black (2006) to gain insight into alumni outcomes and program achievements. It has important connotations for those who manage leadership development programs and who wish to evaluate post-program outcomes.

Carter and Rudd (2000) suggest that the two primary goals of early leadership development programs were to develop leadership skills in the participants and to enhance participants' knowledge of topics. Social learning theory (SLT) of Bandura (1986), adult learning theory (Birkenholz, 1999; Caffarella, 2002; Knowles, 1984; Lieb, 1991), and Rost's (1993) leadership paradigm influenced the theoretical model that was developed in our research for evaluating leadership development programs. The researchers acknowledge the existence of other exemplary leadership research; however, we believe that these theories were best suited to capturing leadership program development outcomes. The proposed model (see Figure 1) attempts to capture the elements relating to participants of leadership programs, which in turn leads to a theory-driven evaluation approach (Bledsoe & Graham, 2005).

This study is designed to address the dearth of evaluation methods available to those who plan and administer leadership development programs. There are relatively few published studies designed to measure the level of change that a participant experiences from his or her leadership program experience and to what degree this change radiates from the participant to the community in which he or she interacts. This article will provide an example of research conducted with a new instrument designed to address this gap.

[FIGURE 1 OMITTED]

We add to the literature by seeking to capture the level of change on personal, business, and community levels experienced by the participants. The study combines quantitative measures of degree of change ratings with qualitative measures to triangulate the data and provide a higher degree of reliability and validity to the instrument. The instrument resulting from this research is applicable to publicly or privately sponsored leadership development programs seeking to evaluate the program's effect. The findings from this research add to the understanding of the effect of leadership development programs post-program and present a novel approach to leadership program evaluation.

Literature Review

Social Learning Theory

As researchers, we often heard from participants of leadership development programs that the "group" influenced personal growth. Bandura (1977) was the first theorist to develop the concept of "imitation" as modeling behavior, where individuals learn from one another by observing behaviors and imitating them.

This process, called social learning theory or "observational learning," was first identified by Bandura (1977, 1986), who points out that the observation of others can help individuals learn from example. According to Bandura, needless errors are eliminated when individuals learn by observing others and then thinking about their actions before performing them. In SLT, modeling behaviors assists the individual's learning through exposure to guides; this is a process that Bandura calls "informative learning." In addition, a person can learn a behavior but may wait until a later time to display that behavior. Bandura proposed that a person's thought processes affect his or her behavior when coupled with exposure to social experiences. These observations and experiences are then drawn on to establish new patterns of behavior that often go beyond those of the exposed levels (Bandura, 1986).

Bandura (1986) found that individuals change because the skills needed to be effective in their efforts to bring about change are demonstrated. He notes that, by empowering people with creative mechanisms, people can exercise influence in areas of their life. Thus, individuals can be empowered with the ability to exercise influence in areas of their lives through social experience and modeling. This modeling helps an individual develop the belief that she or he, too, can accomplish what someone else is observed accomplishing (McGowan, 1986).

"Through modeling we can transmit skills, attitudes, values, and emotional proclivities" (Bandura, 1986, p. 5). SLT argues that individuals should interact and exhibit behavior that enhances self, ability, and role performance.

Based on this theory, a significant difference in activities should be identified in participants of leadership development programs. SLT contends that individuals should interact and exhibit behavior, which enhances self and ability. Therefore, participants in leadership development programs should exhibit behaviors that indicate increased ability in role performance, increased involvement in community activities, and increased perceptions of reality that make them more aware of cultural differences.

Adult Learning

Another area of leadership program planning and delivery occurs through the application of adult learning theories, or "andragogy" (Caffarella, 2002). Knowles (1984), the pioneer of the principles of andragogy, theorized that adults learn experientially and use a problem-solving approach to their pursuit of knowledge. Furthermore, for a program to be successful, adults need to be informed as to why they need to learn and why a topic is of value to them (Caffarella, 2002).

Birkenholz (1999) speculates that adult learning takes place because the individual is motivated to learn. Then, the individual self-selects his or her learning experience because of this motivation. For adults to learn, motivating factors may occur on several levels: (a) to fulfill expectations for oneself or others, (b) to improve one's ability to serve one's community, or (c) for professional advancement.

Finally, adults learn best by interaction through hands-on experiences related first to their knowledge. Lieb (1991) then links Bandura's (1977) SLT to adult learning theory by explaining that adults are motivated by social relationships and the need for associations and friendships. Furthermore, Gibson (2004) states that SLT is very evident in adult learning and emphasizes Bandura's (1977) theory related to adult learning in the areas of attention, retention, …