Academic journal article International Journal of Child and Adolescent Health

Evaluation of a Leadership and Intrapersonal Development Subject for University Students: Experience in Hong Kong

Academic journal article International Journal of Child and Adolescent Health

Evaluation of a Leadership and Intrapersonal Development Subject for University Students: Experience in Hong Kong

Article excerpt

Introduction

Program evaluation is the use of scientific method to measure the implementation process and outcomes of the programs (1). In the field of human services and educational settings, program evaluation aims to improve the program delivery and meet the client's needs and learning outcomes (2). Rosenberg and Brody (3) stated that "evaluation and client feedback are not only necessary for effective service delivery, but also are ethical requirements of the profession" (p. 349). According to Rutman (4), there are two main purposes of program evaluation. First, program evaluation is regarded as a tool for making improvement in the program by collecting the feedback from clients. Second, program evaluation improves the accountability of the program. It serves as an accountability tool to ensure the project in compliance with funding agency guidelines.

Several methods are commonly used in program evaluation, such as quantitative and qualitative approaches, satisfaction studies, needs assessment, cost-benefit analyses, single-subject designs, experimental approaches and models, utilizationfocused evaluation, empowerment and clients' satisfaction evaluation (5). Among these methods, quantitative approach is generally adopted (6-9). Researchers (10-12) argued the importance of using a psychometrically sound instrument to assess the program effectiveness.

Psychometrics refers to the reliability and validity of the measures (13). If an instrument is not psychometrically sound, it fails to reflect the whole picture of an effective program. In other words, the program will "appear to be less effective than it actually is" (p. 219) (14). Royse (1) contented that a valid and reliable instrument generally helps the program implementers understand how the participants perceived the program and what factors contribute to the success of an effective program. Rossi (14) also stated that "a measure that is poorly conceived or chosen can completely undermine the worth of an impact assessment by producing misleading estimates. Only if outcome measures are valid, reliable and appropriately sensitive can impact assessments be regarded as credible" (p. 222). Hence, it is important to develop and use well-developed instruments that are "good indicators of what the programs are attempting to accomplish and psychometrically strong" (p. 272) (1).

In the Western context, there are numerous wellknown assessment tools in higher education to evaluate the feedback of students. For example, Students' Evaluation of Educational Quality (SEEQ) (15, 16), Marsh identified nine dimensions of student's feedback, including teacher enthusiasm, learning, group interaction, individual rapport, breadth of coverage, examinations, workload, assignment or readings and organization. Other popular assessment tools included Instructional Development and Effectiveness Assessment (IDEA) by Cashin (17) and Student Instructional Report (SIR) by Centra (18). In contrast, very few measures are found in the Chinese context, such as the Chinese version of the Students' Evaluation of Educational Quality (SEEQ) (15, 16) and Exemplary Teacher Course Questionnaires (ETCQ) (19).

In response to the need for developing a reliable and valid subjective outcome evaluation tool, Shek (20, 21) developed a subjective outcome evaluation scale (SOES) to tests the three dimensions (i.e., subject content, lecture and instructor) of a program. This scale has been used among Hong Kong secondary schools (11, 22) and university students (20, 23). Results generally showed that the SOES is a sound assessment tool. For example, based on a total of 2,039 questionnaires, Shek (24) found that the SOES is a reliable and valid instrument. This was further supported in another study (25). By using the exploratory factor analysis, the three factors (subject content, lecture and instructor) were extracted from the scale. The findings also demonstrated that the SOES was a sound assessment tool for program evaluation. …

Search by... Author
Show... All Results Primary Sources Peer-reviewed

Oops!

An unknown error has occurred. Please click the button below to reload the page. If the problem persists, please try again in a little while.