This article describes the process used to identify students in and alumni of an instructional design master's and doctoral program in order to evaluate the effectiveness of this program. Two databases were created on these two groups and then later used to develop to datasheet surveys, which were the Survey of Students and the Survey of Alumni. The Student survey was distributed during the new student orientation and via the program's listserv in fall and spring semesters. The Alumni survey was distributed via the US postal service and, again, through the program's listserv during spring semester only. Of the 225 possible participants, approximately 25% of the surveys were returned and used in the data analyses. Results indicated an overall program satisfaction from both students and alumni. Reported weaknesses were used in strategic program planning to implement changes to the curriculum.
The purpose of the project was to establish databases of the students and alumni and then survey them for their impressions of the Program. The information gathered was used to identify issues related to instructional design (ID) master's and doctoral level programs at a Southeastern university. Additionally, the information gathered from these same audiences assisted in evaluating the programs and facilitating strategic planning by the program faculty members. Ultimately, the information gained will serve as a benchmark in order to identify any trends with the student populations and within the ID programs themselves.
The purpose of this article is to describe the process used to track students and alumni in a less than accommodating environment at the time the databases were established. In addition the results of the information gathered will be shared in light of identifying the types of information to collect when developing databases, the types of reports that facilitate program evaluation and strategic planning, and the types of issues, in general, that were found for our particular programs. Specific details will only be shared when relevant to developing, reporting, and affecting program planning and evaluation because while relevant to our particular ID program, the details may be less valuable to others.
Why gather information on the students and alumni?
Accreditation of Universities, Colleges, Programs, and Professionals
Most accreditation bodies of higher education institutions and programs require that programs assess their effectiveness. These accreditation processes often require self-study of individual programs as well as the institution in and of itself. Part of this self-assessment is based on information about the students and those who have graduated from the program. For instance, this University has just completed a 3-year review by its regional college accrediting agency, Southern Association of Colleges and Schools (SACS) and the College of Education (COE) in the University is currently undergoing a review by the National Council for Accreditation of Teacher Education (NCATE).
SACS is the regional accrediting agency for 11 states in the Southeastern U.S. and in Latin America and is one of six such agencies within the U.S. that is recognized by the U.S. Department of Education (SACS, 2004). NCATE is a national accrediting body for schools, colleges, and departments of education authorized by the U.S. Department of Education. NCATE determines which schools, colleges, and departments of education meet rigorous national standards in preparing teachers and other school specialists for the classroom (NCATE, 2004).
Additionally, professional associations or organizations may review programs within colleges and universities for accreditation and based on standards established by the profession, field or discipline. Such professional accreditations may credential or certify a college, program, and its graduates as meeting their established Standards. For example, the American Bar Association and its state affiliations credential law schools and recognize the graduates as members once they pass the exam. For the instructional design profession, the professional group, International Board of Standards for Training, Performance, and Instruction (IBSTPI) has established competencies for designated professions (Richey, Fields, & Foxon, 2001); however, they have yet to establish procedures for accrediting programs or the individual profession (Davidson, 1985; 1987; Davidson-Shivers & Barrington, in preparation; Gustafson, 2002).
No matter which type of accreditation agency is involved, accreditation studies require that faculty and administrators document the effectiveness (or lack of) with data. For instance, regional college accreditation agencies requires information on graduate program effectiveness in specific areas, such as currency of curriculum, innovation of teaching, advising and orientation of students. Additionally, it requires that programs be able to assess progress of students (2001 Institutional Self-Study; 2001 ID Graduate Programs Report for Self-Study). Such specific information requires data on and from the students and alumni in addition to faculty.
Providing data for accreditation is one reason for obtaining data and keeping records of student progress. Another is to monitor students as they progress …