Particularly in Europe, academics with research interests in parapsychology are taking up posts within higher education institutions. Naturally these researchers become involved with teaching students and, as a result, many student projects are being conducted under their supervision. This raises the question of the evidential status of such projects. For instance, might they contain unreported flaws? Should they be included in meta-analyses of psi research? Furthermore, as there are far more student projects than there are parapsychologists, does the database of student projects form a significant file-drawer for parapsychology if these projects remain unpublished? To some extent these questions may be answered by taking into consideration the pedagogical context of the student projects: does the supervisor maintain strict control so that the student is essentially a research assistant, or is the student given some freedom to develop his or her scientific curiosity but also to make mistakes (and--we hope-learn from them)? This paper aims to encourage consideration of these important issues for modern-day parapsychology by looking in detail at the undergraduate student projects conducted at the Koestler Parapsychology Unit (KPU).
The KPU is a research group situated within the School of Philosophy, Psychology, and Language Sciences at the University of Edinburgh, Scotland. The KPU initially grew tip around the Koestler Professor of Parapsychology, Robert Morris, (2) who held this position from 1985 until his death in 2004. The KPU consists of teaching staff, research staff, and postgraduate students whose area of interest and expertise is parapsychology and the psychology of anomalous experiences and beliefs. While Bob Morris was Koestler Professor, KPU researchers included (in alphabetical order) Deborah Delanoy, Peter Lamont, Julie Milton, Fiona Steinkamp, Paul Stevens, and Caroline Watt.
KPU members have always played a role in teaching psychology undergraduate and postgraduate students. The undergraduate psychology course at the University of Edinburgh is 4 years long. The single most substantial piece of work that undergraduate students produce is their final-year project and thesis. In this, students are able to apply their methodological and statistical training to topics in which they have a particular interest, under the supervision of staff with relevant expertise. Students can choose their thesis topic from a selection offered by staff. They are also free to approach staff with their own research idea, in which case the staff member may agree to supervise, so the degree to which the project topic is driven by a supervisor's own research agenda varies depending on his or her pedagogical and research goals. Some may exert strict control so that the project addresses a tightly specified question in their research program and is methodologically sound. In this case, the student is essentially acting in the role of research assistant or coinvestigator. Others may prefer to encourage students to think creatively and follow their scientific curiosity. Here the project is less likely to contribute to a systematic research program, and--in the hands of less able students--study quality may suffer. The project is written up as a thesis of around 11,000 words that is graded and contributes towards the student's overall degree grade.
To illustrate the issues raised by student parapsychology projects, this paper presents a descriptive overview of the final-year projects that have been supervised by KPU staff since its inception. These may act as a large number of pilot projects that might help guide future research. As these final-year projects are mostly unpublished, they may also place published studies in a wider context with respect to possible file-drawer effects. The KPU-supervised projects consist of two types: those that include a psi task and those that do not. As space is limited, I will only briefly describe the non-psi projects before going into the psi projects in greater detail.
Every available 4th year thesis that had been supervised by KPU members was retrieved from the Koestler Library or directly from KPU members. Students are permitted to work in pairs (and, rarely, in groups of three or four) for data collection but have to write up their thesis independently, sometimes resulting in two or more theses written on the same group of data. In these cases, the thesis that was retrieved first was taken to represent the joint project.
Four main types of study characteristics were recorded.
The first is the main "demographic characteristics" of the thesis collection. The number of participants gives some indication of the statistical power of each study (which affects the potential of the study to detect a hypothesized genuine experimental effect to a statistically significant degree), and the sex of participants gives an overall picture of the gender balance in these studies. Finding support for preplanned hypotheses indicates the "success" of the project (in terms of finding support for predicted effects and relationships), though of course a successful study in pedagogical terms could have null findings but be well designed, conducted, and reported.
The second concerns the research area of the project, including whether the study includes a psi measure, and a more detailed breakdown of the study topic.
The third provides a fuller picture of the characteristics of the psi studies in particular. One question of interest is whether the psi studies include a prediction of an overall psi effect (e.g., they include a "proof-oriented" hypothesis). In contrast, overall psi effects in process-oriented studies can be "washed out" if the experimental design includes conditions predicted not to be psi-conducive. As student projects often contain exploratory analyses, the extent to which the projects reported psi findings arising from post hoc analyses was also noted.
The fourth study characteristic of interest was whether the project has reached the public domain, either through journal publication or presentation at a Parapsychological Association conference (where details of method and analyses can be found in the conference Proceedings). In no cases was the full thesis published, but a version of the project could be written up for conference presentation or publication. If neither occurred, then the project is essentially in the "file-drawer." KPU student projects are also sometimes presented at the conference of the Society for Psychical Research. However, this conference provides only abstracts, so many methodological and statistical details are unavailable; one could argue that this lack of detail means the projects are essentially "unpublished."
Before I describe the results of this survey, some caveats are in order. First, the information provided here is meant to give a descriptive, not a quantitative, picture of the content of KPU student projects as described by the students themselves in their theses. This is not a recta-analysis because there are some inherent limitations to this database: the projects vary in detail, making it difficult to combine them. Second, although the projects were supervised by experienced members of the teaching and research staff, normally such supervision would not extend beyond advising the student on study design, helping provide facilities, and advising on analysis and write-up. It would therefore be rare for a supervisor to be present during actual data collection or to check that the students had correctly followed the planned procedure. In other words, in terms of the distinction I set up at the outset of this paper between directing research assistants and encouraging budding scientists, I would say that most of the KPU staff have preferred to follow the latter route. Third, time has passed, students and supervisors have departed, and in most cases we no longer have access to the studies' raw data. So we are required to take the study conclusions at face value and to trust that the students have been both truthful and proficient in how they have collected, analyzed, and reported the data. Finally, although the supervisors may have given students feedback on the thesis contents during the writing process, the finalized theses have not been subjected to peer review. …