Examining Prevalence Differences in Three National Surveys of Youth: Impact of Consent Procedures, Mode, and Editing Rules
Fendrich, Michael, Johnson, Timothy P., Journal of Drug Issues
We examined the potential prevalence impact of differences in consent procedures, mode of administration, and editing protocols in the 1997 National Household Survey on Drug Abuse, the Monitoring the Future study, and the Youth Risk Behavior Survey, limiting analyses to 10th and 12th graders. NHSDA's high level of compliance with federal regulations regarding human subjects research, including thorough parental permission prior to adolescent participation, may in part be responsible for the relatively low prevalence rates obtained in this study. Key mode effects which contribute to observed differences across surveys are survey setting and privacy. The two school-based surveys produce higher prevalence estimates than the NHSDA. Although NHSDA prevalence rates increase with increased interview privacy, the increase is not enough to account for discrepancies with the school surveys. The procedures used to handle inconsistent survey responses and to impute missing data are very different across the three surveys. These differences in editing procedures, however, are unlikely to be responsible for the observed differences in prevalence rates. Based on these analyses, we conclude that the field would benefit from a well-designed experimental study evaluating school vs. household effects as well as the impact of variation in consent procedures. Increased methodological research on the validity of school based drug surveys is also needed.
This report compares the impact of mode of administration, consent, and editing procedures on substance use prevalence estimates derived from three studies: Monitoring the Future (MTF), the Youth Risk Behavior Survey (YRBS), and the National Household Survey on Drug Abuse (NHSDA). Two of these studies (MTF and YRBS) are administered in school samples. NHSDA, like its name implies, is administered in the household. This report is restricted to focusing on two aspects which clearly vary across the studies: Consent procedures and survey mode. We also compare the studies with respect to the potential impact that editing procedures may have on prevalence estimates. We will attempt to hypothetically and, where possible, empirically link study variation on these dimensions to prevalence comparisons. Preliminary conclusions about the potential role of these dimensions in prevalence differences will be drawn and suggestions will be made for future research directions. As a point of departure, it is important to evaluate exactly what the differences in rates are across the three surveys. Discussion of specific procedural differences follows a careful delineation of prevalence differences across four substances for the three studies.
SURVEY ESTIMATE DIFFERENCES
Tables 1 and 2 present lifetime and past month (30 day) prevalence rates for four substances across the three studies in 1997, along with appropriate confidence intervals for tenth and twelfth grade, respectively. Comparisons are limited to the 10th and 12th grade, since these are the only grades systematically assessed in both of the school based studies. To establish comparative samples, the NHSDA survey was converted to grade specific subgroups. Although the group of substances selected is by no means exhaustive, for comparison purposes, we chose four substances of clearly varying degrees of sensitivity and prevalence: Two relatively common, high prevalence and legal substances (cigarettes and alcohol) and two illegal substances of moderate and low prevalence, respectively (marijuana and cocaine).
Inspection of the two tables reveals a fairly consistent trend across substances: The NHSDA consistently yields the lowest estimates and the YRBS consistently yields the highest estimates. Inspection of confidence intervals for Table 1 suggests that NHSDA estimates for tenth grade drug use are significantly lower than both MTF and YRBS for all comparisons except one - past month cocaine use. MTF estimates are significantly below YRBS estimates for cigarettes (lifetime and past month), alcohol (lifetime and past month), and marijuana (past month only). …