Academic journal article
By Wille, Nora Mph, Dipl-Psych; Blettner, Maria; Lemstra, Mark PhD, PhD
Canadian Journal of Public Health , Vol. 99, No. 5
With great interest we read the recent systematic review of the literature written by Lemstra et al.1 (published in March/April 2008 issue of CJPH) on the association between socio-economic status (SES) and depressed mood as well as anxiety in adolescents. However, we would like to point out several limitations of the paper that we consider important.
First, we wonder whether the reported highly significant heterogeneity between the studies allows the application of a fixed effects model. Obviously, the percentage of variance explained by heterogeneity is high and therefore the calculation of a pooled estimator is not recommended.
A closer look at the included studies reveals many sources of heterogeneity. One example is with regard to the different definitions of exposure (SES). SES is defined according to household income,2,3 according to occupation,4 with respect to indicators of poverty5 or by means of adolescents' ratings of their SES in comparison with their peers.6 Even if studies operationalized SES comparably as household income, they chose different cut-off values in order to define the exposure as well as the reference group.2,3 Unfortunately, the authors did not describe the exposure and reference categories underlying the presented rate ratios in Table II. For example, Bergeron et al. present an Odds Ratio (no Rate Ratio as reported in the paper; OR of 3.72) which refers to high family income as a risk factor and motivates their conclusion "that youths who live in families where the socioeconomic level is average or high might constitute a risk group".2, pg.59 Furthermore the Odds Ratios quoted from the Canadian National Population Health Survey refer exclusively to the increased risk due to age group and gender and are not related to SES.7 Therefore the pooled estimator given in the metaanalysis is not only methodologically incorrect due to present heterogeneity, but also biased due to the inclusion of inappropriate estimates of at least two of the cited papers.
Moreover, the highly inconsistent definition of the outcome is not appropriately discussed by the authors. Even though only one study in the meta-analysis addressed anxiety,2 the pooled estimator is also interpreted with regard to anxiety. Regarding measures of depression, the authors fail to give information on the different cut-off points in the CES-D3,5 or child-reported versus parent-reported symptoms.
We also noted that some results of the published studies are not reported for unknown reasons. For example, the lack of an association between poverty and depressed mood in boys5 or in childreported disorders2 or the significant role of the mother's education2 are neither included in the table nor in the discussion. Sometimes it is unclear how the effect estimate was drawn from the cited publication, and in the case of one study, statistically inappropriate procedures have to be assumed.4
Last but not least, the high heterogeneity of the underlying samples leads to a more general reflection regarding the appropriateness of a meta-analysis. Data from Hammack et al.5 were obtained from an urban low-income African- American adolescent sample with high frequency of depressed mood (47%) and 81% fulfilling at least one indicator of poverty. It seems unreasonable to pool results from such a specific sample with population-based samples, e.g., from Quebec, Canada2 or from the middle of Norway.4 Since the consequences of low SES might be very different depending on the social environment, a pooled estimator including studies from different cultures and countries is problematic. The studies presented in the paper show clearly that evidence on the association between SES and depression exists in large samples (e.g., for 14,500 adolescents in the US3) as well as in smaller samples including specific parts of the population (e.g., 1,704 African-American adolescents from the US5). With regard to the assumed different effects as well as the diverse concepts and operationalizations in the various studies, a qualitative rather than a quantitative approach might have been the better approach in order to discuss the heterogeneous results. …