It is assumed that variations in ratings of mental health problems reflect primarily individual differences. But such variations may also reflect societal and cultural differences. The relevant questions are: How much variation is due to societal and cultural differences? and Does the respective variation due to societal and cultural differences vary among different conditions? If the variations are largely due to societal and cultural differences, then a particular instrument may be less useful for individual treatment planning within particular societies/cultures as the instrument is not sensitive to individual differences within that society or culture. Arthur Kleinman described this concern over 40 years ago (Kleinman, Reference Kleinman1977). He questioned research that uncritically exported mental health instruments from one society or culture to another. This question highlights the contrast between emic approaches that focus on characteristics of a given culture v. etic approaches that focus on more universal aspects of cultures (Pike, Reference Pike1967). This tension spurred the rise of transcultural psychiatry, the landmark 2001 WHO Report on Mental Health, and the burgeoning field of global mental health (Prince et al., Reference Prince, Patel, Saxena, Maj, Maselko, Phillips and Rahman2007; World Health Organization, 2001).
It is now clear that certain mental disorders affect people across all regions of the world (Steel et al., Reference Steel, Marnane, Iranpour, Chey, Jackson, Patel and Silove2014; WHO World Mental Health Survey Consortium, 2004). Even so, the construct of mental disorder itself is evolving. The Lancet Commission on Global Mental health recently admitted that ‘the binary approach to the diagnosis of mental disorder … does not adequately reflect the dimensional nature of mental health’ (Patel et al., Reference Patel, Saxena, Lund, Thornicroft, Baingana, Bolton and Eaton2018, p. 4) Mental health, like physical health, exists on a continuum from mild, time-limited symptoms to severe, chronic debilitating conditions. To study the role of culture in mental health problems, one needs standardized measures that cover a broad continuum of human behavior and have been adopted and studied across many societies.
One measure that meets these criteria is the Adult Self-Report (ASR) (Achenbach & Rescorla, Reference Achenbach and Rescorla2003). The ASR is a 15–20 min questionnaire for ages 18–59 that assesses dimensions of behavioral, emotional, social, and thought problems, and personal strengths. It has been adopted widely and used in over 250 published studies. It has also been adapted for use across many societies. First, each non-English version of the ASR is the result of a process in which indigenous researchers make a translation of the ASR in their language and then obtain an independent back-translation to insure the accuracy of the initial translation (Achenbach & Rescorla, Reference Achenbach and Rescorla2015). Next, confirmatory factor analysis has been used to test whether the ASR syndrome structure was supported across 29 societies (Ivanova et al., Reference Ivanova, Achenbach, Rescorla, Turner, Ahmeti-Pronaj, Au and Chen2015). The primary model fit index (root mean square error of approximation) showed good model fit for all samples (<0.03) and good to acceptable fit for secondary indices (e.g. comparative fit index (CFI), Tucker-Lewis Index (TLI)). Only five (0.06%) of the 8598 estimated parameters were outside the admissible parameter space. These findings support similarities in the factor structure and factor loadings across societies. Additional analyses tested similarities in the mean ratings of individual ASR items and scale scores (Rescorla et al., Reference Rescorla, Achenbach, Ivanova, Turner, Althoff, Árnadóttir and Chen2016). As indicated by a mean correlation of 0.77 between the mean ratings of items for all pairs of societies, the rank order of item ratings was similar across societies. The mean scale scores of the scales themselves do vary across societies in a manner consistent with a normal distribution. Together, these findings support the ASR as a useful tool for dimensional assessment of mental health problems in diverse societies. It also provides a way to test effects of societal and cultural difference on adults' self-ratings.
We use ‘society’ in reference to geopolitically demarcated populations that include but are not limited to countries. ‘Culture’ is defined as an accumulated set of beliefs, values, and social norms which impact the behavior of a relatively large group of people (Lustig, Koester, & Halualani, Reference Lustig, Koester and Halualani2006). The Global Leadership and Organizational Behavior Effectiveness study (GLOBE) mapped cultures by analyzing responses by 17 000 participants in 62 societies to questionnaires on cultural dimensions proposed by Hofstede including assertiveness, gender egalitarianism, collectivism, and future orientation (Hofstede, Reference Hofstede1984; House, Hanges, Javidan, Dorfman, & Gupta, Reference House, Hanges, Javidan, Dorfman and Gupta2004). Ten distinct ‘culture clusters’ were derived from these dimensions (e.g. Confucian, Anglo, Latin America, Eastern Europe, Sub-Saharan Africa).
Stankov (Reference Stankov2011) applied the GLOBE findings to test the effects of individual societies, GLOBE-defined culture clusters, and individual differences on personality measures completed by college students in 45 societies. For neuroticism, the personality trait most relevant to mental health, individual differences accounted for 95.3% of the variance, societal differences 2.0%, and culture cluster differences 2.7%. The societal findings for personality are similar to those obtained in another study of 130 602 adults in 22 societies (Kajonius & Mac Giolla, Reference Kajonius and Mac Giolla2017). A similar analytic approach was applied in two studies of ASR-related measures of child/youth mental health problems. For 11–18-year-olds' self-ratings, individual differences accounted for 92.5% of variance across 17 problem scales, societal differences 6.0%, and cultural differences only 1.5%, indicating modest effects of society and culture (Ivanova et al., Reference Ivanova, Achenbach, Turner, Almqvist, Begovac, Bilenberg and Chahed2022). For 6–18-year-olds rated by their parents, individual differences accounted for 90% of the variance across problem scales, societal differences 6%, and cultural differences 4% (Rescorla, Althoff, Ivanova, & Achenbach, Reference Rescorla, Althoff, Ivanova and Achenbach2019).
Purpose of the present study
In 28 societies representing seven GLOBE culture clusters, we tested whether society and culture cluster would account for significant variance in adults' self-rated mental health problems and personal strengths, over and above individual differences. Prior research on 11–18- and 6–18-year-olds suggest modest effects of society and culture. But adults have had much longer exposure to the norms and influences of their society and culture than children and youths. To further examine the effects of society and culture cluster on self-rated mental health problems and strengths, we tested society and gender as predictors in one multi-level model and culture cluster and gender in a second multi-level model.
Method
Samples
Indigenous researchers independently arranged to have ASRs completed by 16 906 18- to 59-year-olds in the 28 societies listed in Table 1. These samples were pooled as part of an international consortium of mental health researchers. Samples averaged 42% male, and Ns ranged from 293 (Egypt) to 2020 (USA). As shown in Table 1, rigorous random sampling methods were used in some societies, resulting in representative population samples. However, in other societies, various methods of convenience sampling were necessary, resulting in samples of unknown representativeness. Follow-up analyses tested models in 15 representative samples. Additional details of individual studies are available from cited references and/or listed primary investigators.
Table 1. ASR samples
a Only age ranges were available for Angola.
b Unpublished data.
c The identical sample sizes for Japan and Korea are coincidental, not errors.
Using the GLOBE culture cluster taxonomy (House et al., Reference House, Hanges, Javidan, Dorfman and Gupta2004), we classified the 28 societies into the following seven culture clusters: Anglo (N = 2362, two societies); Confucian Asia (N = 3182, five societies); Eastern Europe (N = 4475, nine societies); Latin America (N = 2094, four societies); Latin Europe (N = 2094, four societies); Middle East (N = 676, two societies); and Sub-Saharan Africa (N = 826, two societies) (see Table 1). Following the standard ASR procedure, cases that were missing ratings for >8 problem items were excluded from the analyses. Indigenous researchers followed their institutions' procedures for protection of human participants including obtaining informed consent. The lead authors' university human participants board approved the overall project. All data were de-identified.
Instrument and tested model
Indigenous mental health workers conducted translations of the ASR to their language and then obtained independent back-translations to insure the accuracy of the initial translation. The ASR's 120 problem items are rated 0 = not true, 1 = somewhat or sometimes true, or 2 = very true or often true, based on the preceding 6 months. These items tap diverse emotional, behavioral, social, and thought problems, such as I worry about my family; I am stubborn, sullen, or irritable; I argue a lot; and I have thoughts that other people would think are strange. The 11 personal strengths items (e.g. I make good use of my opportunities; I work up to my ability; I am pretty honest; I meet my responsibilities to my family; I try to be fair to others; and I am a happy person) are rated on the same 0–1–2 scale with high ratings indicating positive characteristics.
Our analyses focused on 17 ASR problem scales and one personal strengths scale. The scales included eight syndromes derived from exploratory and confirmatory factor analyses of ratings by adults in US population and clinical samples (Achenbach & Rescorla, Reference Achenbach and Rescorla2003). The syndromes are anxious/depressed, withdrawn, somatic complaints, thought problems, attention problems, rule-breaking behavior, aggressive behavior, and intrusive behavior. We also analyzed three broad-spectrum scales: total problems (comprised of all problem items); internalizing (anxious/depressed, withdrawn/depressed, and somatic complaints syndrome items); and externalizing (rule-breaking behavior, aggressive behavior, and intrusive syndrome items). Six DSM-oriented scales comprise ASR problem items identified by an international panel of experts as being very consistent with particular DSM-5 diagnostic categories (Achenbach, Reference Achenbach2013; Achenbach, Bernstein, & Dumenci, Reference Achenbach, Bernstein and Dumenci2005). The DSM-oriented scales are designated as depressive problems, anxiety problems, somatic problems, avoidant personality, AD/H problems, and antisocial personality. Adults' ratings of strengths items comprise an 11-item personal strengths scale.
For Japan, items assessing illegal behavior [6. I use drugs (other than alcohol and nicotine) for nonmedical purposes; 57. I physically attack people; 82. I steal; and 92. I do things that may cause me trouble with the law] were omitted from the ASR because their endorsement by respondents would have legally obligated the investigators to report them to authorities. To account for these excluded items, we re-wrote our scale-scoring syntax from simply taking the sum of items comprising each scale to, instead, taking the mean of the items comprising each scale (when there were valid responses available for at least 50% of such items) and multiplied that value times the total number of items comprising the scale.
Based on US data, Achenbach and Rescorla (Reference Achenbach and Rescorla2003) reported αs of 0.89–0.97 for the internalizing, externalizing, and total problem scales, 0.51–0.88 for the syndromes, and 0.68–0.84 for the DSM-oriented scales. The ASR's 1-week test–retest correlations were 0.89–0.94 for the broad-band scales, 0.78–0.91 for the syndromes, and 0.77–0.86 for the DSM-oriented scales. ASR items and scales discriminated significantly between demographically similar clinically referred and nonreferred samples of US adults. Additional ASR findings across societies are reported by Achenbach and Rescorla (Reference Achenbach and Rescorla2015); Ivanova et al. (Reference Ivanova, Achenbach, Rescorla, Turner, Ahmeti-Pronaj, Au and Chen2015); and Rescorla et al. (Reference Rescorla, Achenbach, Ivanova, Turner, Althoff, Árnadóttir and Chen2016).
Analyses
The effects of individual, society, and culture cluster contributions to differences on ASR scales were tested with hierarchical linear modeling (HLM) estimated using PROC MIXED in SAS 9.4. (SAS Institute, 2013). Mental health problem scores are positively skewed in general population samples (where many people obtain relatively low scores), but HLM has been found robust to deviations from normality, especially for large samples (Ketelsen, Reference Ketelsen2014; Man, Schumacker, Morell, & Wang, Reference Man, Schumacker, Morell and Wang2022; Schielzeth et al., Reference Schielzeth, Dingemanse, Nakagawa, Westneat, Allegue, Teplitsky and Araya-Ajoy2020). Each ASR scale was tested separately in a multilevel model. Individual differences (i.e. differences between individuals within a society) and unspecified effects (i.e. measurement error) were entered at level 1. Societal differences were entered at level 2. Culture cluster differences were entered at level 3. All multilevel models included intercepts and used the restricted maximum likelihood estimator that provides more robust results. The percent of variance due to predictors at each level was calculated as the ratio of the respective level-specific variance component over total variance. First, we tested the null model in which no predictors were entered at level 1, and society and culture cluster were modeled as random effects at levels 2 and 3. In addition to the random-effects model, all results were retested in fixed-effects models. For a small number of clusters, the fixed-effects model can be more robust than the random-effects model that assumes normality in cluster-specific random intercepts (McNeish & Kelley, Reference McNeish and Kelley2019). Next, we added age and gender as fixed effects at level 1 and reran the model for each ASR scale. Finally, we tested whether the economic status of societies was a stronger predictor of ASR scale scores than culture cluster. To do this, models were rerun with World Bank income group classification of societies based on purchasing power parity (PPP) included as a level 2 variable (World Bank, 2020).
To better understand how society and culture cluster interacted with age and gender in their relations to ASR scores, we used analyses of variance (ANOVAs) to test associations of internalizing, externalizing, and total problem scores with society (28 societies) and culture cluster (seven clusters), plus age, gender, and all possible interactions.
Results
Figure 1 presents internalizing and externalizing scores by culture cluster. The seven clusters are arranged in ascending order for mean internalizing problems score. Significant differences were observed between most culture clusters for both internalizing and externalizing problems using Student–Newman–Keuls (SNK) post hoc tests. For internalizing, there was only exception. The following clusters that did not differ significantly from each other: Sub-Saharan Africa and Middle East. For externalizing, exceptions were Eastern Europe and Latin America; Latin America and Sub-Saharan Africa; and Anglo and Confucian. The rank-ordering of culture cluster for externalizing differed from the rank-ordering for internalizing.
Figure 1. Mean raw scale score by culture cluster for Internalizing and Externalizing Problems.
Table 2 presents the variance components estimated for the multilevel null model for individual differences (level 1), societies (level 2), and culture cluster (level 3). Averaged across the 17 problem scales (i.e. all scales except personal strengths), the percent of variance accounted for by individual differences was 90.7%, by society was 6.3%, and by culture cluster was 3.0%. Results based on fixed-effects models (which are less constrained with a small number of clusters) were similar for problem scales and can be found in online Supplementary Table S1.
Table 2. Percent of variance accounted for by individual, society, and GLOBE culture cluster effects in hierarchical linear models of adult self-ratings on the ASR
All effects of individual differences and society were significant (p < 0.001). Effects of culture cluster did not reach the p < 0.05 level of significance.
For specific problem scales, the variance accounted for by individual differences ranged from 80.3% for DSM-oriented anxiety problems to 95.2% for DSM-oriented avoidant personality; by society: 3.2% for DSM-oriented somatic problems to 8.0% for DSM-oriented anxiety problems; and by culture cluster: 0.0% for DSM-oriented avoidant personality to 11.6% for DSM-oriented anxiety problems. Individual differences explained most of the variance in scores for problem scales, while society explained most of the remaining variance. The variance accounted for by individual differences (80.8%) in personal strengths was substantially smaller than for the 17 problem scales (90.7%). Hence, the variance accounted for by society (10.5%) and by culture cluster (8.7%) was greater for personal strengths than for society and culture cluster averaged across the problem scales (6.3% and 3.0%, respectively). In the fixed-effects models, the variance accounted for by culture cluster for personal strengths was greater still at 15.9%. Figure 2 displays all ASR scales ranked from lowest to highest for total variance accounted for by society and culture cluster in random-effects models. The total variance accounted for by society and culture cluster ranged from close to 5% for DSM avoidant personality to near 20% for personal strengths and DSM anxiety. When all models were retested only including 15 representative samples, the results were similar to findings in Table 2 and online Supplementary Table S1.
Figure 2. ASR scales ranked by lowest to highest for total contribution of society and culture cluster.
The multilevel models were rerun with age and gender as fixed effects at the individual level. Their addition did not significantly change the variance components for individual differences, society, or culture cluster for any scale. We then added the World Bank's PPP Index as a level 2 variable. In these models, individual differences accounted for 93.2% of the variance, society for 3.7%, culture for 2.1%, and the World Bank PPP Index for 1.0%, averaged across 17 problem scales (see online Supplementary Table S2). These findings confirm that most of the variance in ASR problem scale scores was associated with individual differences. For personal strengths, individual differences accounted for 75.8% of the variance, society for 7.2%, culture for 15.5%, and the World Bank PPP Index for 1.4%.
Table 3 presents the variance components from ANOVAs of internalizing, externalizing, and total problems scales. Predictors were society (28 societies) or culture cluster (seven clusters), plus age, gender, and all possible interactions. Results of ANOVAs for internalizing, externalizing, and total problems scores indicated that effects of society (7.5, 5.8, and 7.4%, respectively) were larger than effects of culture cluster (2.9, 1.7, and 2.8%, respectively). In terms of main effects, age had larger effects on externalizing and total problems than on internalizing problems (~5.1% v. 1.8%), whereas gender had larger effects on internalizing problems than on externalizing and total problems (~1.8% v. 0.2%). All of the 24 effects involving age and gender interactions were ⩽1%.
Table 3. Percent of variance accounted for by individual and society or GLOBE culture cluster in ANOVAs of ASR internalizing, externalizing, and total problems scores
p < 0.0001; p < 0.01; p < 0.05.
Discussion
Our goal was to conduct the broadest test to date of the effects of society and culture on differences in adults' self-rated mental health problems and strengths. Several findings are noteworthy. First, most of the variance in adult problem ratings (~90%) was associated with individual differences. Of the remaining variance, society accounted for, on average, double the variance of culture cluster. These estimates, however, varied across constructs: society and culture accounted for only 5% of the variance in DSM avoidant personality but up to 20% of the variance in DSM anxiety problems. The effects of society and culture on the personal strengths scale were twice as large as for the problem scales. Overall, the effects of society and culture on adults' self-rated problems were small to moderate, but they varied broadly – from small to large – across the different scales (Cohen, Reference Cohen1988).
We know of no other studies that have tested the effects of societal and cultural effects on adults' self-ratings of mental health problems. We know of one such study of parent ratings of children (Rescorla et al., Reference Rescorla, Althoff, Ivanova and Achenbach2019) and one of youth self-ratings (Ivanova et al., Reference Ivanova, Achenbach, Turner, Almqvist, Begovac, Bilenberg and Chahed2022). The results converge in three ways. Firstly, all three studies found that about 90% of the variance in problem scale scores was associated with individual differences (parent-ratings: 92.5%; youth self-ratings: 89.8%; adult self-ratings: 90.6%). This is striking given that the two child/youth studies included societies and culture clusters not included here. Next, society generally accounted for more of the variance in ratings of mental health problems than culture in all three studies (parent-report: 6.1% v. 4.2%; youth self-report: 6.0% v. 1.5%; adult self-report: 6.3% v. 3.0%). Finally, there were similarities in the rank ordering of results for the individual mental health scales. The DSM anxiety scale showed the largest combined effects of society and culture in both the adult and the youth self-rating studies (it was third in the child parent-rating study). Also, both the youth study and our study supported larger societal and cultural contributions to personal strengths (16.6% in youth self-ratings and 19.2% in adult self-ratings) than to problem scales.
Why did society and culture account for twice as much of the variance in ratings of strengths as in mental health problems? A similar discrepancy was observed by Stankov (Reference Stankov2011). There, societal and cultural effects on personality scales were smaller than on social attitude and norm scales. Stronger societal/cultural effects on social constructs than on personality scales might be expected. The ASR and Youth Self-Report (YSR) strengths scales, however, do not assess social constructs but rather self-ratings of strengths (e.g. I make good use of my opportunities, I work up to my ability, I am pretty honest). Our notions of strengths may reflect values that are shared or defined within different societal and cultural groups. Our notions of mental health problems, by contrast, may be more universal because of how these problems impair functioning and cause distress. Self-ratings of strengths may also be more affected by social desirability, but it is not clear why this would be the case. Future research should attempt to clarify why the effects of society and culture on self-ratings of strengths were larger than on self-ratings of problems. In any case, the findings for personal strengths suggest caution in comparing personal strengths across societies and cultures.
Limitations
Our study's strengths included (1) use of a standardized mental health measure that has been adopted and tested extensively across many societies; and (2) data from 16 906 adults across 28 societies that represent seven culture clusters. There are also limitations to consider. Our samples were collected under varying conditions in diverse societies by indigenous researchers. Some of our samples were of unknown representativeness because they were obtained using convenience rather than random sampling methods. Second, our findings are limited to the specific problems and strengths assessed by the ASR. Many societies and cultural groups were not included in the study. The inclusion of additional societies and cultures might yield different results. Also, other ways of classifying cultures might yield different results. In HLM analyses, effects associated with individual differences included residual variance. The residual variance may be due to other variables such as the adult's family, work, or local community. Third, our work has previously demonstrated similarity in factor structure and factor loadings, but did not formally test other aspects of measurement invariance. Finally, ‘society’ and ‘culture’ may be associated with genetic differences, as well as with socio-cultural demarcations of populations.
Conclusion
Over the past 30 years, the study of mental health problems has been extended to many societies around the world (Prince et al., Reference Prince, Patel, Saxena, Maj, Maselko, Phillips and Rahman2007; World Health Organization, 2001). In this time, the global health burden for mental health conditions has increased (Vos et al., Reference Vos, Lim, Abbafati, Abbas, Abbasi, Abbasifard and Abdelalim2020). The unmet need for mental health treatment around the world is large and, sadly, growing. Despite the myriad difference between societies and cultures with respect to geographic location, political/economic systems, history, population, ethnicity/race, and religion, the mental health conditions identified and studied across societies and cultures appear rather similar even if there are differences in the mean levels of those conditions. In previous work, our international consortium has obtained large α levels for ASR scales and a good fit for the syndrome structure across societies. The current study suggests that societal and cultural effects on problem scores are modest. Together, our findings suggest that cross-cultural use of standardized measures like the ASR to assess individual mental health problems is warranted but suggest more caution regarding personal strengths.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291723001332
Acknowledgements
Dr Copeland had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Drs Copeland, Achenbach, and Ivanova contributed to the concept and design, all authors contributed to the acquisition, analysis, and interpretation of the data, Drs Copeland, Achenbach, and Ivanova contributed to the manuscript, all authors critically revised the manuscript for important intellectual content, and all authors provided final approval for the submitted manuscript.
Financial support
This work was funded in part by the nonprofit University of Vermont Research Center for Children, Youth, and Families which publishes the Adult Self-Report and from which authors Copeland, Achenbach, and Ivanova receive funding. The remaining authors have declared that they have no competing or potential conflicts of interest.
