 |
 |
|
|
|
It is well known that depressive symptoms may be present from childhood and adolescence, interfering both with performance at school and with interpersonal relationships. Further, these symptoms may lead to the same consequences as are observed in adults, such as the development of psychiatric disorders and alcohol and drug abuse (1). Substantial evidence suggests that the clinical features of depression in children and adolescents do not differ significantly from those in adults (2–4). Accordingly, the ICD-10 and DSM-IV adult diagnostic criteria for depressive disorder are used in diagnosing both adolescent and childhood depression. Screening for depression in adolescents is particularly important because it is an underdetected and undertreated condition (5). Epidemiologic studies indicate a clear relation between the presence of depressive symptoms that do not fulfill diagnostic criteria for major depression in adolescence and the development of depressive disorders in adulthood (6,7). Thus it is also important to identify the presence of depressive symptoms in those who do not meet diagnostic criteria for major depressive disorders. As a first step for detecting depression before assessment with a comprehensive interview (standardized or semistandardized), various questionnaires that assess the presence of depressive symptoms can be used. The Beck Depression Inventory (BDI, 8) is one of the most widely used self-report instruments (9). Several studies support the BDI’s usefulness in measuring and predicting depression in adolescent samples (10,11). The scale’s format is clear; it is simple to administer; and it is easily understood by this population (12). The BDI’s psychometric properties have not been studied among Brazilian adolescents as yet, but its construct validity has been established in different Brazilian patient and college student populations (13,14). This study used the BDI to detect the prevalence of depressive symptomatology and its expression in a nonclinical adolescent student sample. Since this was the first study conducted among Brazilian adolescents, it also investigated whether there is any cultural specificity in the manifestation of depressive symptoms, as shown in other studies (15,16). MethodsSubjects The subjects were 1555 Brazilian adolescents (796 girls and 759 boys), aged 13 to 17 years (mean age 14.9 years, SD 1.2), from private (n = 1027) and public (n = 528) schools in the city of São Paulo, attending day (n = 1173) or evening (n = 382) classes. At the beginning of their regular classes, they were asked to voluntarily answer a set of questionnaires in their classrooms, without identifying themselves. The study was approved by the ethics committee of the Clinical Hospital of the Medical School of University of São Paulo. Instruments We used the Portuguese version of the 21-item Revised BDI (17). The scale items include symptoms and attitudes whose intensity ranges from neutral to maximum severity, rated as 0 to 3. We adopted Kendall and others’ recommendation (18) for nonclinical populations; that is, we considered that scores higher than 15 detected dysphoria and that scores over 20 indicated depression. These cut-off scores have been used in similar studies (19). Statistics We performed analysis of variance (ANOVA) followed by the Tukey t test to compare sociodemographic characteristics. We calculated the internal consistency for the BDI according to the Cronbach alpha coefficient. We compared BDI psychometric properties by sex and level of depressive symptomatology according to Kendall and others’ cut-offs (18) for nonclinical populations: BDI < 15 = “nondepressed”; BDI 16 to 20 = “dysphoria”; BDI > 20 = “depressed.” We used Student’s t test to compare individual item means, with Bonferroni adjustments of P = 0.05 for the 21 comparisons protecting for familywise error rate (individual significant values, P < 0.002). We performed principal components factor analysis with varimax rotation to assess the factor structure of the scale on the total sample and by sex. Two discriminant analysis models were used. In the first model, we tested whether the “depressed” and “nondepressed” subgroups could be separated according to all individual items. In the second model, we tested whether those subgroups could be separated according to depression-specific and nonspecific items defined as follows (20): specific items were “sadness,” “pessimism,” “sense of failure,” “guilty feelings,” “self-dislike,” “suicidal wishes,” and “weight loss”; nonspecific items were “work inhibition,” “sleep disturbance,” ‘fatigability,” and “loss of libido.” ResultsTable 1 presents BDI scores according to sociodemographic characteristics. Sex differences were significant, with women having higher scores than men on both scales. ANOVA showed an age effect on BDI scores, with subjects aged 13 years having significantly lower BDI scores than the subgroup aged 17 years (P < 0.05). Public school students had significantly higher mean scores than private school students in both measures. Also, students attending night classes had higher BDI scores than those studying during the day (P < 0.001). Public school students had higher a percentage of BDI scores over 20 (10.4% vs 6.1%), as did those studying at night (11.8% vs 6.2%).
Mean BDI total score for subgroups according to the cut-off scores are also shown in Table 1. Considering the total sample, 9% of women (n= 72) had scores compatible with depression (BDI > 20), in contrast to 6.1% of men (n = 46). Table 2 shows BDI individual item mean scores and SDs, as well as the item-total correlations for total sample and by BDI cut-off scores. In all individual items, the subgroup with depression showed significantly higher scores than the subgroup without depression. “Irritability” had the highest score in the subgroup without depression and had one of the highest scores in the subgroup with depression. In the subgroups both with and without depression, self-accusations and indecisiveness also had high scores. The items with the next-highest scores in the subgroup with depression were “lack of satisfaction,” “sense of punishment,” and “crying spells.” The lowest scores in both subgroups were for “weight loss” and “loss of libido,” while the lowest-scoring item for the subgroup without depression was “sense of failure.” Higher item-total coefficients were obtained for the items “sense of failure,” “self-dislike,” and “guilty feelings” in the subgroup with depression and for the items “indecisiveness,” “self- accusations,” and “crying spells” in the subgroup without depression.
Mean scores for individual BDI items were mostly similar for the 2 sexes, but girls had significantly higher scores (P < 0.05) than boys for the items “sadness” (mean 0.17, SD 0.52 vs mean 0.33, SD 0.62), “indecisiveness” (mean 0.59, SD 0.88 vs mean 0.79, SD 0.96), “fatigability” (mean 0.48, SD 0.63 vs mean 0.61, SD 0.72), “loss of appetite” (mean 0.33, SD 0.62 vs. mean 0.43, SD 0.71), “somatic preoccupation” (mean 0.32, SD 0.57 vs. mean 0.45, SD 0.68), and “loss of libido” (mean 0.10, SD 0.44 vs. mean 0.18, SD 0.57). Item-total correlations were higher (0.53 to 0.6) for items 10, 1, 3, 13, 7, and 4 and lower (0.25 to 0.4) for items 21, 19, and 11. We evaluated item-total correlation to identify which items were more associated with the BDI total score. According to the BDI factor analysis for the total sample, the first unrotated factor accounted for 25% of the variance, and the second accounted for 5.8% of additional variability. When we considered loadings greater than 0.40 (when 2 loadings were similar, the item was considered to be part of both factors; when different, the highest loading was chosen), principal component analysis with varimax rotation suggested that the 3 BDI factors that could be extracted were related to the following items: for factor 1, items 1, 3, 4, 5, 6, 7, 9, 12, 13, 14, and 15; and for factor 2, items 11, 13, 16, 17, 18, and 19. Based on the items related to factors 1 and 2, Cronbach’s alpha coefficients for the subscales were 0.80 and 0.49, respectively. Factor 1 represents the cognitive–affective dimension, while factor 2 represents items more related to a somatic nonspecific dimension (Table 3).
We extracted 2 factors from the factor analysis for the girls’ subgroup (unrotated factors accounted for 26.1% and 5.6% of the variance, respectively). Principal component analysis with varimax rotation showed that the factors were related to the following items: for factor 1, items 1, 3, 4, 5, 6, 7, 8, 9, 12, 13, 14, 15, and 20; and for factor 2, items 11, 16, 18, and 19 (Cronbach’s alpha coefficients for the subscales were 0.83 and 0.32, respectively). We also extracted 2 factors for the boys’ subgroup (accounting for 23.7% and 6.4% of the variance, respectively). Principal component analysis with varimax rotation suggested that they were related to the following items: for factor 1, items 1, 2, 3, 4, 5, 6, 7, 9, 12, and 14; and for factor 2, items 13, 15, 16, 17, 18, and 20 (Cronbach’s alpha coefficients for the subscales were 0.78 and 0.64, respectively). Discriminant analysis considering all BDI items showed 96.0% correct classification for subjects without depression, 94.4% for subjects with depression, and 95.9% for the total sample. The most powerful discriminating items (those with higher correlations between items and the standardized canonical discriminant function) were (in decreasing order) 3, 4, 1, 7, 5, 12, and 6; the least important were items 11 and 21. In the discriminant analysis according to specific depression items, we obtained 94.2% correct classification for the subgroup without depression, 83.2% for subjects with depression, and 93.5% for the total sample. The most important items were 3, 1, 7, 5 and 9; the least important was item 19. With regard to nonspecific depression items, we obtained 84.1% correct classification for the subgroup without depression, 76.6% for subjects with depression, and 83.5% for the total sample. The most important items were 15, 17, and 16. DiscussionMajor depressive disorder is increasingly common among children and adolescents (21,22). The huge consequences for social functioning, lower academic achievement, more alcohol consumption, and the risk of recurrence, comorbidity, and suicide attest to the need to address this problem. There have been several efforts to improve the early detection of depression and to develop programs to prevent and treat it as soon as possible (23). The precise magnitude of the problem is unknown, since epidemiologic studies usually base information about the onset-age of the symptomatology on retrospective data. Screening for depression and depressive symptoms in a nonclinical population has the advantage of maximizing the chance to identify those at high risk of developing the disorder. Although the BDI is not designed for diagnostic purposes, its epidemiologic utility has been evaluated in several studies, which concluded that it is a reliable instrument for detecting depressive disorders in nonclinical populations. Lasa and others assessed the BDI’s predictive ability to detect depression in a sample of 1250 general population subjects (24). As the validation measure, they used the Schedules for Clinical Assessment in Neuropsychiatry (SCAN), a semistructured interview applied to all subjects with a BDI score equal to or higher than 13 (indicating probable cases) and to 5% of the subjects with a score under the adopted BDI cut-off. Considering a BDI cut-off of 12 to 13, they found 100% sensitivity, 99% specificity, and a positive predictive value of 0.72 (total diagnostic value = 98%). Further, considering the SCAN interview as the gold standard, Canals and others evaluated the epidemiologic utility of the BDI in 304 nonclinical adolescents (25). According to these investigators, the best cut-off scores found for major depression and dysthymia were 16 and 10, respectively. Using the Child Assessment Schedule as a diagnostic instrument and adopting a BDI screening score of 16, Barrera and Garrison-Jones obtained 100% sensitivity and 93.2% specificity for the BDI (10). According to the results of the above reports and considering BDI scores above 20 as indicating depression, it is possible to predict that approximately 90% of these subjects will meet diagnostic criteria, which corresponds to a point prevalence of 6.8% for our sample. To compare our results with other adolescent population, we also reported them considering the cut-off of 16. Our finding agrees with the prevalence of depressive symptoms in adolescents reported in various studies and countries (Table 4).
The BDI factor analysis is a good indicator of the expression of depressive symptomatology. Factor analysis did not show a markedly different symptom pattern between sexes. Just 4 items were factor components in only one sex: “weight loss” and “self-accusations” were found only among girls, while “pessimism” and “fatigability” were present only among boys. Despite these differences, the general structure of the factors was the same: factor 1 represented mainly the cognitive–affective dimension, and factor 2 represented the somatic dimension. The result of this factor analysis was quite similar to that found for college students (13), especially in the female group: 12 out of 13 factor 1 items were present in both samples, and 4 out of 5 factor 2 items in the adolescent analysis matched those found in the college-student sample. In the male sample, 6 out of 10 items for factor 1 and 5 out of 10 items for factor 2 were common to both samples. It seems that the expression of depression is similar in both clinical and nonclinical adolescent samples. Bennett and others found 4 BDI factors in a sample of 328 adolescents with depressive and (or) anxiety disorders (11). The first 2 factors, negative self-attitude and somatic symptoms (with 10 and 3 items, respectively), were similar to the factors found in our sample of adolescents (with 9 items agreeing in the first factor and 3 items in the second). Considering the results of our study and studies that reported BDI factors (12,19,26,27), factor 1 had 5 common items—“sense of failure,” “guilty feelings,” “self-dislike,” “suicidal wishes,” and “distortion of body image”—most of which were related to self-depreciation. These symptoms likely represent the core symptoms of depression in adolescents. Conversely, the BDI items showing higher scores in the group with probable depression were “lack of satisfaction,” “sense of punishment,” “self-accusations,” “crying spells,” “irritability,” “indecisiveness,” and “distortion of body image.” It is noteworthy that irritability, usually considered a prominent symptom of depression in adolescents, appears in this sample as a very common psychological manifestation in this age group, irrespective of the subgroup. The high rate of correct classification for subjects with possible depression, based on specific depression items, and for subjects without depression, based on nonspecific items, as well as the higher misclassification of subjects with depression, based on nonspecific items, also suggests that the BDI measures specific aspects of depression, as stated by Hill and others (28), and not only the general psychopathology in student samples, as claimed by Gotlib (29). Notably scores above zero for the item “suicidal wishes” were present in 19.2% of the girls’ sample (2.9% had scores 2 and 3) and in 14.9% of the boys’ sample (3.3% had scores 2 and 3), which agrees with results reported for a Swedish (19,26) and Brazilian (30) adolescent sample. This is particularly important when we consider epidemiologic data showing a worldwide suicide percentage of 0.8% for individuals aged 5 to 14 years and 12.8% for individuals aged 15 to 24 years (31). In Brazil, the suicide rate among individuals aged 15 to 24 years increased 18 times from 1980 (0.3 per 100 000 inhabitants) to 1992 (5.5 per 100 000 inhabitants) (31). The sex differences found in BDI scores, pointing to significantly higher scores for female subjects, are in line with data observed in other studies of adolescents as well as adults (19,32). The finding that 18.5% of female students, in contrast to 13.6% of male students, had scores compatible with dysphoria or depression also agrees with reports of a higher prevalence of depression in women (33,34): for depression, a 1.5:1 ratio exists in favour of women in the general population (33), in adolescents (35), and in Brazilian college students (13). This relation is maintained when we consider only scores indicating depression (9.0% of the total number of girls in our sample and 6.1% of the total number of boys). In Portugal, another Portuguese-speaking country, similar results were obtained in a sample of 1812 adolescents aged 12 to 16 years: the prevalence of dysphoria or depression was 21.9% for female students and 13.9% for male students (C Rainho, personal communication, 1999). In contrast with other adolescent studies (12,26), we found an age-related difference in BDI scores: subjects aged 13 years had lower mean scores than did subjects aged 17 years. Notably, the mean obtained for the students aged 17 years was the same as that for Brazilian college students aged 18 to 29 years (mean 10.7, SD 7.7) (13). Thus there seems to be a trend to increasing BDI scores from the beginning of adolescence to the beginning of adulthood. Our higher mean scores and percentage of subjects with scores indicating depression in public schools and in those attending night classes agree with epidemiologic studies that identify a low socioeconomic level as one of the risk factors for depression (36). In conclusion, the Portuguese version of the BDI has been shown to be a reliable and inexpensive instrument to detect specific aspects of depression in nonclinical populations. Our results showed high rates of depressive symptoms in a school sample of adolescents, which is compatible with international studies of this age group. This high rate emphasizes the importance of identifying depressive problems in nonclinical populations of this age group and points to the need for routine screening for depressive symptomatology in school-age populations—in line with recent epidemiologic studies (37) suggesting that serious mental disorders usually start in childhood or adolescence and that school-based interventions may help reach young people with mild symptoms and prevent progression to such serious outcomes as drug problems, teen childbearing, school failure, and violent relationships. Funding and SupportDr Gorenstein is supported by CNPq; Ms Zanolo was supported by FAPESP (98/06857-9); and Dr Artes was supported by CNPq (PRONEX 66.2285/1997-2) and FAPESP (99/10611-8). References1. Wittchen HU, Nelson CB, Lachner G. Prevalence of mental disorders and psychosocial impairments in adolescents and young adults. Psychol Med 1998;28:109–26. 2. Lewinsohn PM, Rohde P, Seeley JR. Major depressive disorder in older adolescents: prevalence, risk factors, and clinical implications. Clin Psychol Rev 1998;18:765–94. 3. Winter LB, Steer RA, Jones-Hicks L, Beck AT. Screening for major depression disorders in adolescent medical outpatients with the Beck Depression Inventory for Primary Care. J Adolesc Health 1999;24:389–94. 4. Patton GC, Coffey C, Posterino M, Carlin JB, Wolfe R. Adolescent depressive disorder: a population-based study of ICD-10 symptoms. Aust N Z J Psychiatry 2000;34:741–7. 5. Keller MB, Lavori PW, Beardslee WR, Wunder J, Ryan N. Depression in children and adolescents: new data on ‘undertreatment’ and a literature review on the efficacy of available treatments. J Affect Disord 1991;21:163–71. 6. Pine DS, Cohen E, Cohen P, Brook J. Adolescent depressive symptoms as predictors of adult depression: moodiness or mood disorder? Am J Psychiatry 1999;156:133–5. 7. Fombonne E, Wostear G, Cooper V, Harrington R, Rutter M. The Maudsley long-term follow-up of child and adolescent depression. 1. Psychiatric outcomes in adulthood. Br J Psychiatry 2001;179:210–7. 8. Beck AT, Ward CH, Mendelson M, Mock JE, Erbaugh JK. An inventory for measuring depression. Arch Gen Psychiatry 1961;4:53–63. 9. McDowell I, Newell C. Measuring health: a guide to rating scales and questionnaires. 2nd ed. New York: Oxford University Press; 1996. 10. Barrera M Jr, Garrison-Jones CV. Properties of the Beck Depression Inventory as a screening instrument for adolescent depression. J Abnorm Child Psychol 1988;16:263–73. 11. Bennett DS, Ambrosini PJ, Bianchi M, Barnett D, Metz C, Rabinovich H. Relationship of Beck Depression Inventory factors to depression among adolescents. J Affect Disord 1997;45:127–34. 12. Teri L. The use of the Beck Depression Inventory with adolescents. J Abnorm Child Psychol 1982;10:277–84. 13. Gorenstein C, Andrade L, Vieira Filho AHG, Tung TC, Artes R. Psychometric properties of the Portuguese version of the Beck Depression Inventory on Brazilian college students. J Clin Psychol 1999;55:553–62. 14. Gorenstein C, Andrade L. Validation of a Portuguese version of the Beck Depression Inventory and the State-Trait Anxiety Inventory in Brazilian subjects. Braz J Med Biol Res 1996;29:453–7. 15. Byrne BM, Baron P, Balev J. The Beck Depression Inventory: testing for its factorial validity and invariance across gender for Bulgarian non-clinical adolescents. Person Individ Diff 1996;21:641–51. 16. Stewart SM, Betson CL, Lam TH, Chung SF, Ho HH, Chung TC. The correlates of depressed mood in adolescents in Hong Kong. J Adolesc Health 1999;25:27–34. 17. Beck AT, Rush AJ, Shaw BF, Emery G. Terapia cognitiva da depressno. Rio de Janeiro: Zahar; 1982. 18. Kendall PC, Hollon SD, Beck AT, Hammen CL, Ingram RE. Issues and recommendations regarding use of the Beck Depression Inventory. Cognitive Ther Res 1987;11:289–99. 19. Olsson G, Von Knorring AL. Beck’s Depression Inventory as a screening instrument for adolescent depression in Sweden: gender differences. Acta Psychiat Scand 1997;95:277–82. 20. Salamero M, Marcos T, Gutiérrez F, Rebull E. Factorial study of the BDI in pregnant women. Psychol Med 1994;24:1031–5. 21. Birmaher B, Ryan ND, Williamson DE, Brent DA, Kaufman J. Childhood and adolescent depression: a review of the past 10 years. Part I. J Am Acad Child Adolesc Psychiatry 1996;35:1427–39. 22. Andrade L, Caraveo-Anduaga JJ, Berglund P, Bijl RV, De Graaf R, Volleberg W, and others. The epidemiology of major depressive episodes: results from the International Consortium of Psychiatric Epidemiology (ICPE) surveys. Int J Methods Psychiatr Res 2003;12:3–21. 23. Parker G, Roy K. Adolescent depression: a review. Aust N Z J Psychiatry 2001;35:572–80. 24. Lasa L, Ayuso-Mateos JL, Vazquez-Barquero JL, Diez-Manrique FJ, Dowrick CF. The use of the Beck Depression Inventory to screen for depression in the general population: a preliminary analysis. J Affect Disord 2000;57:261–5. 25. Canals J, Bladé J, Carbajo G, DomPnech-Llabería E. The Beck Depression Inventory—psychometric characteristics and usefulness in nonclinical adolescents. Eur J Psychol Assess 2001;17:63–8. 26. Larsson B, Melin L. Depressive symptoms in Swedish adolescents. J Abnorm Child Psychol 1990;18:91–103. 27. Byrne BM, Byron P. Measuring adolescent depression: tests of equivalent factorial structure for English and French versions of the Beck Depression Inventory. App Psychol Int Rev 1993;43:33–47. 28. Hill AB, Kemp-Wheeler SW, Jones SA. What does Beck Depression Inventory measure in students? Pers Indiv Differ 1986;7:39–47. 29. Gotlib IH. Depression and general psychopathology in university students. J Abnorm Psychol 1984;93:19–30. 30. Feijó RB, Saueressig M, Salazar C, Chaves ML. Mental health screening by self-report questionnaire among community adolescents in southern Brazil. J Adolesc Health 1997;20:232–7. 31. World Health Organization. Figures and facts about suicide. Geneva: WHO; 1999. 32. Steer RA, Beck AT, Brown G. Sex differences on the revised Beck Depression Inventory for outpatients with affective disorders. J Pers Assess 1989;53:693–702. 33. Lehmicke N, Hicks AH. Relationship of response-set differences on Beck Depression Inventory scores of undergraduate students. Psychol Rep 1995;76:15–21. 34. Andrade L, Walters EE, Gentil V, Laurenti R. Prevalence of ICD-10 mental disorders in a catchment area in the city of Sao Paulo, Brazil. Soc Psychiatry Psychiatr Epidemiol 2002;37:316–25. 35. Angold A, Costello EJ, Worthman CM. Puberty and depression: the roles of age, pubertal status and pubertal timing. Psychol Med 1998;28:51–61. 36. Kessler RC, McGonagle KA, Zhao S, Nelson CB, Hughes M, Eshleman S, and others. Lifetime and 12-month prevalence of DSM-III-R psychiatric disorders in the United States. Results from the National Comorbidity Survey. Arch Gen Psychiatry 1994;51:8–19. 37. Bijl RV, de Graaf R, Hiripi E, Kessler RC, Kohn R, Offord DR, and others. The prevalence of treated and untreated mental disorders in five countries. Health Aff 2003; 22:122–33. 38. Hammen CL, Padesky CA. Sex differences in the expression of depressive responses on the Beck Depression Inventory. J Abnorm Psychol 1977;86:609–14. 39. Kaplan SL, Hong GK, Weinhold C. Epidemiology of depressive symptomatology in adolescents. J Am Acad Child Psychiatry 1984;23:91–8. 40. Roberts RE, Lewinsohn PM, Seeley JR. Screening for adolescent depression: a comparison of depression scales. J Am Acad Child Adolesc Psychiatry 1991;30:58–66. 41. Baron P, Perron LM. Sex differences in the Beck Depression Inventory scores of adolescents. J Youth Adolescence 1986;15:165–71. Author(s)Manuscript received January 2004, revised, and accepted May 2004. 1. Associate Professor, Department of Pharmacology, Institute of Biomedical Sciences, University of São Paulo, São Paulo, Brazil; Researcher, Lim-23, Laboratory of Psychopharmacology, Clinical Hospital, Medical School, University of São Paulo, São Paulo, Brazil. 2. Researcher, Lim-23, Laboratory of Psychopharmacology, Clinical Hospital, Medical School, University of São Paulo, São Paulo, Brazil; Assistant, Institute of Psychiatry, Clinical Hospital, Medical School, University of São Paulo, São Paulo, Brazil. 3. Student, Institute of Psychology, University of São Paulo, São Paulo, Brazil. 4. Associate Professor, Ibmec, São Paulo, Brazil. Address for correspondence: Dr C Gorenstein, LIM-23, Instituto de Psiquiatria, Rua Dr Ovidio Pires de Campos, 785 - térreo, CEP 05403-010, São Paulo, SP, Brazil. e-mail: cgorenst@usp.br
1 | 2
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||