 |
 |
|
|
|
Several interventions designed to prevent conduct disorder (CD) are currently at various stages of empirical validation. One genre of intervention—early-age programs—targets children with early-onset aggressive behaviour during the early elementary school grades. Such programs typically use a multifaceted intervention model that includes components for children, parents, peers, and schools. Studies testing the efficacy of multifaceted prevention interventions among aggressive children have produced small-to-medium effect sizes (1,2). These studies have used an intent-to-intervene analytical approach that includes all randomized subjects, regardless of client attrition and participation levels; they provide information on the average effect of the intervention. This analytical approach has several limitations that act to conceal a rich body of information about how a program works for different subgroups of the sample. These limitations include the failure to distinguish program impact for participants who elect high vs low rates of participation (that is, dosage effects), the failure to identify the active ingredients of the program (that is, dismantling effects), and the failure to identify how specific participant characteristics interact with various intervention components (that is, moderator effects). The notion that a particular intervention model can work with any child or family has given way to questions about tailoring the intervention to the perceived and expressed needs of individual families (3). In particular, families recruited for prevention programs may behave differently from families seeking treatment services. These differences may reflect differential rates of participation and varying degrees of benefit across available program components. Identification of specific participant characteristics that affect participation in intervention components can facilitate program tailoring and fit. This may lead to program benefits that maximize acceptability and improvement in client outcomes while preserving scarce community prevention resources. This study examined the goodness-of-fit of the Early Risers “Skills for Success” program. This program is an example of a theory-based, multifaceted prevention model that targets children with heightened risks for developing serious conduct problems as a consequence of early-onset aggressive behaviour and exposure to harmful contextual factors. The intervention model features 2 complementary components: CORE and FLEX. Providers deliver the program components in tandem over several years. The CORE component includes an annual 6-week summer program, a biweekly family program consisting of concurrent child and parent education and skills training groups, and a monitoring and mentoring school engagement program. The FLEX component is a risk-adjusted intervention that offers a menu of support, brief interventions, and community mental health options. FLEX individually tailors service delivery in response to an ongoing assessment of each family’s unique profile of strengths, needs, and barriers to participation. The efficacy of the Early Risers program was demonstrated with a well-characterized sample of aggressive children in a randomized control group design (4–6). Interestingly, a component analysis showed that the intervention’s individual components worked differently for children who showed high vs low aggression (7). Our study sought to elucidate further the goodness-of-fit between client and component. The treatment literature has shown that characteristics which define a parent’s level of global adaptive functioning in terms of high levels of stress or adverse life events, marital discord, family conflict, parental mental health problems, and social insularity are associated with low rates of participation, premature termination, and poor outcomes in clinic-based treatment for children with disruptive disorders (8–11). In the present study, we examined the hypothesis that parents’ level of psychosocial adaptive functioning would moderate their participation in various Early Risers component programs. Participation rate would in turn mediate subsequent benefits from component programs. We anticipated differences between parents with high adaptive functioning and parents with low adaptive functioning and children’s outcomes across the program’s individual components. Social competence was selected as the primary outcome because of its pivotal position in the CD developmental pathway (12–14), its primary focus across all Early Risers intervention components, and the magnitude of program–control differences that were observed on this construct at the 3-year assessment point. MethodParticipants We examined only intervention families and eliminated from this group those whose participation was censored because they had moved out of the area (n = 17), reducing the sample to 107 children (65% boys, mean age 6.6 years at baseline, mean Kaufman Brief Intelligence Test [K-BIT] composite IQ 97.4, and mean baseline aggression T-Score 63.5 on the CBCL–TRF). Retained participants did not differ significantly from those who were dropped from the analysis. Interventions CORE Intervention Component Family Program. In each of the 3 years, parents and their children were invited to attend concurrent education and skills training sessions from October through May. These sessions took place during the evening or on Saturday morning at a local school site. Twelve 2-hour sessions were offered in year 1, eleven 2-hour sessions were offered in year 2, and six 2-hour sessions were offered in year 3 (for a total of 29 sessions and 58 hours). In years 1 and 2, the content of parent-group sessions followed Webster-Stratton’s The Incredible Years Parenting Series (19). Child-group sessions followed interactive play with the life-size puppets of Webster-Stratton’s Incredible Years Dinosaur Curriculum (19), a program designed to dovetail with content presented in the parenting sessions. The curriculum emphasized educating and training children in skills such as emotional regulation, conflict resolution, friend making, and understanding school rules and how to succeed at school. FLEX Intervention Component Measures Parent SES. We used the Hollingshead Rating System (22) to calculate parents’ self-report of occupation and education and yield SES at baseline. Single-Parent Status. This was determined at baseline and year 2 from parent report of the number and relationship of all adults in the household. Parent Nurturance. This was a composite variable computed as the mean of 5 scales from 4 different instruments. Two scales from the Alabama Parenting Questionnaire (APQ) (23) were included: Involvement (4-week retest reliability = 0.61) and Positive Parenting (4-week retest reliability = 0.59). One scale was used from the teacher version and one from the parent version of the Parent-Teacher Involvement Questionnaire (1) that assesses the extent of parent participation in school functions. Finally, one subscale, Cohesion, was used from the Family Relations Scale (FRS) (24). Internal consistencies (alphas) of the Involvement, Positive Parenting, Parent Participation (teacher and parent), and Cohesion scales in the current sample were 0.75, 0.80, 0.68, 0.65, and 0.75, respectively. Parent Discipline. This variable consisted of the APQ Inconsistent Discipline scale (α = 0.73; at 4-week retest, α = 0.68) and a scale from the Parenting Practices Questionnaire (25) designed to measure the effect of discipline (α = 0.81). Parent Stress. This was measured by 3 scales from the Parenting Stress Index (PSI) (26)—Depression, Isolation, and Health—and by the Support scale from the FRS. Alphas for these scales were 0.79, 0.77, 0.70, and 0.70, respectively. Parent Psychosocial Global Adaptive Functioning. The Global Assessment of Functioning (GAF) scale generated a composite index of parents’ global adaptive functioning (27). The index, a hypothetical continuum from mental health to illness scored from 1 (serious dysfunction) to 100 (superior functioning), was computed for the primary caregiver in each family (usually the mother). The convergent and discriminant validity of the GAF have been reported elsewhere (28). The family advocate (with consultation from an experienced clinical supervisor) assigned GAF scores for each caregiver, based on information reflecting a broad array of strengths and problems, including availability of formal and informal social and family supports, occupational satisfaction, and symptoms of psychiatric distress. Scores ranged from a low of 32 to a high of 93. The mean score was 72, SD 12; the median score was 74. Preliminary analyses grouped participants by GAF ( < 70 was considered low functioning [LF]; > 71 was considered high functioning [HF]) to determine whether there were group differences in the mode across groups. They were used in subsequent analyses after differences were determined to exist (LF n = 45; GAF mean 61, SD 8.6. HF n = 53; GAF mean 81, SD 5.1). As shown in Table 1, LF parents relative to their HF counterparts had lower levels of nurturance, higher levels of stress, children with lower levels of overall functioning, fewer social relationships, and fewer community activities.
Measures of Participation. Family participation in the family program and child participation in the summer program were computed as the percentage of sessions attended out of total sessions offered over 3 years of intervention (29 and 72 sessions for family and summer programs, respectively). The total time (in official contact minutes) between parent and family advocate over a period of approximately 2 years was used as the parent participation measure for the FLEX program. As expected for this tailored intervention component, families varied considerably in their participation time (range 0 to 69 hours). In addition, there was a highly positive skew in the distribution of time spent in FLEX, with one-half of the families receiving fewer than 3 hours (mean 6.5 hours). Because of the variability and shape of the distribution, the percentage of time parents spent in FLEX had a base of 300 minutes (5 hours). This represented the family advocate’s goal over the time interval in which they implemented this component. Scores of 100% were assigned to parents who received more than 300 minutes of FLEX. For each FLEX family contact, family advocates recorded the focus of the contact (that is, child, parent, or family) and the time spent discussing each domain immediately after each contact. We computed percentage of time spent in each domain over the total time spent in FLEX. Because family advocates could spend the same interval of time on both child and parent issues, times spent in each domain were not mutually exclusive, and percentages do not necessarily sum to 100. Social Competence Outcome Measure. This was a composite variable based on the mean of 3 scales (Social Skills, Leadership, and Adaptability) from both the teacher and parent forms of the Behavioral Assessment System for Children (BASC, 29) and the Social Acceptance scale from the Teacher’s Scale of Child’s Actual Competence and Social Acceptance (30). For the teacher and parent scales of the BASC, respectively, the alphas were as follows: Social Skills (0.91, 0.86), Leadership (0.81, 0.78), and Adaptability (0.75, 0.73). For the Social Acceptance scale, the alpha was 0.92. Timeline for Assessments. Measures of outcome, including child social competence (that is, social skills) and parent nurturance, discipline, and stress, were administered at baseline (that is, in the spring of the children’s kindergarten year and prior to the first summer program) and every spring thereafter for 3 consecutive years. We continuously recorded participation as intervention components came on-line over the 3-year trial. Data collected during the first 6 months of year 1 were accumulated to allow the family advocates sufficient time to access valid data to determine parent GAF (grouping variable). Statistical Analysis To test the principal hypothesis that parent psychosocial functioning (measured by GAF) plays a significant role in determining which program components clients choose to participate in, we assessed the effect of each specific program component on child social competence. We tested the relation between each part of the model (child, parent, and family baseline variables, as well as rates of participation and outcome) and the mean differences between LF and HF families. Second, we tested whether the regression weights (path coefficients) in the overall model differed between LF and HF parents. We did this to determine whether a single model or a 2-group model fit the data. ResultsRates of Participation Group Differences in Family Factors, Attendance, and Child Social Competence For predictors of participation, the difference in fit between the 2 models was significant (n = 98; c2 = 14.7, df 4; P = 0.005). Tests of between-groups effects revealed a significant difference between groups on SES (LF mean 37.86, SD 7.93; HF mean 44.55, SD 10.63; F1,93 = 11.53; P = 0.001) but not on level of child aggression, single-parent status, or child IQ. The constrained model was rejected for rates of participation in the 3 intervention components (n = 98; c2 = 18.7, df 9; P = 0.028), and the difference in fit between the 2 models was marginal (n = 98; c2 = 7.7, df 3; P = 0.053). Tests of between-group effects revealed a significant difference between groups on attendance in the family program (F1,96 = 7.45, P = 0.008): families in the LF group had lower mean attendance in the family program (mean 31.03%, SD 27.75%) than families in the HF group (mean 47.37%, SD 30.93%). The effect size (Cohen’s d) for this group difference was 0.55. There were no significant between-group differences in attendance at the summer program or in contact time in FLEX. For child social competence, the difference in fit between the 2 models was significant (n = 98; c2 = 12.50, df 2; P = 0.002). Results from a repeated measures MANOVA revealed significant effects for time (F1,87 = 19.60, P < 0.001) and parent functioning group (F1,87 = 12.10, P = 0.001). Children in both the LF group (t41 = –2.13, P = 0.04) and the HF group (t46 = –4.37, P = < 0.001) significantly improved in social competence over time. In addition, GAF groups differed at baseline (t87 = –2.66, P = 0.009) and at year 3 (t87 = –3.35, P = 0.001): children in the LF group scored lower on social competence at baseline (mean –0.86, SD 0.53 vs mean –0.51, SD 0.69) and at year 3 (mean –0.63, SD 0.74 vs mean –0.13, SD 0.66) than did children in the HF group. However, there was no significant time × GAF interaction (F1,87 = 1.14, P = 0.29), suggesting that the rate of change in social competence did not differ by parent functioning. Modelling Participation in GAF Groups Part I: Who Attended? Predictors of Participation Attendance in the Summer Program. For the LF group, both child IQ (β = 0.29, P < 0.05) and SES (β = 0.42, P < 0.01) predicted attendance: only higher levels of IQ and SES were associated with higher levels of attendance. For the HF group, children from single-parent families had lower attendance rates than children from 2-parent families (β = –0.30, P < 0.05). No other variables predicted attendance. Contact Time in FLEX. In the LF group, only the single parent variable was more likely to accumulate higher amounts of FLEX contact time (β = 0.37, P < 0.01). In the HF group, no baseline variables predicted contact time. Part II: Who Benefited? Predictors of Outcome  For the HF group, when we controlled for baseline social competence and participation in the other program components, there was a significant positive relation between contact time in FLEX and social competence at year 3 (β = 0.27, P < 0.05). There was no significant relation between participation in the other 2 program components and outcome in social competence at year 3. Figure 2 displays the results of the model for the HF group.  Exploratory Analyses For the LF group, we originally predicted a positive relation between participation in FLEX and child social competence in year 3, but instead, we found a negative relation. We speculated that, as contact with the family advocate increased, parents developed more clarity about the extent of their child’s dysfunctional behaviour. This caused them to rate their child’s social competence less favourably at outcome. For the LF group, we predicted a positive relation between contact time in FLEX and child social competence in year 3 if we used only teacher ratings of child social competence. To test this reformulated hypothesis, we re-ran the above structural equation model with only teacher ratings. The teacher model fit the data well (n = 98; c2 = 18.63, df 14; P = 0.18). For the LF group, instead of the predicted positive relation, we found no relation between FLEX contact time and teacher-rated social competence at year 3. All the other significant relations were the same. To understand why FLEX worked differently for the different GAF groups, we examined possible differences in how participants used time in FLEX and in how they applied time in FLEX. The 4 FLEX domains used for both analyses were percentage of time focused on child personal, behavioural, and emotional issues; on parent personal, behavioural, and emotional issues; on child social functioning; and on parent social functioning. Results showed that, for the LF group, no domains predicted social competence at year 3. However, for the HF group, time spent on parents’ social functioning predicted child social competence at year 3 (F2,44 = 16.25, P < 0.001; R2 = 0.43, β = 0.25, P < 0.037). To better understand this relation for the HF group, we speculated that parent nurturance at year 2 mediated the relation between time spent on parents’ social functioning at year 1 and child’s social competence at year 3. We tested this hypothesized mediation effect in accordance with principles stipulated by Baron and Kenny (32) and Holmbeck (33): we examined the partial correlation coefficients between these 3 variables after controlling for baseline levels of child social competence. In the LF group, we did not find significant relations between any of these variables. However, we found several significant relations in the HF group. First, there was a significant positive relation between percentage of time spent on parents’ social functioning at year 1 and child social competence at year 3 (n = 39, r = 0.34, P = 0.028). Second, there was a significant positive relation between percentage of time spent on parents’ social functioning at year 1 and parent nurturance at year 2 (n = 42, r = 0.31, P = 0.040). Third, there was a significant positive relation between parent nurturance at year 2 and child social competence at year 3 (n = 42, r = 0.31, P = 0.04). Fourth, after we controlled for the effects of parent nurturance at year 2, there was a nonsignificant relation between percentage of time spent on parents’ social functioning at year 1 and child social competence at year 3 (n = 36, r = 0.17, P = 0.31). Thus increased parent nurturance accounted for the relation between time spent on parents’ functioning in year 1 and child social competence at year 3. DiscussionBaseline characteristics, participation rates, and social competence outcomes significantly differed between parent GAF groups. The HF group had higher SES and attended more family program sessions. However, this group did not differ from the LF group in rates of child attendance in the summer program or in the amount of time they used in the FLEX component. Although children in the HF parent group scored higher in social competence at both baseline and year 3 than did those in the LF parent group, there was no group difference in the rate of change over time. Thus the program may have been equally efficacious for both high- and low-GAF families. Below, we discuss the results for each individual intervention component. Family Program Summer Program FLEX Family Support Component Clinical Implications The findings suggest that SES and level of adaptive family functioning may be key determinants in tailoring interventions. LF families may struggle with adverse life events, marital discord, social isolation, unreliable transportation, and competing occupational commitments, all of which may determine which components of a program they choose to attend and how efficacious those components are. Although interventionists are in no position to change factors such as SES or single-parent status, they can tailor to particular subgroups interventions that are known to produce high levels of participation and benefit. Also of clinical significance is the finding that social skills of children with HF parents appeared to grow as an indirect result of time spent by program staff on parental social functioning. Consistent with the social learning perspective, parents’ involvement in social activities and the quality of their ensuing social support network may provide opportunities for modelling of prosocial behaviours that reinforce such behaviours in the child. The path that connects parent social functioning, nurturance of the child, and social competence is also compatible with attachment theory. As parents become more focused on improving social function, they may plan activities and participate in community social events that also include their child. Parent availability and responsivity to the child during such activities may serve to strengthen emotional parent–child bonds that in turn provide the impetus for gains in social competence. Even later in development, the quality of a mother’s interpersonal relationships, especially in her marriage and social network, predict the quality of adolescent friendships (35). Limitations Second, we assessed participation in terms of the rate at which children and parents attended the summer program and family program sessions. Although attendance rate provides important information about dosage received in a program component, it does not necessarily reflect the participant’s interest or engagement. Indeed, Orvell-Pallante and others found that the level of therapeutic engagement between program facilitator and parent was positively related to both rate and quality of participation (36). Further research is needed in this area. Third, while this study identified several informative relations between child, parent, and family characteristics and participation and outcome, the mechanisms underlying these relations require clarification. Fourth, the present findings address only the relation between participation and one outcome, specifically, children’s social competence. A different set of predictors may be relevant for other proximal and distal outcomes. Last, we examined only linear relations. The relations between participation and outcome may not be linear for some groups. For example, one subgroup may need a particular threshold of participation for changes to occur, or another subgroup may improve up to a point of participation, then decline. Future studies will examine these complex relations. Funding and SupportThis study was supported as part of a cooperative, multisite agreement (Prevention Intervention Studies on Predictor Variables by Developmental Stage) sponsored by the Substance Abuse and Mental Health Services Administration–Center for Substance Abuse Prevention (SAMHSA-CSAP). References1. Conduct Problems Prevention Research Group. Initial impact of the Fast Track prevention trial for conduct problems: I. The high-risk sample. J Consult Clin Psychol 1999;67:631– 47. 2. Conduct Problems Prevention Research Group. Evaluation of the first 3 years of the Fast Track prevention trial with children at high risk for adolescent conduct problems. J Abnorm Child Psychol 2002;30:19–35. 3. Abikoff H. Tailored psychosocial treatments for ADHD: the search for a good fit. J Clin Child Psychol 2001;30:122–5. 4. August GJ, Realmuto GM, Hektner JM, Bloomquist ML. An integrated components preventive intervention for aggressive elementary school children: The EARLY RISERS Program. J Consult Clin Psychol 2001;69:614–26. 5. August GJ, Hektner JM, Egan EA, Realmuto GM. The EARLY RISERS longitudinal prevention trial: examination of three-year outcomes in aggressive children with intent-to-treat and as-intended analyses. Psychol Addict Behav 2002;16(4 Suppl):S27–S39. 6. August GJ, Egan EA, Realmuto GM, Hektner JM. Four years of the Early Risers early-age-targeted preventive intervention: effects on aggressive children’s peer relations. Behav Ther 2003;34:453–70. 7. August GJ, Egan EA, Realmuto GM, Hektner JM. Parceling component effects of a multifaceted prevention program for disruptive elementary school children. J Abnorm Child Psychol 2003;31:515–27. 8. Ambruster P, Kazdin AE. Clinical, sociodemographic, and systems risk factors for attrition in a children’s mental health clinic. Am J Orthopsychiatry 1994;64:577–85. 9. Kazdin AE, Mazurick JL. Dropping out of child psychotherapy: distinguishing early and late dropouts over the course of treatment. J Consult Clin Psychol 1994;62:1069–74. 10. Miller GE, Prinz RJ. The enhancement of social learning family interventions for childhood conduct disorder. Psychol Bull 1990;108:291–307. 11. Wahler RG. The insular mother: her problems in parent-child treatment. Journal of Applied Behavioral Analysis 1980;13:207–19. 12. McMahon RJ. Diagnosis, assessment, and treatment of externalizing problems in children: the role of longitudinal data. J Consult Clin Psychol 1994;62:901–17. 13. Patterson GR, DeBaryshe BD, Ramsey E. A developmental perspective of antisocial behavior. Am Psychol 1989;44:329–33. 14. Reid JB. Prevention of conduct disorder before and after school entry: Relating interventions to development findings. Dev Psychopathol 1993;5:243–62. 15. Achenbach TM. Manual of the Teacher’s Report Form and 1991 profile. Burlington (VT): University of Vermont, Department of Psychiatry; 1991. 16. August GJ, Realmuto GM, Winters KC, Hektner JM. Prevention of adolescent abuse: targeting high-risk children with a multifaceted intervention model: The EARLY RISERS “Skills for Success” Program. Appl Prev Psychol 2001;10:135–54. 17. Pelham WE, Greiner AR, Gnacy EM, Hoza B, Martin L, Sams SE, Wilson T. Intensive treatment for ADHD: a model summer treatment program. In: Roberts M, editor. Model programs in child and family mental health. Mahwah (NJ): Erlbaum; 1996. p 193–214. 18. Pelham WE, Hoza B. Comprehensive treatment for ADHD; Intensive Summer Program and Follow-up. In: Hibbs ED, Jensen PS, editors. Psychosocial treatment research of child and adolescent disorders. Washington (DC): American Psychological Association; 1995. p 311–40. 19. Webster-Stratton C. Early intervention with videotape modeling: programs for families of children with oppositional defiant or conduct disorder. In: Hibbs ED, Jensen PS, editors. Psychosocial treatments for child and adolescent disorders: empirically based strategies for clinical practice. Washington (DC): American Psychological Association; 1996. p 435–74. 20. Dunst CJ, Trivette CM, Deal AG. Resource-based family-centered intervention practices. In: Dunst CJ, editor. Supporting and strengthening families. Volume 1. Methods, strategies, and practices. Cambridge (MA): Brookline Books; 1994. p 140–51. 21. Werthamer-Larsson L, Kellam SG, Wheeler L. Effect of first-grade classroom environment on shy behavior, aggressive behavior, and concentration problems. Am J Community Psychol 1991;19:585–602. 22. Hollingshead AA. [Four-factor index of social status, 1979]. Located at: Yale University, New Haven, Connecticut. 23. Shelton KK, Frick PJ, Wooten J. Assessment of parenting practices in families of elementary school-age children. J Clin Child Psychol 1996;25:317–29. 24. Gorman-Smith D, Tolan PH, Zelli A, Huesmann LR. The relation of family functioning to violence among inner-city minority youths. J Fam Psychol 1996;10:115–29. 25. Tolan PH, Gorman-Smith D, Huesmann LR, Zelli A. Assessment of family relationship characteristics: a measure to explain risk for antisocial behavior and depression among urban youth. Psychol Assess 1997;9:212–23. 26. Abidin RR. The Parenting Stress Index. 2nd ed. Charlottesville (VA): Pediatric Psychology Press; 1986. 27. American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4th ed. Washington (DC): American Psychiatric Association; 1994. 28. Patterson DA, Lee M. Field trial of the global assessment of functioning scale-modified, Am J Psychiatry 1995;152:1368–88. 29. Reynolds CR, Kamphaus RW. BASC—Behavioral Assessment System for Children. Manual. Circle Pines (MN): American Guidance Service; 1992. 30. Harter S. Self-Perception Profile for Children. Denver (CO): University of Denver; 1985. 31. Arbuckle JL. AMOS 3.6. Chicago (IL): Smallwaters; 1999. 32. Baron RM, Kenny DA. (1986). The Moderator-mediator variable distinction in social psychological research: conceptual, strategic and statistical considerations. J Pers Soc Psychol 1986;51:1173–82. 33. Holmbeck GN. Toward terminological, conceptual, and statistical clarity in the study of mediators and moderators: samples from the child-clinical and pediatric psychology literatures. J Consult Clin Psychol 1997;65:599–610. 34. Entwisle D, Alexander K, Olson L. Children, schools, and inequality. Boulder (CO): Westview Press; 1997. 35. Markiewicz D, Doyle AB, Brendgen M. The quality of adolescents’ friendships: associations with mothers’ interpersonal relationships, attachments to parents and friends, prosocial behaviors. J Adolesc 2001;24:429–45. 36. Orell-Valente JK, Valente E, Laird RD, Conduct Problems Prevention Research Group. If it’s offered, will they come? Influences on parents’ participation in a community-based conduct problems prevention program. Am J Community Psychol 1999; 27:753–83. Author(s)Manuscript received August 2003, revised, and accepted January 2004. 1. Associate Professor, Division of Child and Adolescent Psychiatry, University of Minnesota Medical School, Minneapolis, Minnesota. 2. Professor, Division of Child and Adolescent Psychiatry, University of Minnesota Medical School, Minneapolis, Minnesota. 3. Research Scientist, Social Development Research Group, University of Washington, Seattle, Washington. Address for correspondence: Dr GM Realmuto, F256/2B West, 2450 Riverside Avenue, Minneapolis, Minnesota 55454 e-mail: realm001@umn.edu
1 | 2
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||