Editorial Credits/ Crédits éditorials

Subscription Rates /Prix d'abonnements

Advertising Rates / Tarifs publicitaires (PDF)

Review Paper
Panic in the Emergency Room
Patrick Lynch, Kim M Galbraith
(PDF)

Medication Strategies in Childhood Aggression: A Review
Lindley Bassarath
(PDF)

Original Research
Old and Homeless: A Review and Survey of Older Adults Who Use Shelters in an Urban Setting
Vicky Stergiopoulos, Nathan Herrmann
(PDF)

Correlates of Therapeutic Response in Panic Disorder Presenting With Palpitations: Heart Rate Variability, Sleep, and Placebo Effect
Brian Baker, Yaariv Khaykin, Gerald Devins, Paul Dorian, Colin Shapiro, David Newman
(PDF)

Randomization in a Substance Abuse Treatment Study: Participants Who Consent vs Those Who Do Not
Peter Seraganian, Thomas G Brown, Jacques Tremblay
(PDF)

Fright (Effroi) and Other Peritraumatic Responses After a Serious Motor Vehicle Accident: Prospective Influence on Acute PTSD Development
Guillaume Vaiva, Alain Brunet, François Lebigot, Virginie Boss, François Ducrocq, Patrick Devos, Philippe Laffargue, Michel Goudemand
(PDF)

Children With Psychiatric Disorders: The Al Ain Community Psychiatric Survey
Valsamma Eapen, Mona Essa Jakka, Mohammed T Abou-Saleh
(PDF)

A Follow-Up Study of Persons Found Not Criminally Responsible on Account of Mental Disorder in British Columbia
James D Livingston, Derek Wilson, George Tien, Lynda Bond
(PDF)

Brief Communication
Anxiety-Related Visits to Ontario Physicians Following September 11, 2001
Peter C Austin, Muhammad M Mamdani, Benjamin TB Chan, Elizabeth Lin
(PDF)

Book Reviews
(PDF)

Impact of Narcissism—The Errant Therapist on a Chaotic Quest.
Reviewed by
Douglas H Frayn, MD, FRCPC

Living Longer Depression Free.
Reviewed by
Matt Robillard, MD, FRCPC

Hyperactivity and Attention Disorders of Childhood. 2nd ed.
Reviewed by
Margaret Weiss, MD, PhD, FRCP

Dementia.
Reviewed by
Andrew Wiens, MD

Letters to the Editor
(PDF)

Risperidone Liquid Treatment of Acute Mania

Methylphenidate and the Cytochrome P450 System

Reply: Methylphenidate and the Cytochrome P450 System

Antiaggressive Action of Combined Risperidone and Quetiapine in a Patient With Schizophrenia

Ultrarapid Response to an Antidepressant: A Clue to Bipolarity?

Developmental Alcohol Exposure, Circadian Rhythms, and Mood Disorders

Attouchements sexuels pratiqués par un adolescent substitué en testostérone

Dichotomization and Manipulation of Numbers

Reply: Dichotomization and Manipulation of Numbers

Original Research

Randomization in a Substance Abuse Treatment Study: Participants Who Consent vs Those Who Do Not

Peter Seraganian, PhD¹, Thomas G Brown, PhD², Jacques Tremblay, MD³

A core issue in the research–practice gap noted in the substance abuse treatment domain is the representativeness of patients under experimental scrutiny with respect to those in the clinical treatment setting (1). For example, a seminal research study such as Project MATCH—unquestionably the largest randomized multisite study of patient treatment matching for alcoholism ever conducted—represented a milestone in the promotion of internal validity (2). Nevertheless, a deliberate choice was made to focus on a sample of pure, relatively stable subjects with alcoholism, which differed with respect to both the multiple abuse patterns, as well as the instability that is commonly exhibited by patients now presenting for treatment. Consequently, despite the methodological elegance of Project MATCH, it may have yielded data that, in terms of addressing global policy and clinical management decisions in the mental health domain, have questionable utility. Indeed, Humphreys and Weisner have argued that the use of stringent exclusion criteria in alcohol-treatment outcome studies can compromise findings in such a way that a parallel stream of outcome research with few exclusion criteria is needed to improve generalizability to vulnerable populations (3). The enhanced rigour that the promotion of design efficacy can bestow may come at a stiff price to external validity (4).

One strategy that is used to increase the external validity of treatment evaluation studies broadens participant inclusion by studying the now ubiquitous clients who abuse multiple substances (5,6). Still, it does not necessarily follow that the effort to be more inclusive with respect to substance abuse profile will satisfactorily clarify the generalizability issue. Another source of potential bias is patient refusal to submit to randomization. These individuals are, under normal circumstances, excluded from most randomized trials. However, in a study of matching different after-care approaches to patient attributes, approximately 15% of clients who refused randomization into experimental after-care treatments did subsequently accept to be followed up (5). The individuals who chose not to participate in a randomly selected experimental after-care intervention were making a choice about their preferred after-care experience. Thus, their inclusion in follow-up analysis closely represents the approach taken in a comprehensive cohort design (7). Arguably, these individuals are members of a distinct, inadequately studied subsample of all substance abuse patients. Their study could extend our understanding of the external validity of randomized clinical trials in substance abuse treatment research.

The present study compares patients who consented to full participation in the research project (consenting group [CG]) with those unwilling to undergo randomization to different experimental after-care treatments but who, nevertheless, underwent the same 4 posttreatment follow-up assessments (nonconsenting group [NG]) over a 6-month period. We address 3 specific questions: First, at study intake, prior to exposure to intensive treatment, how similar are CG patients to NG patients on demographic, substance abuse, and psychological indices? Second, how well do CG patients fare posttreatment, compared with NG patients? Third, over the time course from intake to 6-month follow-up, how do key demographic, substance abuse, and psychological indices fluctuate across the 2 groups? Answers to each of these questions could help shed light on the issue of comparability of research participants with the substance abuse population and on whether potential differences impact upon outcome.

Method

Participants
At 3 Montreal-region residential treatment centres, designated members of the treatment staff approached male and female adult patients, aged 18 years and over, who entered treatment for alcohol and drug abuse. These patients were informed about the existence of the research project, were asked to read, and if willing, were requested to sign an informed consent that attested to their willingness to participate. Patients who provided written informed consent were eligible for study participation if they met the following inclusion criteria: diagnostic criteria for psychoactive substance abuse or dependence, as classified by a structured clinical interview for DSM-III-R (SCID); no severe organic brain syndrome or severe psychosis; ability to read and write in either French or English at least at a grade 5 level; and residency within a 50-km radius of Montreal.

Sites
Participants were recruited from among patients at 3 treatment centres in the Montreal region: 1) Pavillon Foster, a publicly funded 20-bed inpatient and 12-place outpatient facility that serves English-speaking patients from across Quebec; 2) Maison Jean Lapointe, a private, nonprofit, 42-bed inpatient facility that serves francophone patients in central Montreal; and 3) Le Virage, a provincially funded 30-bed, in- and outpatient rehabilitation facility that serves francophone patients from Montreal’s suburban and rural, south-shore region. The 3 treatment centres share certain attributes: a multimodal treatment orientation that stresses heightened awareness of the negative impact of substance abuse, personal autonomy from dependence on psychoactive substances, improved social adjustment, and after-care involvement.

After-Care Treatments
Patients in the CG were assigned at random to participate in either structured relapse prevention (8) or in 12-step facilitation (9) after-care programs, in addition to the usual after-care programs offered by their treatment centre. Two main features were common to both experimental after-care programs. First, both interventions were provided in a 10-week, 90- minute, closed-group format, with groups comprising 4 to 8 participants. Second, highly trained and regularly supervised counsellors delivered both interventions using a manualized format. Those in the NG received only the standard after-care provided by the attended treatment centres. Standard after-care treatment involved approximately 10 to 13 weeks of weekly group meetings, conducted by and held at the participating site. One or 2 therapists would conduct group sessions with clients, providing psychosocial support for problems related to lapse or relapse, craving, social reintegration (for example, employment and family), health and psychological adjustment, and access to ancillary services. Although these sessions were strongly recommended, clients were free to attend when they chose.

Sociodemographics
Sociodemographic and other pertinent information (that is, age, education, and marital status) were gathered via specific portions of the Addiction Severity Index (ASI).

Alcohol and Drug Use
Timeline Follow-Back (TLFB) presents patients with a calendar (essentially an aid to recall) and asks that they recall instances of drinking or substance use on a daily basis over the past 90 days (10). The quantity and frequency data that this technique yields have been found to minimize the tendency to underreport substance use (11). Days of substance use and days of abstinence prior to first relapse (that is, number of days prior to 3 or more consecutive days of use) were dependent variables drawn from this instrument.

Substance Abuse and Psychosocial Functioning
The ASI is a semistructured interview protocol that has been found valid and reliable in assessing a spectrum of addiction-related behaviours and consequences in both evaluative and matching investigations (12,13). In addition to the 2 composite scores specifically related to the severity of alcohol and drug use over the past 30 days, 5 other subscales of psychosocial functioning were derived: medical, employment, legal, family, and psychiatric severity.

Diagnostic Classification
Diagnoses of substance use were made via the nonpatient version of the Structured Clinical Interview for the DSM-III-R (SCID-NP) (14). This structured interview protocol consists of items that are keyed to the major psychiatric, diagnostic features contained in the DSM-III-R (15). The structured interview protocol has been well standardized and yields data that are both valid and reliable if subjects answer candidly. For this study, only those sections that provide diagnostic classification of substance abuse or dependence disorders (Axis I) were administered.

Psychological Status
Psychological status was assessed by employing the Symptoms Checklist-90 (SCL-90) (16), which is frequently used as a simple, yet effective, screening device for detecting psychological disturbance in previous treatment studies of alcohol and drug abusers. This self-administered, paper-and-pencil instrument provides 9 clinical scales, as well as 3 general indices of symptom severity. The Global Severity Index (GSI), an aggregate score that considers scores on all 9 clinical scales, was used as a dependent variable in subsequent analyses.

Procedures
Within 2 days following admission into intensive treatment, patients were approached by a designated member of the treatment staff, were informed about the existence of the research project, and were asked to read and, if willing, to sign an informed consent attesting to their willingness to participate at either the CG or NG level. The Concordia University and Douglas Hospital Ethics Review Committees accepted the informed consent form.

Other than a brief description of the study, including the 2 levels of participation, the informed consent indicated that reimbursements of $10 and $20 would be provided for completion of the post–after-care and 6-month follow-up assessment sessions, respectively. If the patient was in withdrawal or had used psychoactive substances during the week prior to entering treatment, study participation was delayed for up to 1 week to reduce data contamination from the effects of withdrawal.

Multidimensional assessment of all participants occurred 4 times: at intake into intensive treatment (T0), following completion of intensive treatment (T1), following the 10-session after-care program (T2), and 6 months following completion of intensive treatment (T3). PhD psychology candidates conducted each of these assessment sessions, which took approximately 2 hours to complete.

Major Analytic Strategies
To compare groups on continuous sociodemographic measures, t-tests were used, whereas the chi-square statistic was employed for categorical (for example, sex and substance abuse patterns) comparisons. Four forward (Wald P2) logistic regressions were performed for each assessment (T0 to T3) to determine whether any of the ASI scales, the SCL-90 GSI, or the number of substance use days in the past 90 days could predict group membership better than chance, a statistical strategy chosen owing to its robustness with respect to distribution anomalies (17). The Kaplan–Meier survival analysis clarified differences in delay to relapse. Given the exploratory nature of this secondary analysis of data from a previous study (5), family-wise alpha was relaxed to a more liberal 0.10. Thus, using a Bonferroni correction, predictive significance of individual regressions was inferred at α < 0.025.

Results

Refusal to Participate and Attrition
Of the individuals who were approached, 47 refused participation in the study. The main reasons included lack of interest (27.7%), after-care sites being too far away (25.5%), and lack of time or inconvenience (21.2%). Statistical comparisons between these individuals and the final sample on age, marital status, employment status, ethnic background, and sex failed to yield any significant differences.

Likewise, 87 individuals were lost from follow-up after randomization. Those lost to attrition were significantly younger (mean 35.8, SD 9.0 vs mean 38.3, SD 9.5), were less educated (mean 11.4, SD 3.0 vs mean 12.2, SD 2.7), had less time at their current job (mean 6.9, SD 5.7 vs mean 9.2, SD 8.2), had poorer employment functioning on the ASI Employment Scale (mean 0.64, SD 0.30 vs mean 0.55, SD 0.30), reported more previous treatments for alcohol problems (mean 0.93, SD 2.7 vs mean 0.34, SD 0.72), and had spent less money on drugs in the previous 30 days (mean 156.7, SD 437.4 vs mean 417.7, SD 1458.3) than those retained in the study (P < 0.05) .

Consenting Participants vs Nonconsenting Participants
Table 1 presents the demographic and selected substance abuse statistics for patients in the CG (n = 133) and for those in the NG (n = 21). Although similar in age, ethnicity, marital, and educational status, the 2 groups differed. Patients in the CG were more likely to be women (c² = 4.6, df 1, P < 0.05), had fewer years at their current job (t = 2.6, df 148, P < 0.01), worked less in the past 6 months (t = 2.1, df 151, P < 0.05), and reported almost 3 times as many years of cocaine abuse (t = 2.2, df 151, P < 0.05).

Figure 1 Mean composite scores on the alcohol Addiction Severity Index (ASI) for the consenting group and the nonconsenting group

T0 = at intake into intensive treatment; T1 = following completion of intensive treatment; T2 = following the 10-session after-care program;T3 = 6 months following completion of intensive treatment.

Figure 2 Mean composite scores on the drug ASI for the consenting group and the nonconsenting group

See Figure 1 for definitions of T0 to T3

Figures 1 through 3 present ASI composite scores on alcohol, drug, and psychiatric symptom severity at T0 to T3, respectively. Logistic regression at T0 failed to reveal any significant association between intake characteristics and group membership at the designated alpha level. At T1, however, the ASI drug entered into the regression to significantly improve prediction of group membership over chance: c² = 7.62, df 1, P = 0.006, Nagelkerke r² = 0.10, where the CG had more severe drug problems than did the NG. No significant predictors were uncovered at T2, but at T3, a significant 3-variable predictive solution was uncovered (c² = 20.2, df 3, P < 0.001, r² = 0.22), with the combination of greater ASI drug and psychiatric severity but less alcohol severity predicting membership in the CG.

Figure 4 presents survival curves in delay to first relapse (that is, time to first substance use on 3 or more consecutive days). Although not attaining significance at corrected alpha, these curves hint at a tendency for the NG to resist relapse longer than those in the CG, especially in the first 100 days of the 6-month follow-up period. However, this trend drops off quickly after this point, and at the end of the follow-up period, survival rates (that is, no relapse) for the 2 groups were 69% and 62% for NC and CG, respectively. Abstinent days in the last 90 taken at intake vs 6-month follow-up are quite similar, with the number of abstinent days at intake and at 6-month follow-up equal to 44.3 and 78.7 days, respectively, for the CG, and 43.5 and 80.5 days, respectively, for the NG.

Figure 3 Mean composite scores on the psychiatric Addiction Severity Index (ASI) for the consenting group and the nonconsenting group

See Figure 1 for definitions of T0 to T3

Figure 4 Survival curves of time to relapse for the consenting group and the nonconsenting group

See Figure 1 for definitions of T0 to T3

Discussion

Although not altogether consistent, a pattern emerges at intake, over treatment, and at follow-up. Patients in the CG are more likely to be women, exhibit longer drug abuse histories, have less occupational stability, and tend not to do as well during recovery with their drug and psychiatric severity as those in the NG. These findings are therefore consistent with those reported elsewhere: individuals consenting to randomization into an experimental protocol appeared to be less stable and to exhibit greater addiction severity or more widespread psychosocial difficulties at intake to treatment (18,19). These observations may reflect a desire in more distressed patients in the CG, or more “desperate” according to Strohmetz and others, to avail themselves of the extended, structured treatment regimens that participation in the experimental after-care programs offer (20). Sex differences seen in this study are noteworthy. Women show a greater tendency than do men to seek help for health matters, but not in specialized substance abuse treatment settings (21). This finding indicates that once women have overcome the personal, logistic, social, and systemic barriers often encountered by women who grapple with substance abuse (22), they will seek more specialized services, if offered.

Given that patients in the 2 groups differed somewhat to begin with, the issue of posttreatment outcome becomes more problematic. Nevertheless, it appears that patients in the CG did not manage as well at posttreatment and at the 6-month follow-up in terms of drug and psychiatric severity—a finding also hinted at by the nonsignificant survival analysis of days prior to first relapse. A study that compared cocaine abusers who were randomly assigned to either inpatient or outpatient rehabilitation with patients who were allowed to self-select in these treatment settings has reported that, among randomized patients, those who were worse off at baseline exhibited greater improvement by 3-month follow-up (19). Our data are not consistent with this report; rather, they suggest that the more disturbed patients who consented to randomization may continue to fare more poorly at outcome. Clear differences exist across studies, especially the discrepancies in terms of cocaine vs multiple substance abuse, as well as the randomization of intensive treatment vs after-care. More systematic analysis of this issue would seem to be called for—particularly in light of DeLeon’s contention (23) and supported by our previous work (5)—in that patient preference for treatment may be a critical factor in outcome success.

For several ASI subscales, a consistent trend in group differences was seen across the 4 assessment sessions from intake to 6-month follow-up. This stability in differences between groups, even during the onslaught of change associated with intensive treatment for substance abuse and in the 6 months that follow, speaks to the robustness of these indices. At the same time, a caveat with respect to the relatively small sample size of the NG is in order. Possibly, other differences between the groups could have been detected if the NG, and subsequently statistical power, had been larger.

Although this study succeeded in documenting the outcome of individuals who might otherwise be excluded from the study because of the refusal to undergo randomization, it did not succeed in following 47 individuals who refused to participate in the study or 87 others who were lost at follow-up through attrition. The former group was not found distinguishable from the final sample, based upon the somewhat limited data available to us. In contrast, the group lost through attrition was younger, less stable, more psychosocially impaired, and had more previous treatments for alcohol problems, compared with those who were followed up successfully. Thus, this latter, more problematic group seemed to more closely resemble the study dropouts who are frequently described in the treatment evaluation literature (24), although their actual outcome remains unknown. Overall, these findings suggest that the characteristics of participants who are lost from experimental scrutiny—and inversely, the characteristics of those who remain—may be influenced by the circumstances underlying their loss; that is, whether through researcher exclusion (that is, rejection owing to refusal to accept randomization) or through patient factors (that is, refusing recruitment or attrition at follow-up).

Adoption of the comprehensive cohort design was not an original feature of the study methodology. Indeed, it was only during a small pilot study to fine-tune the methodology that we found that some patients who chose not to undergo randomization would participate in the repeated-measures assessment protocol. Clearly, opportunistic alteration of the protocol after a research project is underway is less desirable than a prospective approach. Nevertheless, contrary to the view held by other investigators (3) who propose parallel streams of research with either stringent or minimal exclusion criteria, the comprehensive cohort design seems capable of credibly shedding some light on the generalizability issue within a single-study methodology.

Clinical Implications
The present study provides insight into what impact the loss of potential study participants may have on our ability to generalize from treatment effectiveness data to clinical decision making. Recruiting a naturalistic sample increases the likelihood of external validity. However, we discovered that 2 frequently held positions—that patients who do not fully comply with experimental protocols should be considered as treatment failures (24) and that their omission from analysis would result in an overly optimistic estimation of the real impact of experimental psychosocial therapies (3)—were not supported. Accordingly, these findings diminish the significance of dropouts to the utility of treatment-outcome data when employing naturalistic samples.

A parallel may also be drawn with respect to professional treatment recommendations. Clinicians often confront substance abuse clients who refuse to comply with all suggested treatments. There may be a tendency to interpret this situation as a predictor of poorer outcome. To the contrary, our findings suggest that patient failure to fully comply, especially with respect to posttreatment strategies, does not necessarily jeopardize their prognosis.

Funding and Support

This study was funded by the National Health and Research Development Program, Health Canada, and the Quebec Council of Psychosocial Research (career award for Dr Brown).

References

1. Wells KB. Treatment research at the crossroads: the scientific interface of clinical trials and effectiveness research. Am J Psychiatry 1999;156:5–10.

2. Project MATCH Research Group. Matching alcoholism treatments to client heterogeneity: Project Match post treatment drinking outcomes. J Stud Alcohol 1997;58:7–29.

3. Humphreys K, Weisner W. Use of exclusion criteria in selecting research subjects and its effect on the generalizability of alcohol treatment outcome studies. Am J Psychiatry 2000;157:588–94.

4. Slade M, Priebe S. Are randomized controlled trials the only gold that glitters? Br J Psychiatry 2001;179:286–7.

5. Brown TG, Seraganian P, Tremblay J, Annis H. Matching substance abuse after-care treatments to client characteristics. Addict Behav 2002;26:585–604.

6. Miller WR, Bennett ME. Treating alcohol problems in the context of other drug abuse. Alcohol Res Health 1996;20:118–23.

7. Jadad A. Randomized controlled trials. London: BMJ Books; 1998.

8. Annis HM, Davis CS. Relapse prevention. In: Hester RK, Miller WR, editors. Handbook of alcoholism treatment approaches. New York: Pergamon Press; 1989. p 170–82.

9. Nowinski, J, Baker S. The twelve-step facilitation handbook. New York: Jossey Bass; 1992.

10. Sobell LC, Maisto SA, Sobell MB, Cooper AM. Reliability of alcohol abusers’ self-reports of drinking behavior. Behav Res Ther 1979;17:157–60.

11. Babor TF, Brown J, Del Boca FK. Validity of self-reports in applied research on addictive behaviors: fact or fiction? Behav Assess 1990;12:5–31.

12. McLellan AT, Luborsky L, Woody GE, O’Brien CP. An improved diagnostic evaluation instrument for substance abuse patients: the Addiction Severity Index. J Nerv Ment Dis 1980;168:26–33.

13. McLellan AT, Woody GE, Luborsky L, O’Brien CP, Druley KA. Increased effectiveness of substance abuse treatment: a prospective study of patient– treatment “matching.” J Nerv Ment Dis 1983;171:597–605.

14. Spitzer RL, Williams JB, Gibbon M, First, MB. User’s Guide for the Structured Clinical Interview for DSM-III-R. Washington (DC): American Psychiatric Association Press; 1990.

15. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders (DSM-III- R). Washington (DC): APA Press; 1987.

16. Derogatis LR, Lipman RS, Covi L. The SCL-90: an outpatient psychiatric rating scale. Psychopharmacol Bull 1973;9:13–28.

17. Tabachnick BG, Fidell LS. Using multivariate statistics. 4th ed. Boston: Allyn and Bacon; 2001.

18. Rychtarik RG, McGillicuddy NB, Connors GJ, Whitney RB. Participant selection biases in a randomized clinical trial of alcoholism treatment settings and intensities. Alcohol Clin Exp Res 1998;22:969–73.

19. McKay JR, Alterman AI, McLellan AT, Boardman CR, Mulvaney FD, O’Brien CP. Random versus nonrandom assignment in the evaluation of treatment of cocaine abusers. J Consult Clin Psychol 1998;66:697–701.

20. Strohmetz DB, Alterman AI, Walter D. Subject selection bias in alcoholics volunteering for a treatment study. Alcohol Clin Exp Res 1990;14:736–8.

21. Smith L. Help seeking in alcohol-dependent females. Alcohol Alcohol 1992;7:3–9.

22. Health Canada. Best practices treatment and rehabilitation for women with substance use problems. Ottawa: Health Canada; 2001.

23. DeLeon G. Commentary: reconsidering the self-selection factor in addiction treatment research. Psychol Addict Behav 1998;12:71–7.

24. Caetano R. Editorial: non-response in alcohol and drug surveys: a research topic in need of further attention. Addiction 2001;96:1541–5.

Author(s)

Manuscript received January 2003 and accepted February 2003.

1. Director, Applied Alcohol and Drug Addiction Research Unit, Department of Psychology, Concordia University, Montreal, Quebec.

2. Director, Addiction Research Program, Douglas Hospital, Verdun, Quebec; Head of Research, Pavillon Foster Addiction Treatment Center, Montreal, Quebec; Assistant Professor, Department of Psychiatry, McGill University, Montreal, Quebec.

3. Adjunct Assistant Professor, Concordia University, Department of Psychology, Montreal, Quebec; Faculty Lecturer, Department of Psychiatry, McGill University, Montreal, Quebec.

Address for correspondence: Dr TG Brown, Addiction Research Program, Douglas Hospital Research Center, McGill University, 6875 LaSalle Blvd, FBC-1, Verdun, QC H4H 1R3

e-mail: brotho@douglas.mcgill.ca

1 | 2