|
Presence of Clinically Relevant Elements.
By summing the presence of 20 elements in these published studies
(see Table
2a,b), we derived a secondary measure of methodological quality.
We included counts of these elements to help readers decide whether
study results may be generalized to particular samples and settings.
We identified these as clinically important elements by consensus;
however, the specific impact of these characteristics on validity
has not been supported by research evidence. Because no evidence
exists on the relative weights of each element, the count of each
should not be regarded as a quality score.
The following is a brief description of these elements and their
possible theoretical effects on the validity and applicability of
research on the treatment of ADHD.
Sampling Issues. The number of eligible patients
constitutes the sampling frame of subjects available for study.
If more subjects than needed are eligible, probability sampling
should be used to identify study participants, so that the study
findings can be generalized to a defined group. The investigator
should also report on the response to enlistment, including both
the number of subjects who decline and those who agree to participate.
The ratio of these 2 groups is a good indicator of the level of
acceptability associated with the treatment options. In studies
where a high percentage of subjects decline, it is possible that
treatment is only applicable to a very small group of patients with
distinctive features.
Number of Patients Randomized and Analyzed. The proper
denominator for evaluating treatment effectiveness is the number
of patients randomized. Subject withdrawals and losses to follow-up
often lead to fewer subjects for analysis. The magnitude and distribution
of these losses across treatment groups will have important implications
for understanding treatment acceptability and effectiveness. Without
this information, study results cannot be interpreted.
Patient Characteristics. Intervention outcomes may
vary as a function of age, sex, intellectual abilities, and family
characteristics. In addition, findings in a specific ethnic group
may not apply to others.
Treatment Setting. The treatment setting provides
the context for an investigation. Many features are embedded in
context (for example, the investigators’ professional affiliations
and the facility’s reputation, acceptability, and accessibility).
These may affect both the characteristics of the subjects within
a study catchment area and the acceptability and effectiveness of
treatment.
Inclusion and Exclusion Criteria. These criteria
define the “target population,” or the subjects for whom
the study results are intended to apply. In the absence of these
criteria, it is very difficult, if not impossible, to assess the
limits of generalizability for a particular study.
Identification of the Primary Outcome. If the primary
outcome is not specified a priori (or at all) and all outcomes are
treated alike, authors are at increased risk of highlighting those
with the most striking results. In addition, the more outcomes analyzed,
the greater the risk of finding false-positive, statistically significant
results, merely by chance. Identifying the primary outcome is also
an essential step to estimating the study’s power to detect
true-negative results.
Diagnosis-Related Issues. Diagnostic criteria for
ADHD have evolved over time. Various diagnostic models may well
define samples that differ in natural history, severity, comorbidity,
ADHD subtype, and response to treatment (43).
Comorbid Conditions. Information on the presence
of comorbid disorders is important to judge the generalizability
of results to a particular clinical setting. In addition, comorbid
disorders may be associated with different responses to treatment
or different levels of treatment adherence.
Individual(s) Who Made the Diagnosis. In general,
informants show low agreement on the presence of the core ADHD symptoms.
Current diagnostic models require evidence that symptoms are pervasive.
Relying on a single informant may generate biased study samples
that differ in severity or comorbidity from those generated by other
informants. For example, teacher-rated ADHD is more strongly associated
with academic achievement than is parent-rated ADHD (44).
Sample Origin. Subjects for treatment studies may
come from clinic (inpatient and outpatient) or nonclinic populations,
or both. The sample’s origin is likely to have a strong bearing
on the severity and complexity of the cases being treated, and on
their prognosis.
Treatment Fidelity and Monitoring. Fidelity reflects
the extent to which treatment is delivered correctly. Fidelity also
ensures that interventions which are not part of the treatment protocol
are not administered inadvertently. It can be enhanced in several
ways, including training professionals who administer treatment,
conducting treatment according to treatment manuals, self-monitoring
of treatment administration, and conducting independent adherence
checks. Studies that monitor and report fidelity allow readers to
determine whether potentially effective treatments were fairly tested.
Further, the treatment manuals developed to support clinical trials
can help to disseminate effective treatments to community practitioners.
Treatment Compliance. Compliance reflects the extent
to which patients correctly carry out treatment plans. Compliance
might require the timely administration of correct doses of medication,
the completion of parent-training homework projects, or the consistent
application of classroom behaviour-management strategies.
Availability of Baseline Test Scores in the Report.
Baseline test scores permit researchers to identify atypical samples
(for example, those characterized by severity or comorbidity) and
to evaluate the comparability of the subjects in various study groups
at the outset.
Estimation of Effect Size
We assessed the feasibility and validity of conducting a metaanalysis
of these treatment outcome data. For a given treatment, we identified
few studies of sufficient methodological quality that employed a
similar outcome measure. Moreover, based on the means and standard
deviations, outcome scores were not often normally distributed,
and we had no access to individual patient data. Many studies had
few subjects. Under these circumstances, metaanalysis can result
in erroneous conclusions.
For descriptive purposes only, we calculated the size of medication
treatment effects in 2 cases (core ADHD behaviour and reading) where
there was some consistency in the outcome measure (Conners’
Rating Scale and the Wide Range Achievement Test [WRAT] reading).
It should be noted that these studies used many different versions
of the Conners’ scale. Effect size was calculated using the
following formula:
| effect
size = |
placebo
mean at baseline – active mean at end of study
placebo standard deviation at baseline
|
|
|
|
Results
Literature Search Yield
Electronic databases searches, reference list reviews, and referrals
from experts yielded 2402 citations. Of 521 potentially eligible
articles (based on the citation information) we obtained hard copies
of 519 (2 are unobtainable due to incorrect indexing). Of these,
429 described comparative studies, while 90 described noncomparative
studies. Of the 429 comparative studies, 293 were RCTs, and 136
were non-RCTs. Of these, we identified 14 RCTs that met criteria
for long-term treatment studies (22–35). Studies using the
same subjects were combined and considered as a single data set
(45).
General Characteristics
The total number of subjects in these studies was 1379, of which
579 (42%) were from a single study (the MTA study [33]) (Table
1). Four studies included 10 subjects or fewer per treatment
arm. Five studies involved treatment of 26 weeks or longer, of which
3 involved treatment of 52 weeks or longer. The most frequently
evaluated medication was methylphenidate (MPH, 9 studies) followed
by dextroamphetamine (DEX, 3 studies), lithium (1 study), imipramine
(1 study), and thioridazine (1 study). Eight studies evaluated nonpharmacologic
interventions and combined interventions: child therapy of various
types (3 studies), cognitive-behavioural therapy (2 studies), combined
psychosocial treatments (1 study), supportive therapy (1 study),
parent training (1 study), and EEG biofeedback (1 study). Six studies
permitted a comparison of the effects of combined pharmacologic
and behavioural intervention. Twenty-five different outcomes were
measured in the 14 studies (Table
1). The most frequently measured outcomes were the core and
global symptoms of inattention, hyperactivity, and impulsivity (11
studies), followed by social behaviour (9 studies), academic attainment
(8 studies), and internalizing symptoms (5 studies). Twenty-six
different tests were used to measure the outcomes. Conners’
Rating Scale (7 studies) was the most frequently used test, but
the version of the questionnaire varied across studies. Only 6 studies
used both parents and teachers as informants to assess outcome.
Few studies included direct observations of child behaviour as outcome
measures. Seven studies were funded by industry.
The main reasons for suboptimal quality scores were as follows:
the method of randomization was not described (13 studies); there
was no report of double blinding (3 studies), methods to achieve
double blinding were not described (9 trials), and withdrawals and
dropouts were poorly described (9 trials). Twelve trials did not
mention concealment of allocation to study groups. In the MTA study,
primary outcome measures (parent and teacher behavioural ratings)
were not blind to the treatment group.
Only the MTA study included information on all 20 clinically relevant
elements selected a priori for extraction from the articles. One-half
the studies did not describe the number of eligible subjects, and
less than one-quarter of the trials mentioned ethnicity of participants.
Three of the studies did not report or were vague about inclusion
criteria, and 5 trials did not report the exclusion criteria. The
primary outcome of interest to the researchers was not specified
in 13 of the 14 studies. No study involved preschool children or
adults. Family characteristics were not mentioned in 8 studies.
Treatment fidelity was not described in 4 studies, and compliance
was not measured in 6 studies.
Effect on ADHD Behaviour
Seven of the 11 studies reported treatment effects on behaviour
(core or global symptoms). Stimulant medication (MPH or DEX) was
superior to placebo in 4 studies and not superior to placebo in
2 studies. MPH was superior to thioridazine in 1 study and equivalent
to imipramine in 1 study. Four studies permitted comparison of MPH
and a nonpharmacologic intervention, or of MPH and combined pharmacologic
and behavioural therapy. The MTA study reported that medication
management was equivalent to combined medication and behavioural
intervention; both were superior to behaviour therapy alone or to
assessment and referral to care in the community. No treatment effect
was noted on directly observed behaviour . In each of the 5 studies
for which we calculated effect size, the effect size of active MPH
was greater than that of the control condition. Typically, the effect
size for MPH was 50% to 100% greater than that of the control treatment
(data available at http://hiru.mcmaster.ca/ADHD/effects). Biofeedback
was superior to no treatment in 1 study, and cognitive-behavioural
therapy was superior to supportive therapy in 1 study.
Effect on Academic Performance
Three of 7 studies that assessed academic outcomes reported a treatment
effect. Based on an unspecified teacher report of arithmetic, but
not on a teacher report of reading, Gittleman-Klein and others (29)
reported that MPH with or without thioridazine resulted in greater
improvement than thioridazine alone. Linden and others (32) reported
that a group treated with EEG biofeedback showed significantly greater
improvement on an intelligence test than did a wait-list control
group. The MTA study reported the superiority of combined treatment
over community care and the behaviour treatment strategies. However,
combined treatment was not superior to medication management, nor
did medication management, community care, and behaviour treatments
differ in their impact on academic outcomes. The MTA study found
no differential effect of treatments for arithmetic and spelling.
The effect sizes observed for reading test scores with MPH treatment
were not large (in the range of 0.1 SD) and were not greater than
effect sizes observed for placebo treatment (0.1 SD).
Effect on Social Behaviour
Of the 9 studies that assessed the effect of treatment on conduct
and oppositional and social behaviour, all but 3 showed a treatment
effect. In each case, the beneficial effects could be attributed
to either MPH or DEX, rather than to the addition of other treatments
such as parent training, thioridazine, or behaviour therapy. The
MTA study included 7 measures of social behaviour and interaction.
Outcomes were not identical for all measures. In general, medication
management was equivalent to combined therapy, and both were superior
to behaviour treatment alone and to community care. Combined therapy,
but not medication management, was superior to behaviour treatment
or community care on oppositional or aggressive symptoms, teacher-rated
social skills, and parent-child relationships.
Effect on Internalizing Behaviour
Of the 5 studies that examined internalizing symptoms, one study
indicated an improvement in self-esteem with cognitive-behavioural
therapy. For internalizing symptoms, the MTA study found that combined
treatment had an advantage over medication management and that both
combined therapy and medication management had an advantage over
behaviour therapy. This additional benefit of combined treatment
was found only in parent-rated anxiety and was not confirmed by
either teacher report or child self-report of anxiety symptoms.
Children with ADHD and comorbid anxiety improved more with behaviour
treatment than with community care, despite the fact that two-thirds
of community care subjects received medication.
|
