Original Research

The Centre for Addiction and Mental Health Concurrent Disorders Screener

Juan C Negrete, MD, FRCPC¹, Jane Collins, MSc², Nigel E Turner, PhD³, Wayne Skinner, MSW⁴

Objective: To review the characteristics of psychiatric screening tools currently available in addiction treatment services for rapid assessment of comorbid pathology and to introduce the Centre for Addictions and Mental Health Concurrent Disorders Screener (CAMH-CDS), a computer-administered questionnaire that screens for the occurrence of 11 Axis I disorders plus all substance use disorders, as well as for a history of conduct disorder. Method: We describe the structure, contents, and application of the CAMH-CDS. We undertook a sensitivity and specificity trial involving 171 subjects, a test–retest reliability study with 301 participants, and an open-label concordance study with 656 respondents. All subjects were regular clients of a major addiction treatment facility. Results: The CAMH-CDS was easily and effectively used by addiction counsellors with limited or no mental health training. It has a low rate of false-negative responses, and it yields excellent test–retest reliability figures. It is highly sensitive to identifying persons with psychiatric disturbances; however, its ability to discriminate among specific disorders appears to be more limited. Conclusion: The CAMH-CDS can be reliably used to rule out the presence of psychiatric comorbidity in addiction service populations. As with other psychiatric screening instruments, its sensitivity values are stronger than its specificity values. The use of nonstructured clinical evaluations as the gold standard for diagnosis and a likely variance in the patients’ symptom reports between the 2 examinations may have contributed to the latter finding. (Can J Psychiatry 2004;49:843–850) Click here for author affiliations.  Clinical Implications This instrument will help addiction workers perform a more comprehensive clinical evaluation of their clients. The instrument’s simplicity and ease of administration should facilitate its use in all types of addiction treatment facilities, even when counsellors have limited professional qualifications. Limitations This instrument has been validated in an addiction treatment population only. Its psychometric properties have been tested against clinical diagnoses that were not independently confirmed and that were not made through structured interviews. Key Words: psychiatric assessment, screening instruments, addiction, comorbidity Résumé :L’instrument de dépistage des troubles concurrents du Centre de toxicomanie et de santé mentale

Psychiatric disorders are highly prevalent among persons seeking care in addiction treatment services (1). The rates of comorbidity found in the population attending such facilities are in fact higher than those observed in substance abusers surveyed in nonclinical settings (2). Additional psychiatric disturbances adversely affect the course and therapeutic response of addictive disorders (3). Not only do substance abusers with psychiatric comorbidity show poorer results in addiction treatment services, they also tend to make excessive use of health and social services as a whole (4). If cooccurring psychiatric disorders are identified and attended to in a timely fashion, the clinical outcome of these complex cases can be significantly improved.

However, the capacity to perform comprehensive psychiatric examinations does not exist in a large number of addiction treatment programs, and it is not realistic or sensible to expect such procedures to be a mandatory component of the standard clinical protocols. Indeed, a complete psychiatric evaluation performed universally for all clients would be costly, impractical, and even unwarranted: current psychiatric disorders—those requiring specific clinical management at the time—can be expected to affect only about one-third of the client population processed at any given treatment centre (5).

There is therefore a need for a procedure that will help identify the group of addiction clients whose treatment-planning assessment must definitely include an in-depth psychiatric examination. Such a procedure should reliably select the individuals who require referral to a specialized professional for a more complete diagnostic work-up; it should be routinely, effectively, and cost-efficiently applicable by front-line addiction clinicians with limited or no mental health training.

Several existing instruments facilitate the identification of psychiatric disorders in health care settings, mostly by nonspecialized clinicians. Administration of these questionnaires is usually to be followed by a medical evaluation; their purpose is merely to sensitize and guide clinical examiners by screening for initial evidence of psychiatric disturbance. The following are some of the best-known psychiatric screening instruments currently available in English: the Quick Psychodiagnostics Panel (QPD, 6), a self-administered procedure that covers 6 common disorders (excluding the schizophrenia spectrum) and requires the patient to use a hand-held computer; the Symptom Driven Diagnostic System for Primary Care (SDDS-PC; 7,8), which covers some nonpsychotic disorders only and consists of a self-administered, 26-item screening questionnaire plus a set of 6 diagnostic modules to be selectively applied by a clinical examiner to confirm the presence of disorders that screened positive; the Primary Care Evaluation of Mental Disorders (PRIME-MD; 9–12), also a 26-item self-report questionnaire (with a yes–no format), followed by a structured clinical interview containing 5 diagnostic modules that correspond to the disorders covered by the procedure (excluding psychoses and drug dependence); the Psychiatric Diagnostic Screening Questionnaire (PDSQ; 13,14), a 126-item, paper and pencil, self-administered instrument that explores the presence of 13 Axis I disorders (anorexia, dysthymia, and bipolar disorder are excluded ); and the Mini International Neuropsychiatric Interview (MINI, 15), a short diagnostic interview requiring a well-trained user.

Major problems in clinical practice are the poor reliability of symptom reports and the questionable validity of psychiatric diagnoses in persons who are abusing psychoactive substances. Indeed, individuals who are intoxicated or in withdrawal may express complaints that mimic psychiatric disorders but that have no real permanence or diagnostic significance. Although some substance-induced psychiatric disturbances may linger for some time, most such symptoms tend to diminish or subside rapidly in conditions of sustained abstinence.

Cross-sectional clinical assessments are often insufficient to ascertain the nature of psychiatric findings in this complex population and to determine whether these findings correspond to self-limited disturbances or to cooccurring, independent disorders that require specific treatment. It may be necessary to reevaluate the patient in drug-free conditions that require an abstinence of several weeks’ duration in some cases. Of course, a reliable clinical history that meticulously examines the onset and course of each diagnostic parameter in relation to intoxication and withdrawal states can always yield helpful information in a single assessment. This is what is intended with the Psychiatric Research Interview for Substance and Mental Disorders interview schedule (PRISM; 16,17). However, the PRISM examination is quite laborious: it can last more than 90 minutes, and it must be performed by highly trained interviewers.

The complexity of the instruments that can identify psychiatric disorders with greater validity and reliability precludes their use by addiction workers without extensive training in mental health, while the simpler ones are rather too incomplete or too imprecise (18) to serve the clinical needs of addiction centres. For this reason, we undertook to develop and test an alternative screening procedure.

The Centre for Addiction and Mental Health Concurrent Disorders Screener

The Centre for Addiction and Mental Health Concurrent Disorders Screener (CAMH-CDS) was introduced as a segment of the intake assessment package at a large clinical addiction facility to select new arrivals who require referral to the mental health services (specifically, the Concurrent Disorders Program). As with all existing screening instruments, the CAMH-CDS results are not to be considered as a complete diagnostic assessment, nor is it possible to determine by the one-time administration of this instrument whether the disorders elicited are substance induced or independent. However, the CAMH-CDS can establish that the minimum set of syndromic elements were present at the same time, that the disturbances lasted the minimum time required to have diagnostic significance, that they caused sufficient functional impairment, and that they were present in the last 30 days (if not present, it can establish the age at which the individual last experienced them). In other words, this procedure was designed to determine whether DSM-IV diagnostic criteria are likely to be met, both for currently active disorders and for those in remission, because it is important to establish the presence of illnesses that must be monitored owing to their recurring nature (for example, schizophrenia and bipolar disorder).

In its present version, The CAMH-CDS covers the following Axis I conditions: major depression, dysthymia, manic episode, panic disorder, agoraphobia, social phobia, schizophrenia, schizophreniform disorder, anorexia nervosa, bulimia nervosa, pathological gambling, alcohol abuse, alcohol dependence, barbiturate abuse, barbiturate dependence, benzodiazepine abuse, benzodiazepine dependence, cannabis abuse, cannabis dependence, cocaine abuse, cocaine dependence, hallucinogen abuse, hallucinogen dependence, heroin or opium abuse, heroin or opium dependence, over-the- counter or prescription opiate abuse, over-the-counter or prescription opiate dependence, inhalant abuse, inhalant dependence, stimulant abuse, stimulant dependence, and other drug or polydrug dependence. The CAMH-CDS also inquires about history of conduct disorder, but it does not screen for any other personality (Axis II) pathology.

A computerized self-report questionnaire tool (CSRT3, unpublished) is used to present the CAMH-CDS. The software is designed to run in Windows 95 and later versions. There is a separate screening module for each disorder, and the screening session can cover the complete battery or just the individual categories selected by the examiner.

Although the instrument possesses self-administration capability, it is recommended that it be used with the assistance of an examiner. This will ensure a complete and coherent response. By and large, the questions are self-explanatory, but some respondents may need clarifications that, for the sake of brevity, are not included in the program. An examiner with minimum training standing by can offer explanations if and when they are needed.

Highly sensitive screening questions are asked for each disorder. If the respondent answers “no” to these questions, the program advances to the next disorder. A respondent who does not endorse any of the screening statements will be asked a total of 24 questions. A “yes” answer to a screening question is followed by the minimum number of questions required to meet DSM-IV criteria for that particular disorder. Depending on the number of disorders reported, administering the CAMH-CDS takes from 5 to 20 minutes. In short, the CAMH-CDS is a computerized questionnaire (with a yes–no format) that can quickly gather information on past and current occurrence of psychiatric syndromes. Following the interview, assessment workers can immediately view the screening results and print a hard copy without leaving the program.

CAMH-CDS Validation Trials

Participants

We validated the instrument at the Centre for Addiction and Mental Health, Toronto, Ontario, using 3 separate samples. A total of 171 individuals completed both the CAMH-CDS and a standard Psychiatric Diagnostic Interview (PDI) as part of a double-blind research protocol. Another 656 participants underwent both examinations as part of the open-label clinical assessment normally conducted in the Centre’s Concurrent Disorders Program, and 301 randomly selected respondents participated in a test–retest reliability trial in which one disorder module was presented twice on the CAMH-CDS schedule. All participants were recruited from among the normal flow of clients who presented to the CAMH seeking assistance for a substance-use disorder; the samples reflect the age and sex distribution as well as the multiple substance preferences of such a general addiction clientele. The research protocols were reviewed and approved by the University of Toronto Ethics Board.

Sensitivity Study

Whether they scored positively or negatively on the CAMH-CDS, a random sample of clients were asked to undergo a PDI. They provided informed consent and were paid for their participation. Of the 171 participants who completed both the CAMH-CDS and PDI assessments, 70.2% were men (n = 120), and 29.8% were women (n = 51). The mean age of the samples was 38.17 years, SD 9.78 years, and they had completed a mean 12.69 years, SD 3.17 years, of education. Alcohol was reported as the main substance of abuse by 57.9%, followed by cocaine (21.6%), opiates (9.4%), and cannabis (7.0%). A sizeable 53.2% of this group had a history of previous psychiatric contact.

Neither the respondent nor the clinical examiner were informed of the CAMH-CDS results. The PDI always took place within 2 weeks of the CAMH-CDS screening; it was conducted by a group of fully trained psychiatrists. These clinicians were not required to perform the interview in the structured manner prescribed by the Structured Clinical Interview for DSM-III-R (SCID). However, they were asked to complete a form indicating whether, in their clinical judgement, certain disorders (specifically, those screened for by the CAMH-CDS were present or absent earlier on, at the time the subject was screened with the CAMH-CDS.

Table 1 gives a breakdown of global results from the 2 assessment methods (that is, presence or absence of any of the disorders screened for). The “positive but” category comprises 21 clients who scored positive for a given disorder on the CAMH-CDS but were given another Axis I diagnosis at the PDI and 10 respondents who scored positive for an Axis I disorder on the CAMH-CDS but who received an Axis II diagnosis at the clinical assessment.

Table 1  Sensitivity results (raw frequencies) with a random, double-blind research sample  (n = 171)
	CDS positive	CDS negative
PDI positive	84	4
PDI positive but for other Axis I or II disorders	31	6
PDI negative	12	34
Collapsing PDI positive and PDI negative but for other Axis I or II disorder
PDI positive	115	10
PDI negative	12	34
Controlled sample: sensitivity = 0.92; specificity = 0.739; positive predictive value = 0.905; negative predictive value = 0.773; k = 0.668; proportion of overall agreement = 0.871; proportion of specific agreement for positive ratings = 0.913, for negative ratings  = 0.756  CDS = Concurrent Disorders Screen; PDI = Psychiatric Diagnostic Interview

As Table 1 indicates, the negative rate depends on how it is scored. Of the 44 individuals who were negative according to the CAMH-CDS, only 9% were given positive scores on the PDI for disorders screened by the CAMH-CDS; another 13% had a PDI diagnosis for a disorder not covered by the CAMH- CDS. Thus, for disorders covered by the CAMH-CDS, the false-negative rate is only 9% of the negative responses. The false-positive rate also depends on how it is scored. If one considers only those disorders included in the CAMH-CDS, a relatively high number of the respondents with positive scores turn out not to have those specific disorders (33%). However, if the focus is on the presence of any disorder, the number of false positives drops to only 10%. In practical terms, the CAMH-CDS appears capable of excluding most people who do not have the Axis I disorders it screens for; but it will nonspecifically identify as positive a fairly large number of people who actually would receive a clinical diagnosis.

The results of this trial permit us to conclude that the CAMH- CDS has a low false-negative rate but a moderately high false-positive rate. When the CAMH-CDS indicates that a person’s condition warrants a clinical psychiatric assessment, one can be reasonably sure that he or she does actually have a psychiatric problem. However, the instrument often fails to discriminate between Axis I and Axis II disorders. Conversely, when the CAMH-CDS indicates that the person does not need additional assessment, it is 90% certain that he or she in fact does not.

Concordance Data on Regular Referrals to the Concurrent Disorders Program

In addition to the previous controlled study, we also collected clinical data on the routine use of the instrument in a concurrent disorder–addiction treatment system. A total of 656 clients—referred to here as “clinical sample”—received both CAMH-CDS and PDI assessments: 64.2% were men (n = 421); 35.8% were women (n = 235); their mean for age was 37.67 years, SD 9.97, and their mean education level was 12.43 years, SD 3.46. The main substance use disorders were alcohol (53.05%), cocaine (15.7%), opiates (13.87%), and cannabis (10.37%). In this group, too, a large number of participants (50.0%) had been previously seen in psychiatry. This sample should consist mostly of true-positive and false- positive CAMH-CDS respondents, but some CAMH-CDS negative respondents were also included because, despite their negative responses, the intake worker felt that they needed a clinical psychiatric assessment. These data are shown in Table 2.

Table 2  Sensitivity results (raw frequencies) with open referrals to the Concurrent Disorders Program (n = 656)
	CDS positive	CDS negative
PDI positive	358	43
PDI positive but for other Axis I or II disorders	111	33
PDI negative	67	44
Collapsing PDI positive and PDI negative but for other Axis I or II disorder
PDI positive	469	76
PDI negative	67	44
Clinical sample: sensitivity = 0.860; specificity = 0.396; positive predictive value = 0.875; negative predictive value = 0.367; k = 0.249; proportion of overall agreement = 0.782; proportion of specific agreement for positive ratings = 0.868, for negative ratings = 0.381  See Table 1 for abbreviations

The “positive but” category in Table 2 comprises 85 people who were given clinical Axis I diagnoses other than the ones indicated by their CAMH-CDS responses and 26 people who instead received a clinical diagnosis of Axis II disorder. The data in this table show that only 12.5% of the clients who scored positive according to the CAMH-CDS were clinically found not to meet criteria for any psychiatric disorder. However, if the “positive but” category is excluded, the inaccuracy rate goes up to 33%. These numbers are similar to those found in the controlled sensitivity study. The false-negative rate in this sample is higher than in the previous study, but it should be noted that this was a skewed sample: all the subjects who were negative according to the CAMH-CDS were systematically included because intake clinicians suspected them of giving inaccurate answers. Thus these data contain a nonrandom overrepresentation of unreliable responders.

Specificity by Disorder

Table 3 shows a breakdown of depressive disorders in terms of specificity and sensitivity for both controlled and clinical samples. Most often (87.8% of cases), when the CAMH-CDS elicited a depression, the PDI also detected a disorder; however, the PDI diagnosed a specific depression problem in only 64.3% of the cases, while in the remaining 23.5% it diagnosed some disorder other than those defined as depression in the CAMH-CDS.

Table 3  Depression spectrum: major depressive episode, dysthymia, bipolar disorder–depressed, schizoaffective disorder–depressed
	CDS positive	CDS negative
Double-blind sample (n = 171)a
PDI positive	57 (TP 64.04%)	14 (FN 17.07%)
PDI negative	32 (FP 35.96%)	68 (TN 82.93%)
Clinical sample (n = 656)b
PDI positive	255 (TP 64.39%)	76 (FN 29.23%)
PDI negative	141 (FP 35.61%)	184 (TN 70.77%)
aSensitivity = 0.803; specificity = 0.680; positive predictive value = 0.640; negative predictive value = 0.829; k = 0.466; proportion of overall agreement = 0.731; proportion of specific agreement for positive ratings = 0.712, for negative ratings = 0.747  bSensitivity = 0.770; specificity = 0.566; Positive predictive value = 0.644; negative predictive value = 0.708; k = 0.337; proportion of overall agreement = 0.669; proportion of specific agreement: for positive ratings = 0.701, for negative ratings = 0.629  TP = true positives, TN = true negatives; FP = false positives, FN = false negatives. See Table 1 for abbreviations.

Most often (in 91.41% of the cases), if the CAMH-CDS detected an anxiety problem, the PDI also diagnosed a disorder; however, the PDI registered an anxiety disorder diagnosis in only 36.87% of the cases, while in 54.54% it indicated other, mostly Axis II, conditions (Table 4).

Table 4  Anxiety spectrum: generalized anxiety disorder, panic disorder, panic disorder with agoraphobia, social phobia, single phobia, posttraumatic stress disorder, obsessive–compulsive disorder, anxiety
	CDS positive	CDS negative
Double-blind sample (n = 171)a
PDI positive	31 (TP 40.79%)	9 (FN 9.47%)
PDI negative	45 (FP 59.21%)	68 (TN 90.53%)
Clinical sample (n = 656)b
PDI positive	115 (TP 35.94%)	76 (FN 16.37%)
PDI negative	205 (FP 64.06%)	184 (TN 83.63%)
aSensitivity = 0.775; specificity = 0.656; positive predictive value = 0.408; negative predictive value = 0.905; k = 0.329; proportion of overall agreement = 0.684; proportion of specific agreement for positive ratings  = 0.534, for negative ratings = 0.761  bSensitivity = 0.676; specificity = 0.578; positive predictive value = 0.359; negative predictive value = 0.836; k = 0.198; proportion of overall agreement = 0.604; proportion of specific agreement for positive ratings = 0.469, for negative ratings = 0.684  See Tables 1 and 3 for abbreviations

Most often (in 87.74% of the cases), when the CAMH-CDS indicated a schizophrenia spectrum problem, the PDI also diagnosed a disorder; however, the PDI concurred with the diagnosis of schizophrenia spectrum disorder in only 30.88% of the cases, while in 56.86% it indicated some other problem (Table 5).

Table 5   Schizophrenia spectrum: schizophrenia, schizophreniform disorder, schizophrenic-like disorder, schizoaffective disorder
	CDS positive	CDS negative
Double-blind sample (n = 171)a
PDI positive	15 (TP 42.86%)	5 (FN 3.68%)
PDI negative	20 (FP 57.14%)	131 (TN 96.32%)
Clinical sample (n = 656)b
PDI positive	48 (TP 28.4%)	30 (FN 6.16%)
PDI negative	121 (FP 71.60%)	457 (TN 93.84%)
aSensitivity = 0.750; specificity = 0.868; positive predictive value = 0.429; negative predictive value = 0.963, k = 0.466; proportion of overall agreement = 0.854; proportion of specific agreement for positive ratings = 0.545, for negative ratings = 0.913  bSensitivity = 0.615; specificity = 0.791; positive predictive value = 0.284; negative predictive value = 0.938; k = 0.270; proportion of overall agreement = 0.770; Proportion of specific agreement: for positive ratings = 0.545, for negative ratings = 0.913  See Tables 1 and 3 for abbreviations

Most often (in 88.77% of the cases), if the CAMH-CDS screened positive for a manic episode, the PDI did identify a disorder; however, the PDI agreed with such diagnosis in only 21.39% of the cases, while in 67.38% it indicated some other problem (Table 6).

Table 6  Manic disorder spectrum: manic episode, bipolar disorder–manic,  schizoaffective-manic
	CDS positive	CDS negative
Double-blind sample (n = 171)a
PDI positive	31 (TP 40.79%)	9 (FN 9.47%)
PDI negative	45 (FP 59.21%)	68 (TN 90.53%)
Clinical sample (n = 656)b
PDI positive	115 (TP 35.94%)	76 (FN 16.37%)
PDI negative	205 (FP 64.06%)	184 (TN 83.63%)
aSensitivity = 0.571; specificity = 0.860; positive predictive value = 0.267; negative predictive value = 0.957; k = 0.284; proportion of overall agreement = 0.836; proportion of specific agreement for positive ratings (ps+) = 0.363, for negative ratings (ps–) = 0.906  bSensitivity = 0.627; specificity = 0.793; positive predictive value = 0.204; negative predictive value = 0.962; k = 0.216; proportion of overall agreement = 0.78; proportion of specific agreement for positive ratings = 0.308, for negative ratings = 0.870  See Tables 1 and 3 for abbreviations

Test–Retest Reliability Study

To measure the degree of reliability with which the subjects responded to this instrument, we required a randomly selected sample (n = 301) to answer some of the questions twice during a single screening. Rather than having the participants redo the entire instrument on separate occasions, one randomly selected module was administered a second time. All participants in this trial were informed that one disorder module would be repeated and gave formal consent to the procedure. A group of 100 respondents answered the mood disorder module twice; 101 of them answered an anxiety disorder set twice, and 100 others answered the schizophrenia–schizophreniform module twice. Three-fourths of the participants were men (n = 227), and 25% were women (n = 74); the total sample’s mean age was 38.0 years, SD 9.4. The largest number reported alcohol as the main problem (51.6%), followed by cocaine (19.6%), opiates (13.6%), and cannabis (4.3%). They had an mean 12.69 years of education, SD 3.2; 57.7% of them had previous psychiatric examinations. Listed below are the retest reliability coefficients for each separate disorder.

1. CAMH-CDS mood disorder modules (n = 100)

      Depression current: k = 0.865

      Manic episode current: k = 0.807

2. CAMH-CDS Anxiety Disorders modules (n = 101)

      Panic disorder with or without agoraphobia current: k = 0.720

      Panic disorder with or without agoraphobia past: k = 0.884

      Social phobia current: k = 0.917

3. CAMH-CDS schizophrenia or schizophreniform disorders modules (n = 100)

      Schizophrenia current: k = 0.693

      Schizophrenia past: k = 0.556

      Schizophreniform current: k = 0.656

      Schizophreniform past: k = 0.653

Commentary

Like most of the instruments mentioned in the introduction, the CAMH-CDS is a multiple disorder assessment tool. There are, of course, many questionnaires that screen specifically for single disorders such as bipolar spectrum disorder (19), depression (20–22), anxiety disorder (23–25), and borderline personality disorder (26). We have not referred to these because they would not be of use for the more comprehensive mental health evaluation that should be performed during a complete addiction assessment.

According to the publications listed to date, the CAMH-CDS appears to be the first instrument of its kind to have been specifically and extensively tested in an addiction treatment facility. There were no significant differences in demographic profile or clinical presentation between the study samples and the general pool of CAMH clients.

Not only were the respondents regular and spontaneous addiction service attenders, the psychiatric screening was performed by addiction assessment staff with no particular training in mental health. It could therefore be reasonably assumed that the observations reported here are likely to reflect the performance of the tool in any addiction setting.

The CAMH-CDS proved easy to administer in such an environment. Few clients refused to answer the complete set of modules, and most respondents readily understood the questions. The procedure was conducted during the first visit to the clinic, and some respondents could therefore have been under the influence of psychoactive substances at the time. It was left to the assessment worker to judge whether the client was capable of meaningful participation. Indeed, it appears reasonable to recommend that those who abuse substances complete this instrument in the presence of a staff person. As would be the case with any such questionnaires, a few clients appeared to behave in an invalid, even playful manner (for example, responding “no” to the stem questions just to expedite passage to the next module and shorten the exercise). The assessment worker who witnessed such behaviour could choose to refer these individuals for clinical evaluation regardless of the CAMH-CDS results. Of course, a certain number proved to be false negatives (see clinical sample results in Table 2). Most responders did not find the CAMH-CDS screening either long or cumbersome; the average completion time was 17 minutes (range 8 minutes to 26 minutes).

An unstructured clinical evaluation performed by a psychiatrist was taken as the gold standard against which we compared the CAMH-CDS results. Admittedly, this particular option is not without flaws, and a more tightly designed concordance study would have required that the same diagnostic questions be presented in both evaluations (that is, the CAMH-CDS modules and the SCID). There is no certainty that the target disorders were explored in exactly the same manner across both procedures or across the several clinical examiners involved in the validation trial. The clinicians’ diagnostic conclusions were not, in fact, independently confirmed. Moreover, it was not possible to arrange for the PDI to take place the same day as the CAMH-CDS, and the patients’ complaints may have changed over the time elapsed between the 2 evaluations, as is so often the case with respondents who abuse substances. In the double-blind study, the interrater kappa values were considerably better when the subject had the PDI within a week of the CAMH-CDS, compared with values when the PDI was done later (0.70 vs 0.57).

Thus the instrument’s psychometric properties were tested in a less-than-accurate manner, perhaps to its disadvantage. However, this choice was made on purpose, because the intention was to test how the CAMH-CDS findings compare with the diagnostic conclusions that psychiatrists come to in standard clinical practice, not those of research interviewers following structured schedules to the letter.

All the same, the validation trial demonstrated that this instrument is quite capable of screening out substance abusers without other significant psychiatric pathology and that those who use it can be reasonably certain that no diagnosis according to the CAMH-CDS means no need for further psychiatric involvement at that time. Indeed, the negative predictive values are high (between 0.70 and 0.96), and it is reassuring that the most accurate negative prediction corresponds to the most severe disorders (specifically, the schizophrenia spectrum and bipolar disorder).

While highly sensitive to identifying persons with psychiatric pathology in need of intervention, the CAMH-CDS presents a problem of low specificity in regard to precise diagnoses. Some of the positive prediction values are rather low. In that, the CAMH-CDS does not differ from other screening instruments, most of which have been found to yield significant false-positive figures for specific diagnoses, particularly in the mood and anxiety spectra (18). Perhaps it was too ambitious to expect a screening instrument to be able to differentiate between disorders that involve a similar subjective experience of distress (for example, social anxiety vs agoraphobic behaviour or restlessness in depression vs generalized anxiety disorder). It is also clear that several of the CAMH-CDS symptom clusters that were intended to identify Axis I disorders were endorsed by individuals whom the clinicians saw as presenting a personality disorder. Of course, this does not mean that the symptoms endorsed by the respondents were not present, only that the clinicians opted for a more general diagnosis that portrayed the bigger picture rather than limiting it to the isolated syndromes detected by the CAMH-CDS.

The test–retest kappa values obtained in the present trial ranged from very good to excellent. Some may think that not enough time was allowed to elapse before the retesting and that the respondents may have recalled their previous answers to the repeat questions. Be it as it may, consistent answers also demonstrate that the subjects had a firm understanding of the questions and were able to participate meaningfully in the screening exercise.

From the clinical experience gained so far with the CAMH-CDS, and in light of the data yielded by the validation trials, we feel confident in recommending the use of this screening instrument to select individuals who need psychiatric care from among the larger pool of addiction service clients.

Funding and Support

The development and testing of the CAMH-CDS was supported by an internal grant from the Addiction Research Foundation Division of the CAMH.

References

1. Ross HE, Glaser FB, Germanson T. The prevalence of psychiatric disorders in patients with alcohol and other drug problems. Arch Gen Psychiatry 1988;45:1023–31.

2. Regier DA, Farmer ME, Rae DS, Locke BZ, Keith SJ, Judd LL, and others. Comorbidity of mental disorders with alcohol and drug abuse. JAMA 1990;264:2511–8.

3. Woody GE, McLellan AT, Luborsky L. Severity of psychiatric symptoms as a prediction of benefits from psychotherapy: The Veterans Administration–Penn study. Am J Psychiatry 1984;141:1172–7.

4. Bartels SJ, Teague GB, Drake RE, Clark RE, Bush PW, Noordsy DL. Service utlilization and costs associated with substance abuse among schizophrenic patients. J Nerv Ment Dis 1993;181:227–32.

5. Miller NS, Fine J. Epidemiology of comorbidity in addiction psychiatry. In: Miller NS, editor. The principles and practice of addictions in psychiatry. Philadelphia (PA): WB Saunders Co; 1997. p 11–7.

6. Shedler J, Beck A, Bensen S. Practical mental health assessments in primary care. Validity and utility of the Quick Psychodiagnostics Panel. J Fam Pract 2000;49:614–21.

7. Leon AC, Olfson M, Weissman MM, Portera L, Fireman BH, Blacklow RS, and others. Brief screens for mental disorders in primary care: a validation study. J Gen Intern Med 1996;11:426–30.

8. Weissman MM, Broadhead WE, Olfson M, Sheehan DV, Hoven C, Fireman BH, and others. A diagnostic aid for detecting (DSM-IV) mental disorders in primary care. Gen Hosp Psychiatry 1998;20:1–11.

9. Spitzer RL, Williams JB, Kroenke K, Linzer M, de Gruy FV, Hahn SR, and others. Utility of a new procedure for diagnosing mental disorders in primary care. JAMA 1994;272:1749–56.

10. Kobak KA, Taylor L, Dottl SL, Greist JH, Jefferson JW, Burroughs D, and others. A computer-administered telephone interview to identify mental disorders. JAMA 1997;278:905–10.

11. Spitzer RL, Kroenke K, Williams JB. Validation and utility of a self-report version of PRIME-MD: the PHQ primary care study. Primary care evaluation of mental disorders. JAMA 1999;282:1737–44.

12. Valenstein M, Kales H, Mellow A, Dalack G, Figueroa S, Lawton BK, and others. Psychiatric diagnosis and intervention in older and younger patients in a primary care clinic: effect of a screening and diagnostic instrument. J Am Geriatr Soc 1998;46:1499–505.

13. Zimmerman M, Mattia JI. The Psychiatric Diagnostic Screening Questionnaire: development, reliability and validity. Compr Psychiatry 2001;42:175–89.

14. Zimmerman M, Mattia JI. A self-report scale to help make psychiatric diagnoses: the Psychiatric Diagnostic Screening Questionnaire. Arch Gen Psychiatry 2001;58:787–94.

15. Lecrubier Y, Sheehan DV, Weiller E. The Mini International Neuropsychiatric Interview (MINI): a short diagnostic structured interview—reliability and validity according to CIDI. Eur Psychiatry 1997;12:224–31.

16. Hasin DS, Trautman KD, Miele GM, Samet S, Smith M, Endicott J. Psychiatric Research Interview for Substance and Mental Disorders ( PRISM). Am J Psychiatry 1996;153:1195–201.

17. Hasin D, Trautman K, Endicott J. Psychiatric Research Interview for Substance and Mental Disorders: phenomenologically based diagnosis in patients who abuse alcohol or drugs. Psychopharmacol Bull 1998;34:3–8.

18. Leon AC, Portera L, Olfson M, Weissman MM, Kathol RG, Farber L, and others. False positive results: a challenge for psychiatric screening in primary care. Am J Psychiatry 1997;154:1462–4.

19. Hirschfeld RM, Williams JB, Spitzer RL, Calabrese JR, Flynn L, Keck RE, and others. Development and validation of a screening instrument for bipolar spectrum disorder. Am J Psychiatry 2000;157:1873–5.

20. Rizzo R, Piccinelli M, Mazzi MA, Bellantuono C, Tarnella M. The Personal Health Questionnaire: a new screening instrument for detection of ICD-10 depressive disorders in primary care. Psychol Med 2000;30:831–40.

21. Sheeran T, Zimmerman M. Case identification of depression with self-report questionnaires. Psychiatr Res 2002;109(1):51–9.

22. Weyerer S, Killman U, Ames D, Allen D. The Even Briefer Assessment Scale for Depression (EBAS-DEP). Int J Geriatr Psychiatry 1999;14:473–80.

23. Gilbody SM, House AO, Sheldom TA. Routinely administered questionnaires for depression and anxiety: systematic review. BMJ 2001; 322:406–9.

24. Connor KM, Kobak K, Churchill LE, Katzelnick D, Davidson JR. The Mini-SPIN: a brief screening assessment for generalized social anxiety disorder. Depress Anxiety 2001;14:137–40.

25. Scragg P, Grey N, Lee D, Young K, Turner S. A brief report on the Penn Inventory for posttraumatic stress disorder. J Trauma Stress 2001;14:605–11.

Author(s)

Manuscript received October 2003, revised, and accepted February 2004..

1 Formerly, Professor and Head, Addiction Psychiatry Program, University of Toronto and the Centre for Addictions and Mental Health, Toronto, Ontario; now, Professor and Senior Psychiatrist, McGill University and the Montreal General Hospital, Montreal, Quebec.

2 Research Associate, Centre for Addictions and Mental Health, Toronto, Ontario.

3 Scientist, Centre for Addictions and Mental Health, Toronto, Ontario.

4 Assistant Professor and Deputy Clinical Director, Addictions Program, University of Toronto and Centre for Addictions and Mental Health, Toronto, Ontario.

Address for correspondence: Dr JC Negrete, 1604 Pine Ave West, Montreal, QC H3G 1B4

e-mail: juan.negrete@mcgill.ca.

1 | 2