|
The Thought Disorder Questionnaire
Scale Refinement
The second phase of the project included a large sample, which consisted of 205 subjects, using a revised 210-item version of the TDQ. The sample comprised 105 subjects from the general population, 68 subjects from the inpatient wards in various hospitals, and 32 patients with schizophrenia.
At this stage of the project, analysis was based on the steps outlined in Jackson’s sequential system for personality scale development (7). An item was retained only if it cleared the following 6 hurdles:
Determined whether the proportion of subjects who endorsed the item was within an acceptable range. That is, items with a P-value below 0.05 or above 0.95 were eliminated as too weak in informational value. Evaluated convergent and discriminant validity. If an item correlated higher with a content scale other than the one for which it was written, the item was deleted. Estimated the degree to which the item elicited tendencies to respond desirably. That is, if an item correlated higher with the Social Desirability scale than with its own scale, it was deleted. Assessed the item’s saturation as indicated by the level of its correlation with the total scale. Evaluated the item’s content saturation in relation to its desirability bias. Provided a final substantive review to improve generalizability and representativeness of scale content.
The computer program specifically designed for questionnaire use was the Item Efficiency Index (IEI), which takes into account the item total correlation and item correlations with all other irrevelant scales (8). Thus, any item that correlated over 0.05 with another scale and that had an IEI below 0.40 was eliminated. Based on the above statistics, 50 items and 1 subscale (amount) were deleted. At this stage of questionnaire development, the TDQ had 160 items and 12 content and 2 bias subscales.
Analysis of Pared-Down Scales
Next, a sample study of 152 subjects (30 healthy control subjects, 54 individuals with schizophrenia, and 68 patients with mental disorders) revealed that schizophrenia patients, compared with the general population sample, differed significantly on 8 of the 12 content subscales. Total means were higher for the schizophrenia group than for the healthy control subjects. As anticipated, both the Social Desirability and the Infrequency subscales—included in the questionnaire to check for artificially positive answering or random responses—yielded no significant differences among the groups. This observation was compatible with our view that schizophrenia patients can give accurate self-reports about the content domain of cognitive functioning. Meanwhile, patients with mental disorders, compared with the healthy control subjects, differed significantly on 9 of the 12 subscales. Total means were higher for the mental disorder group than for the healthy control subjects. Reliability measures for all 152 subjects, excluding the infrequency scale, ranged from 0.7827 to 0.9097 (Table 2). Content validity ranged from 0.65 to 0.88 for correlations between scales. These findings confirm our hypothesis that we are measuring a single construct, operationally defined as “disordered thinking,” in a reliable fashion using self-report.
|
Table 2 Coefficient alpha reliability output for 14 subscales on the TDQ
160 items
|
|
|
All subjects
n = 152
|
Healthy control subjects
n = 30
|
Schizophrenia sufferers
n = 54
|
Patients
n = 68
|
|
Concentration (14)
|
0.9097
|
0.8497
|
0.8920
|
0.9349
|
|
Speed (10)
|
0.9033
|
0.9349
|
0.8120
|
0.9414
|
|
Content (10)
|
0.8125
|
0.6824
|
0.7904
|
0.8385
|
|
Control (12)
|
0.8150
|
0.7508
|
0.8307
|
0.8075
|
|
Orientation (8)
|
0.7827
|
0.6420
|
0.8001
|
0.7930
|
|
Perception (12)
|
0.7952
|
0.8012
|
0.8001
|
0.7891
|
|
Fantasy (10)
|
0.8451
|
0.7412
|
0.7767
|
0.8975
|
|
Response (10)
|
0.7886
|
0.6868
|
0.6740
|
0.8356
|
|
Insight (10)
|
0.8835
|
0.7575
|
0.8460
|
0.9218
|
|
Efficiency (24)
|
0.8961
|
0.8926
|
0.815
|
0.9227
|
|
Symptom (12)
|
0.8550
|
0.7810
|
0.8208
|
0.8936
|
|
Concept formation (10)
|
0.8350
|
0.8403
|
0.7686
|
0.8605
|
|
Infrequency (9)
|
0.4270
|
0.3947
|
0.3909
|
0.5008
|
|
Social desirability (8)
|
0.8360
|
0.8897
|
0.7831
|
0.8667
|
For this sample, total scores for patients with mental disorders were slightly higher than were scores for schizophrenia patients. Hence, we obtained a new sample of 32 schizophrenia patients for comparison and for test–retest reliability. These schizophrenia patients had a slightly higher total score than did the patients with mental disorders, differing on the control and orientation scales. Finally, the results of the test–retest reliability that involved 22 schizophrenia patients over a 1-week waiting period ranged from 0.76 to 0.94.
Consequently, using a sequential system for personality scale development involving 486 subjects, we developed a 160-item questionnaire that measured disordered thinking. The questionnaire contained 12 scales that measured aspects of thinking, as well as 2 control scales for infrequency and social desirability. Preliminary testing suggested significant differences in test scores between healthy control subjects and subjects hospitalized for mental disorders. Schizophrenia patients could accurately and reliably report on their thinking.
Validity
The final stage of test construction involved further item analysis and evaluation of the test’s content and construct validity, using a multifaceted approach. We recruited a sample of 244 patients, including 173 subjects from the general population, 21 schizophrenia patients, and 50 patients with mental disorders, with a diagnosis other than schizophrenia. Subjects were asked to complete the TDQ and the Rust Inventory of Schizotypal Cognitions (RISC) (11). However, the other disorders patient sample also completed the Basic Personality Inventory (BPI) (8), and the schizophrenia sample completed the Whittaker Index of Schizophrenic Thinking (WIST) (12).
Items with the lowest efficiency were dropped to produce a 140-item scale.
The scale consisted of 11 subscales, each with the following 10 items:
concentration (a = 0.87), speed (a = 0.91), content (a = 0.78), control
(a = 0.83), orientation (a = 0.73), perception (a = 0.76), fantasy (a =
0.81), response (a= 0.83), insight (a = 0.88), symptoms (a = 0.81), and
concept formation (a = 0.82). One subscale, efficiency (a = 0.89), included
20 items. The total TDQ scale alpha was 0.976. A 10-item social desirability
scale was retained in this version, because low scores may reflect defensive
testing (a = 0.87). The infrequency scale was removed, owing to the absence of careless or inattentive testing bias.
Preliminary analysis showed significant correlation in patients between
the total TDQ score and the thought disorder scale of Jackson’s Basic Personality
Inventory (BPI) (r = 0.75, P < 0.01). Six subscales of the TDQ had significant
correlations with the RISQ total score, but the correlation between the
totals of both scales was not significant (r = 0.12). The total TDQ total
score did not significantly correlate with the WIST.
Feasibility and Diagnostic Validity
Between 1997 and 1998, at Riverview Hospital in Vancouver, British Columbia, the principal author conducted a pilot study of the TDQ’s ability to discriminate between chronic schizophrenia patients and other patients. Only 12 of the 60 subjects who were approached were able to sign an informed consent, read and complete the TDQ, and agree to a structured diagnostic interview. The TDQ was not a useful questionnaire in this setting: self-report was too challenging for a very disturbed clientele.
From 1999 to 2001, at Surrey Memorial Hospital, the TDQ was given to 80 subjects, 20 of whom had a chart diagnosis of schizophrenia. We found the same mean scores on these tests as in previous samples. The TDQ, however, did not discriminate among other diagnoses. In fact, personality disorders had the highest scores, often 200. (Interestingly, research literature does not focus on disordered thinking in personality disorders.)
A reanalysis of the above data demonstrated that 6 scales (content, control, orientation, perception, fantasy, and symptoms) differed significantly between healthy subjects and schizophrenia patient populations (P < 0.01). Figure 1 presents the norms for these scales from the new study and those for the healthy control subjects from previous studies. Thus, we now have a more feasible 60-item scale, but one that lacks diagnostic accuracy (30 total score for healthy control subjects and 102 total score for schizophrenia patients).
Figure 1 Scores for schizophrenia patients and control subjects
CNT= content; CTL = control; ORI = orientation; PER = perception; FAN = fantasy; SYM = symptoms
Discussion
Developing psychometrically sound questionnaires is complex and time-consuming. In fact, this process has taken more than a decade. The presentation of complex statistical analysis may seem excessive or irrelevant to clinicians; nevertheless, we should remember that our discussion of the effects of atypical antipsychotics on negative symptoms of schizophrenia was facilitated by Stanley Key’s development of the Positive and Negative Syndrome Scale (PANSS). We hope, for example, that the TDQ will allow us to assess whether medications have a greater or lesser effect on content of thought vs control of thought.
There are many difficulties encountered in scale development. Because psychiatrists traditionally are taught that people with schizophrenia lack insight that they have a mental disorder, it was difficult for reviewers to accept that these subjects could accurately report their experiences. A second conceptual problem is that “thought disorder” has several meanings that are easily confused with disordered thinking in the psychotherapy of schizophrenia. A major practical problem with such a time-consuming project is attempting to keep track of the database.
Despite these obstacles, my colleagues now have a relatively efficient and inexpensive test that their patients seem to enjoy completing and that provides information about severity of disturbance and response to treatment. The self-report method also avoids time-consuming training and interrater reliability issues.
Summary
In conclusion, from a sample of 870 subjects, we developed a 60-item, self-report measure of disordered thinking with established reliability and content validity. Now, we can establish the criterion and construct validity of the TDQ by evaluating 1) the degree to which disordered thinking, as measured by this scale, is characteristic of mental disorder patients, especially those diagnosed as schizophrenia patients; 2) whether disordered thinking is a risk factor for schizophrenia; and 3) the relation of disordered thinking to the course and outcome of schizophrenia (Figure 2).
Figure 2 Thought Disorder Questionnaire scores
TDQ scores
Funding and Support
The development of this questionnaire was initially funded by a grant from the Medical Research Council of Canada.
References
1. Waring EM, McCarty T, Abraham B, Fry R. Thought disorder: current assessment, techniques, and future trends. Psychiatr J Univ Ott 1989;14:520–8.
2. Waring EM, Lefcoe DH, Carver C, Barnes S, Fry R, Abraham B. The course and outcome of early schizophrenia. Psychiatr J Univ Ott 1988;13:194–7.
3. Waring EM, McCarty T, Neufeld RWJ, Abraham B, Fry R, Carver C. [Letter]. Schizophr Bull 1989;15:1–3.
4. Waring EM. The psychobiology of first episode schizophrenia. Can J Psychiatry 1995;40:33–7.
5. Waring EM, Neufeld RWJ, Schaefer B. Development and reliability of the Thought Disorder Questionnaire. Presented at Schizophrenia Conference; October 1994;Vancouver (BC).
6. Waring EM, Neufeld RWJ, Schaefer B. Development and reliability of the Thought Disorder Questionnaire. Presented at Schizophrenia Conference; October 1996; Vancouver (BC).
7. Jackson DJ. A sequential system for personality scale development. In: Spielberger CD, editor. Current topics in clinical and community psychology. New York: Academic Press; 1970. p 61–96.
8. Andreason NC. Thought language and communication disorders: diagnostic significance. Arch Gen Psychiatry 1979;36:1325–30.
9. Estes WK. Learning theory and intelligence. Am J Psychol 1974;29:740–9.
10. Embretson WS. Construct validity: construct representation versus nomothetic span. Psychiatry Bull 1983;93:179–97.
11. Rust J. The Rust Inventory of Schizotypal Cognitions (RISC). Schizophr Bull 1988;14:317–22.
12. Whittaker LC. Manual for the Whittaker Index of Schizophrenic Thinking (WIST). Los Angeles: Western Psychological Services; 1973.
Author(s)
Manuscript received February 2002 and accepted October 2002.
1. Clinical Professor, University of British Columbia, Vancouver, British Columbia; Clinical Director, Surrey Mental Health Services, Surrey, British Columbia; Chief, Psychiatry, Surrey Memorial Hospital, Surrey, British Columbia.
2. Professor, Psychology, University of Western Ontario, London, Ontario.
3. Research Assistant, University of Western Ontario, London, Ontario.
Address for correspondence: Dr EM Waring, Surrey Memorial Hospital, 13750 96th Avenue, Surrey, BC V3V lZ2
e-mail: ted.waring@sfvhr.hnet.bc.ca
1 | 2 | 3
|
