Imagine this typical, open-label trial of a drug. People coming to psychiatrists’ offices are screened for depression with the Hamilton Rating Scale for Depression (HRSD). Those who score 16 or more are started on an antidepressant and followed carefully for 8 weeks. At the end of the two months, a second HRSD is administered. The investigators (and, needless to say, the drug company) are ecstatic that most patients now have scores within the normal range and claim that this new drug, which costs 3 times as much as existing ones, should be the treatment of choice.

In another study, patients with schizophrenia are selected if their scores on a measure of social functioning are below some criterion. They are entered into a program emphasizing social skills training, work-appropriate behaviour, and independent living. At the end of 6 months, most of the patients have significantly higher scores on the scale, and the investigators conclude that this intervention is highly successful with these people.

Manuscript received March 2000 and accepted August 2000.
This is the 19th article in the series on Research Methods in Psychiatry. For previous articles please see Can J Psychiatry 1990;35:616–20, 1991;36:357–62, 1993;38:9–13, 1993;38:140–8, 1994;39:135–40, 1994;39:191–6, 1995;40:60–6, 1995;40:439–44; 1996;41:137–43, 1996;41:491–7, 1996;41:498–502, 1997;42:388–94, 1998;43:173–9, 1998;43: 411–5, 1998;43:737–41, 1998;43:837–42, 1999;44:175–9, and 2000;45:833–36.
¹Director, Kunin-Lunenfeld Applied Research Unit, Baycrest Centre for Geriatric Care; Professor, Department of Psychiatry, University of Toronto, Toronto, Ontario.
Address for correspondence: Dr DL Streiner, Kunin-Lunenfeld Applied Research Unit, Baycrest Centre for Geriatric Care, 3560 Bathurst Street, Toronto, ON M6A 2E1.
e-mail: dstreiner@rotman-baycrest.on.ca

Are the researchers justified in their enthusiasm for these treatments? The answer is a resounding “No,” for a multitude of reasons. First, the natural history of the disorder may be such that positive change can be expected over the time course of the study, even in the absence of an intervention. This is more likely in the first example, but can never be completely ruled out in the second. A second possible alternative hypothesis is the so-called “Hawthorne effect”: simply paying attention to a subject in a study may produce positive changes. Although what actually happened at the bank-wiring plant in Hawthorne probably had nothing to do with the Hawthorne effect itself (1,2), it serves as a handy label for describing the nonspecific effects of filling out questionnaires, talking to clinical and research staff, and being treated in a different and special way. It is for these, and other, reasons that treatment studies should always have a control or comparison group in which it is expected that the natural history and Hawthorne effect will be comparable.

In this paper, though, I want to discuss a third possible explanation for the results, “regression toward the mean.” As we’ll see, this does not mean reverting to an anal sadistic stage of development but, more prosaically, is a statistical concept relating to people’s scores when they are tested on 2 or more occasions. It is a function of the tests themselves and more particularly, the manner in which they are used to select subjects.

The Role of Tests in Regression Toward the Mean

To understand regression toward the mean, it is necessary to take a brief detour into test theory. “Test,” in this context, does not refer only to a paper-and-pencil instrument such as