Editorial Credits/ Crédits éditorials

Subscription Rates /Prix d'abonnements

Advertising Rates / Tarifs publicitaires (PDF)

Guest Editorial
Needed: Clinical Research in Mood Disorders
Martin Alda, Michael Bauer
(PDF)

In Review
The Antisuicidal and Mortality-Reducing Effect of Lithium Prophylaxis: Consequences for Guidelines in Clinical Psychiatry
Bruno Müller-Oerlinghausen, Anne Berghöfer, Bernd Ahrens
(PDF)

Lithium Augmentation Therapy in Refractory Depression: Clinical Evidence and Neurobiological Mechanisms
Michael Bauer, Mazda Adli, Christopher Baethge, Anne Berghöfer, Johanna Sasse, Andreas Heinz, Tom Bschor
(PDF)

Prophylaxis Latency and Outcome in Bipolar Disorders
Christopher Baethge, Leonardo Tondo, Irene M Bratti, Tom Bschor, Michael Bauer, Adele C Viguera, Ross J Baldessarini
(PDF)

Review Paper
Clinical Features of Bipolar Disorder With and Without Comorbid Diabetes Mellitus
Martina Ruzickova, Claire Slaney, Julie Garnham, Martin Alda
(PDF)

The Cortisol Awakening Response in Bipolar Illness: A Pilot Study
Dorian Deshauer, Anne Duffy, Martin Alda, Eva Grof, Joy Albuquerque, Paul Grof
(PDF)

Implementing Quality Management in Psychiatry: From Theory to Practice—Shifting Focus From Process to Outcome
Brent M McGrath, Raymond P Tempier
(PDF)

Original Research
Mental Disorders and Reasons for Using Complementary Therapy
Badri Rickhi, Hude Quan, Sabine Moritz, Heather L Stuart, Julio Arboleda-Flórez
(PDF)

Readiness to Participate in Psychiatric Research Daniele Zullino, Philippe Conus, François Borgeat, Charles Bonsack
(PDF)

Toward Benchmarks for Tertiary Care for Adults With Severe and Persistent Mental Disorders
Alain D Lesage, Daniel Gélinas, David Robitaille, Éric Dion, Diane Frezza, Raymond Morissette
(PDF)

Brief Communication
Patient Attitudes Regarding Causes of Depression: Implications for Psychoeducation
Janaki Srinivasan, Nicole L Cohen, Sagar V Parikh
(PDF)

Book Reviews
(PDF)

Helping the Helpers Not to Harm: Iatrogenic Damage and Community Mental Health.
Reviewed by
Peter Moore, MD, FRCPC

L’Homme de Vérité.
Revue par
Maurice Dongier, MD, FRCPC

Letters to the Editor
(PDF)

Re: The Combined Use of Atypical Antipsychotics and Cognitive-Behavioural Therapy in Schizophrenia

Reply: The Combined Use of Atypical Antipsychotics and Cognitive-Behavioural Therapy in Schizophrenia

Re: Should Psychologists Be Granted Prescription Privileges? A Review of the Prescription Privilege Debate for Psychiatrists

Reply: Should Psychologists Be Granted Prescription Privileges? A Review of the Prescription Privilege Debate for Psychiatrists

Re: Should Psychologists Be Granted Prescription Privileges? A Review of the Prescription Privilege Debate for Psychiatrists

Reply: Should Psychologists Be Granted Prescription Privileges? A Review of the Prescription Privilege Debate for Psychiatrists

Breath-Holding in Anxiety Disorders

Bright Light, Serotonin Turnover, and Psychological Well-Being

Review Paper

Implementing Quality Management in Psychiatry: From Theory to Practice—Shifting Focus From Process to Outcome

Brent M McGrath, BSc, MSc¹, Raymond P Tempier, MD, MSc, FRCPC²

Ten years have come and gone since Ellwood’s prediction that, to “move medicine forward as a sound applied biological and social science, adoption of a technology like outcome management is essential” (1). Clinical outcome measures and databases have been developed for many medical specialties, but psychiatry continues to lag behind (2–5). Several reasons have been given to explain this delay, including the lack of concrete definitions of what constitutes good quality care, a lack of valid and reliable outcome measures, and a historical assumption that the expert knows best in terms of changes in and improvements to patient health. Today, most clinicians do not need to be convinced of the importance of clinical outcome measurement and management (6,7) for continuous quality improvement (4,8–10), as well as analysis of treatment effectiveness (11) and cost-efficiency (12,13). However, the field of psychiatry encompasses a highly complex range of interacting services and diversified interventions (14). Further, mental health care differs from mainstream medical-surgical care (15) in that the relation between a patient’s diagnosis and the type, intensity, and duration of care is not always straightforward (16). As such, the need to implement effective means of monitoring, evaluating, and improving treatment outcome is an overriding issue in today’s mental health system (5). With the increasing use of evidence-based medicine and the welcome introduction of innovations such as computer technology into daily practice, the opportunity now exists to link these new approaches to quality measurement and assessment within psychiatry (9). The increasing emphasis on the satisfaction of patient-clients (17), balanced against the need for cost-efficient treatment, is another dimension that makes quality management an ever-increasing concern for mental health care providers (16): the growing tide of health consumerism is one of the most compelling forces for the implementation of mental health information management and quality-measurement systems (18). Successful functioning and participation in this new era of mental health care will be greatly enhanced by, and may even depend upon, the development and implementation of new and powerful management tools (19).

Advances in the use of computers have made data pooling and data management one of the most rapidly developing sciences (1). Properly programmed, computers are capable of dramatically enhancing information management in psychiatric care (20,21), yet it is still the most underutilized science for outcome management in psychiatry. A general principle should apply to developing and implementing computer-based management systems in the field of medicine: data collection should be exhaustive but simple, and systems should be user-friendly (3). When building up a system of information in psychiatry, a balance of simple, exhaustive, and valid data reflecting accurate activities according to a biopsychosocial model (22) should be the main goal (23). It is important that the system be flexible and easily modified to meet new requirements. The current state of ambulatory medical records (24,25) and the problem of moving reliable data from them into a computerized database are among the most serious technical barriers to computerized outcome management (1). Improving quality of care and patient outcome requires sophisticated systems for collecting comprehensive data on patient populations and patient satisfaction, as well as on treatment costs, settings, modalities, and efficacy (26). At its core, a quality-management system will have information on individual patients (27). Thus, a process that focuses on medical record documentation (28) represents an initial step in implementing a quality-management system (29). According to Guze and associates (27), a quality-management system in a psychiatric setting must apply analytic methods to measure demonstrable improvement in quality over time (12) in health practice, policy, and research.

This paper discusses the importance of a quality-management program in psychiatry. We appraise the value of continuous quality management from the perspectives of clinical administrators, psychiatrists, and patients. We discuss the importance of shifting focus from the process measurement of treatment quality to outcome. We explore treatment outcomes and their measurement and present a model for selecting not only dimensions of outcome to measure but also the measurement tools themselves. We conclude with some remarks on our experiences implementing and using a computerized quality management system

The View for Multiple Perspectives

As Burlingame and colleagues state, “the age of treatment accountability in mental health care has arrived” (30). Further, Canadians have recently reaffirmed their desire for a national system of accountability (28). Those involved in psychiatric services need information and the means to manipulate and use it (20). Administrators need information to assess both the quality and the cost-effectiveness of treatment. Clinicians require outcome data to ensure rational decision making and to help them better understand the intricate relation of treatment, clinical course, and prognosis. As health care consumers, patients deserve tangible results outlining their treatment progress and improvement. Outcome management absolutely depends on the participation and cooperation of the entire health enterprise (1). As an ideal, Berwick notes that psychiatrists must establish and hold to a shared vision of a health care system undergoing continuous improvement (31). In the coming paragraphs, we focus on quality management from the perspective of the 3 most important partners: the administrator, the clinician, and the patient.

Clinical Administrator Perspective
Administrators in psychiatry are becoming increasingly accountable for both the cost-efficiency and the quality of care provided in their centres (29). When considering the patient population as a whole, administrators must be able to determine whether the availability and use of services accurately reflects patient needs. Only with information in hand can clinical managers turn to data to make program decisions and to empirically evaluate the efficacy of those decisions (19). It is important that administrators view all outcome assessment data as informational, using assessed failure to achieve significant improvement or change as an opportunity to work with practitioners to improve treatment or assessment. In this regard, where perceived threats to employment and lowered incentives exist, deceptively derived positive outcomes rather then quality improvement may result (30). Indeed, for quality management to work, it must be viewed as a tool for positive change and not as a measurement of clinical performance (32).

Physician Perspective
In the past, psychiatrists, with the appropriate level of involvement from their patients, were the sole source of authority and legitimacy in the clinical decision-making process. Today, the psychiatrist–patient relationship is no longer uninfringeable by the outside world (12). Because there is variance in our practice, some of it unsupported by scientific evidence (15,33,34), the question is no longer whether there will be intervention in mental health services to assure quality but who will intervene and what methods will be used (35). According to Ellwood, if psychiatrists want to remain in control of their profession, they must have the motivation to track and evaluate mental health outcomes routinely (1). Indeed, contrary to some suggestions (32), quality improvement has little chance of success in mental health care organizations without the understanding, the participation, and in many cases the leadership of individual psychiatrists (7,31), because it is psychiatrists who retain ultimate responsibility for the delivery of care (26). Further, psychiatrists are the only mental health professionals who can make a comprehensive biopsychosocial diagnosis. Psychiatrists need to be able to monitor their patients’ treatment progress objectively by evaluating quantitative outcome data. Accordingly, psychiatrists need a powerful management tool to enable them to anticipate and evaluate the impact of medical care on their patients’ quality of life (1). It is important that such tools be user-friendly, that they be integrated with patient care routines, and that they not disrupt the ongoing treatment of patients (2). Further, they must demonstrate advantages of empirically based, compared with feeling-based, decisions (30). Psychiatrists must not feel burdened with tedious documentation and paperwork or restricted in their professional freedom (4).

Patient Perspective
The patient perspective is even more important for outcome assessment, in part because many important outcomes (such as patient satisfaction, attitudes toward treatment, and quality of life) can be determined only by asking the patient (19,36). There is also growing recognition that patients, as consumers of care, are important health care partners (18). Several studies have noted a relation between treatment compliance and patients’ understanding and awareness of their treatment progress (37,11): patients who were presented with a graph representing their treatment progress displayed enhanced treatment compliance (37). Further, assessing patient satisfaction provides a particularly valuable reflection of the consumer perspective on quality of care (38); however, it would be wrong to conclude that individuals perceiving treatment more positively necessarily show better outcome (39). Thus, it is important to assess not only patients’ subjective perceptions of treatment progress and outcome but also more objective perceptions. Murphy underlines the need for patient input by noting that even when “those who need help are in contact with mental health services this does not necessarily mean that their needs are being taken care of” and that “dissatisfaction rather than recovery causes patients to drop out of treatment” (40). Indeed, data on attitudes, satisfaction, and quality of life, for example, allow for examination of the interpersonal indicators of quality health care (41).

Quality Management—A Team Approach
Ideally, an outcome-management system will consider a range of perspectives, what each values about the operation of the mental health system, and what each wants to know about it (18). Change has become the rule, and the daily interplay between administrator and practitioner will be the fundamental driver of positive change (26). Psychiatrists and mental health care administrators who use outcome assessment to study and apply the principles of continuous quality management daily, will probably experience better efficiency, greater effectiveness, lower cost, and more satisfied patients. They will be able to make better decisions and carry them out more faithfully (31,42). As several authors have observed, the more patients improve, the more satisfied they are (43,44). In this regard, feedback from patient-clients provides the data needed to establish a reiterative process of continuous quality improvement (18): “efficient communications flow between all partners will be the energizing catalyst that elevates outcome measurement to outcome management, the process by which treatment practices will be continually measured, reviewed, tested, and redefined toward greater effectiveness, efficiency, and value” (26). Indeed, the regular feedback of information to participating stakeholders—especially treating psychiatrists and their patients—is perhaps the most important part of an outcome-management system (45,36). Apart from promoting continuous quality improvement, such feedback also keeps various partners enthusiastically involved in the initiative by giving them tangible results for their effort. It is important that the primary focus of any quality-management system be improved quality of care and treatment effectiveness, with cost-effectiveness a welcome and likely companion. Ethically managed psychiatry aspires to assign important but limited resources to achieve the best use and not only to reduce costs (46). When available, outcome data help demonstrate the relative benefits of existing treatment alternatives. Ultimately, outcome management can generate facts and insights that help clinicians and administrators make sounder decisions (1). In the next section, we shift attention from process to outcome.

Shifting Attention From Process to Outcome

In 1982, Donabedian and colleagues presented a framework for better understanding the different aspects of quality in health care (46). Within this framework, the relation between resource expenditures and quality of care can be examined systematically. These authors distinguish among quality related to structure (that is, the provider’s physical, organizational, and technological resources), to process (that is, how care is provided to patients), and to outcome (that is, the results of care) (Figure 1) (4,33,46).

Figure 1 Interactive characteristics of quality in health care (McCarthy and others [6])

Structure, process, and outcome each relate to the overall quality of care. It has been assumed that these 3 types of evaluation measures are highly related and that structure and process measures can serve as proxies for outcome measures. However, only when the relation between structure or process and outcome has been established is it legitimate to substitute measures of structure or process for measures of outcome. Often, both structure and process are at best indirect assessment measures (or indicators) of the most critical aspect of quality health care—its effect, or outcome, for the patient (6)—because only outcomes summarize all that has been achieved (4). To evaluate the overall quality and efficacy of health care, it is at minimum essential to assess measures of process and outcome (6). Process measures are relatively inexpensive and provide quick feedback; therefore, they have received the most attention to date (48). However, their correlation to long-term outcome cannot be determined without formally assessing outcomes (49). For these reasons, outcomes are a much more direct indicator of quality than are either structure or process (50). Experience has taught us how difficult it is to specify which variable from an interrelated array of variables is causally related to an observed outcome (39). Negative outcomes have been found to be associated with more severe psychiatric status, lower quality-of-life ratings, and lower levels of care (51); positive outcomes are thus the strongest indicator of high-quality treatment (26). That said, quality measurement and management should focus on treatment outcomes. It is here that the gap between theory and practice is bridged: quality is ultimately improved through measurement (4) and subsequent comparison with established guidelines and standards of care (52–55). If one treatment costs 5 to 10 times more than another treatment, we must be able to demonstrate that the outcome varies significantly (42). The increased need for reliable data is balanced by the increased availability of outcome measurement tools and data management technologies; this will assist us to assess and refine professional performance (26).

Defining and Measuring Treatment Outcome

In the past, success was often measured in terms of physical location, with treatment failures occurring in the hospital and treatment successes in the community (56). Today, as the focus of clinical care has shifted from the inpatient setting—where quality assurance originated—to ambulatory and community care, quality measures must assess care at a range of levels and in various settings (8). This, combined with the confusion and controversy regarding definitions (57) and recommended approaches for outcome evaluation (32), helps explain the hesitancy in initiating such quality- management systems. There are many dimensions of psychiatric treatment outcomes, such as social level of functioning (58), economic level of functioning, relapse rate, and patient satisfaction with treatment (2). Further, as attention to clinical outcomes has increased (59), it has been difficult to agree on what should be measured (37) and how this should be done (32,38). Clearly, patients are the central concern of outcome management and research (16). Accordingly, to understand the meaning of high quality, we must know something about the specific patients to whom it applies (60). For example, treatment outcome may need to be defined in limited terms for a patient suffering from long-term schizophrenia who experiences remission of positive symptoms but is left with negative symptoms and an inability to develop meaningful relationships. In such a case, improved treatment outcome could include the patient’s ability to sustain good hygienic practices or engage in minimal social contact: “it is important that psychiatrists and other members of the mental health care team impress upon nonclinician administrators the fact that for some patients, progress is at times nearly imperceptible and that the quality of care cannot always be measured by dramatic results, such as employment status” (60). Thus, one must be sure that the chosen outcome measures are suitable for the patient population evaluated (30). Also, the outcomes that different stakeholders prioritize will depend on what each considers an essential area for improvement (18,59), as well as on each stakeholder’s goals.

Figure 2 Conceptual model for selecting outcome measures and routine dimensions for outcome assessment in psychiatry

It is important that agreement be reached on the most significant dimensions for outcome management and on the attributes outcome measures should possess. To that end, we propose a conceptual model of outcome dimensions and minimum outcome–measure properties, based on recommendations cited in the literature (Figure 2). Goeree outlines the 5 most common and consistent dimensions that appear in the mental health field (49). These include symptomatology, patient functioning, quality of life, patient satisfaction, and cost. To this, we add 7 minimum properties or criteria that outcome measures should possess for both usefulness and consensus (4,8,30,49,61–64). First, a selected outcome measure should be widely accepted and clinically credible (49,61). Second, outcome measures should be comprehensive: the instruments should assess all important treatment variables. Third, outcome measures should be suitable and meaningful for the user’s objectives (4,8,30). Suitability refers to the measure’s appropriateness: is it measuring what the user intends it to measure? For example, if symptom improvement is the goal, the measure must assess a clinically important dimension that is relevant to the underlying disorder. Meaningfulness is as much a quality of what is to be measured as it is of the instrument itself. Users must be sure that what they are measuring is useful in terms of what they are trying to accomplish. Fourth, measures should be sensitive to change (62). This will allow for the measurement of treatment progress and improvement over time, which may indicate the quality of care provided. Fifth, outcome measures should have established psychometric properties (8,30). These include high reliability (both internal and test–retest) and validity. Only then can one confidently base decisions on results obtained from outcome measures. Sixth, data obtained from a particular outcome measure should be amenable to statistical analyses (49,61). This will allow for accurate interpretation of outcome data. Finally, a selected outcome measure should be straightforward to administer and should provide results that can be acted upon. A practical measurement tool will be easier to integrate into mental health practice (30). Results that can be acted upon means that any deficiencies revealed by the measure will have solutions within the user’s (that is, the clinician’s) control (8).

The evolution of outcome measurement has resulted in multiple measures with good psychometric properties reflecting various sources, content areas, and measurement methodologies for assessing the complex and multidimensional nature of change (63). Our model is intended only as a guide to help psychiatrists and researchers wade through the murky and often crowded waters of outcome measurement. Examples of outcome measures include the Global Assessment of Functioning Scale (GAF) (65), the Sickness Impact Profile (SIP) (66), the Drug Attitude Inventory (DAI) (67), and the Positive and Negative Syndrome Scale (PANSS) (68). The American Psychiatric Association has published a more detailed list (69). Obviously, no single measure can comprehensively assess the quality of care being provided; which measures are ultimately used will depend on the type of patient population being managed and on the specific outcome dimensions prioritized by the various stakeholders.

Conclusion

In a climate of fiscal restraint and health care cutbacks, patient needs may not be met, or they may be met inadequately. Without evaluation, one cannot determine whether or to what extent patient needs are met, what patients’ changing needs are over time, and how best to respond to these needs (49,61). As Kissling and associates point out, “regular evaluation of quality is the indispensable basis for continuous quality improvement” (4). The need to continually inform and improve care delivery by collecting and channelling treatment data is more apparent now than ever before. Although the efficacy of some treatments is known under the special conditions of randomized controlled trials (RCTs), we seldom know how those treatments perform in everyday practice, wherein patient improvement is rarely rated (43). RCTs are less well suited to assessing clinical effectiveness (defined as actual results obtained by an intervention in unselected patient populations seeking care from routine clinical services) (70). Quality management through outcomes assessment will broaden our knowledge base (57) and provide much-needed data on underinvestigated, real-world practices (for example, combination antipsychotic therapy in the treatment of schizophrenia) (71,72). Techniques such as assessment of fidelity (32) provide important information on adherence to evidence- based practices; however, they rely solely on the currency and comprehensiveness of clinical practice guidelines (CPGs). In this regard, “no single CPG appears to address all treatment needs faced by practising clinicians” (52). Further, Montgomery recently made the important point that “prescribing-based evidence” provides data that “complement” data gathered from RCTs during the development of “evidence-based prescribing” guidelines (73). Accordingly, good management should be based on continuous and permanent collection of clinical data; it should not be restricted to elementary data, such as the psychiatric diagnosis, but expanded to the most clinically significant items, such as patients’ psychosocial characteristics (74). These include, for example, patient attitudes toward medication, their treatment knowledge, and their quality of life. Administrators and psychiatrists alike require a system to assess not only the need for but also the effects of various health care interventions—a system offering accurate and reliable data that psychiatrists and their patients can use to make better overall decisions about treatment (75,76). With the availability of accurate and detailed quality- and treatment-outcome data, there will be much progress in the quality of patient care (26) and, perhaps, lowered cost. It is important that outcome measurement and quality management not be viewed as simply just another research project but as integral and vital components of the overall mental health care system. How, then, do we incorporate such a quality-management system into the daily routine of mental health care providers without adding more cost and workload to an already overburdened system?

With the advent of computer technology, data analysis has become less labour intensive. Current statistical software does not require extensive training in statistical theory, only basic understanding of its procedures. While few programs to date have been introduced specifically for use in a psychiatric setting, we have recently incorporated one such program into our daily practice: the Psychiatry Quality Measurement (PQM), a computerized documentation and quality-measurement system (9). According to its developers, the PQM system has been designed to document all aspects of a psychiatric patient’s treatment, admission, and subsequent follow-up; reduce the time spent on administration; and increase the availability of data from a physician’s practice. The system incorporates a program displaying descriptive statistics and can calculate a large range of quality measures, based either on the full population in the database or on preselected subgroups. Data can also be exported for more extensive statistical analysis. The system is designed to replace traditional paper-and-pencil charting systems, which promotes continuous and less time-consuming quality assessments. The program itself has been revised several times to incorporate advice from individual users. It is password-protected and can be networked through various computers, allowing both administrators and psychiatrists to monitor patient outcomes and treatment progress. Although we received no formal training, we found the system quite easy to implement and use. The biggest limitation is the amount of time required to enter patient data from clinic charts into the system. Even with this limitation, however, the ultimate benefits far outweigh the cost.

It is now apparent that mental health care providers must track treatment progress and outcome to improve the quality of care they provide. Even with the recent increase in outcome- measurement tools and computer technology, such management procedures have only been established in a few, research-based settings. For psychiatry to expand its knowledge base and better help patients, new methods to improve quality should be explored. Implementing a system like the one described will offer the opportunity to track the clinical effectiveness of underresearched practices and to identify which type of patient responds most favourably to which treatment. It will also help to determine whether psychiatric services meet or exceed consumer expectations and practice guidelines recommendations, as well as whether these services produce clinically significant outcomes.

Funding and Support

The present report was prepared without the financial support of any granting agency. The Lundbeck Institute is acknowledged for providing the PQM at no charge.

References

1. Ellwood PM. Shattuck lecture outcomes management: a technology of patient experience. N Engl J Med 1988;318:1549–56.

2. Davis DF, Fong ML. Measuring outcomes in psychiatry: an inpatient model. Journal of Quality Improvement 1996;22(2):125–33.

3. Modai I, Ritsner M, Silver H, Kurs R. A computerized patient information system in a psychiatric hospital. Psychiatr Serv 2002;53:476–8.

4. Kissling W, Seemann U, Piwernetz K. Quality management in psychiatry. Int Clin Psychopharmacol 2001;16(Suppl 3);S15–S24.

5. Lamb HR. Introduction. Outcomes assessment in mental health treatment [special issue]. Psychiatr Serv 1998:2.

6. McCarthy PR, Gelber S, Dugger DE. Outcome measurement to outcome management: the critical step. Adm Policy Ment Health 1993;21(1):59–70.

7. Carman JM, Shortell SM, Foster RW, Hughes EFX, Boerstler H, O’Brien JL, and others. Keys for successful implementation of total quality management in hospitals. Health Care Manage Rev 1996;21(1):48–60.

8. Hermann RC, Palmer RH. Common ground: a framework for selecting core quality measures for mental health and substance abuse care. Psychiatr Serv 2002;53:281–7.

9. Joubert AF. PQM—psychiatry quality measurement. Int Clin Psychopharmacol 2001;16(Suppl 3):S25–S33.

10. Boerstler H, Foster RW, O’Conner EJ, O’Brien JL, Shortell SM, Carman JM, Hughes EF. Implementation of total quality management: conventional wisdom versus reality. Hosp Health Serv Adm 1996;41:143–59.

11. Smith GR, Rost KM, Fischer EP, Burnam MA, Burns BJ. Assessing the effectiveness of mental health care in routine clinical practice: characteristics, development, and users of patients outcomes modules. Eval Health Prof 1997;20(1):65–80.

12. Bartlett JC. Practice guidelines and outcomes management: future roles for psychiatrists. In: Lazarus JA, Sharfstein SS, editors. New roles for psychiatrists in organized systems of care. Washington (DC): American Psychiatric Press; 1998. p 123–40.

13. Yates PD. Understanding managed care. In: Guimon J, Sartorius N, editors. Manage or perish? The challenges of managed mental health care in Europe. New York: Kluwer Academic/Plenum Publishers; 1999. p 1–9.

14. Bollorino F, Piaggio A. Research and communication of mental health data. In: Guimon J, Sartorius N, editors. Manage or perish? The challenges of managed mental health care in Europe. New York: Kluwer Academic/Plenum Publishers; 1999. p 375–80.

15. Boronow JJ, Sharfstein SS. The identity of the field in the context of changing roles. In: Lazarus JA, Sharfstein SS, editors. New roles for psychiatrists in organized systems of care. Washington (DC): American Psychiatric Press; 1998. p 239–60.

16. Mirin SM, Namerow MJ. Why study treatment outcomes? Hosp Community Psychiatry 1991;42:1007–13.

17. McGrath BM, Tempier R. Experiencing the PQM in a Canadian clinic for long-term care and follow-up. The Lundbeck Institute Magazine (Denmark). Forthcoming.

18. Campbell J. Toward collaborative mental health outcomes systems. New Dir Ment Health Serv 1996;71:69–79.

19. Butler SF, Docherty JP. A comprehensive system for value accounting in psychiatry. Value Accounting 1996;23:479–91.

20. Greist JH. Computers and psychiatry. Psychiatr Serv 1995;46:989–91.

21. Rouan G, Marks E, Tuchfarber B, Redington TJ, Lorenzi N. Development, effects, and educational outcomes of reporting from a clinical database. Medical Decision Making 1991;11(Suppl):S80–S89.

22. Engel GL. The need for a new medical model: a challenge for biomedicine. Science 1977;196(4286):129–36.

23. Tempier R. Minimal psychiatric summary: the French way. In: Guimon J, Sartorius N, editors. Manage or perish? The challenges of managed mental health care in Europe. New York: Kluwer Academic/Plenum Publishers; 1999. p 143–51.

24. Ellwood PM, Lundberg GD. Managed care: a work in progress. JAMA 1996;276:1083–6.

25. Luo J, Hales RE, Servis M, Gill M. Use of personal digital assistants in consultation psychiatry. Psychiatr Serv 2002;53:271–2, 279.

26. Geraty RD. Administrative psychiatry and managed care: advances and opportunities. In: Lazarus JA, Sharfstein SS, editors. New roles for psychiatrists in organized systems of care. Washington (DC): American Psychiatric Press; 1998. p 71–85.

27. Guze BH, Duim S, McGuire MT, Moreau D, Poster E. Databases for clinical psychiatry. MD Comput 1996;13:210–5.

28. Romanow RJ. Building on values: the future of health care in Canada. Ottawa: National Library of Canada; 2002. ISBN 0-662-33043–9.

29. Baker JG, Shanfield SB, Schnee S. Using quality improvement teams to improve documentation in records at a community mental health center. Psychiatr Serv 2000;51:239–42.

30. Burlingame GM, Lambert MJ, Reisinger CW, Neff WM, Mosier J. Pragmatics of tracking mental health outcomes in a managed care setting. J Ment Health Adm 1995;22:226–36.

31. Berwick DM. Continuous improvement as an ideal in health care. N Engl J Med 1989;320(1):53–6.

32. Bilsker D, Goldner EM. Routine outcome measurement by mental health-care providers: is it worth doing? Lancet 2002;360:1689–90.

33. Fauman MA. Monitoring the quality of psychiatric care. Psychiatr Clin North Am 1990;13(1):73–85.

34. Marder SR. Can clinical practice guide a research agenda? Schizophr Bull 2002;28:127–9.

35. Kessner DM, Kalk CE, Singer J. Assessing health quality-the case for tracers. N Engl J Med 1973;288:189–94.

36. Smith GR, Manderscheid RW, Flynn LM, Steinwachs DM. principles for assessment of patient outcomes in mental health care. Outcomes assessment in mental health treatment [special issue]. Psychiatr Serv 1998:3–6.

37. Marks IM, Blanes T, McKenzie N. Computerized clinical benefit-cost audit of mental health care I: Theoretical and practical issues. J Ment Health 1995;1:63–9.

38. Burnam MA. Measuring outcomes of care for substance use and mental disorders. New Dir Ment Health Serv 1996;71:3–17.

39. Schulberg HC, Bromet E. Strategies for evaluating the outcome of community services for the chronically mentally ill. Am J Psychiatry 1981;138:930–5.

40. Murphy HBM. Results from evaluating a Canadian regional mental health program. Hosp Community Psychiatry 1973;24:533–9.

41. Davis AR, Ware JE Jr. Involving consumers in quality of care assessment. Health Aff 1988;Spring:32–48.

42. Goldstein LS. Linking utilization management with quality improvement. Psychiatr Clin North Am 1990;13(1):157–69.

43. Marks I. A computerized system to measure treatment outcomes and cost. Psychiatr Serv 1996;47:811–2.

44. McKenzie N, Blanes T, Marks IM. Computerized clinical benefit–cost audit of mental health care II: Time input, costs, patient satisfaction. J Ment Health 1995;1:71–8.

45. Clardy JA, Booth BM, Smith LG, Nordquist CR, Smith GR. Implementing a statewide outcomes management system for consumers of public mental health services. Outcomes assessment in mental health treatment [special issue]. Psychiatr Serv 1998:25–9.

46. Guimon J. Ethical problems in the practice of pyschiatrists under managed care. In: Guimon J, Sartorius N, editors. Manage or perish? The challenges of managed mental health care in Europe. New York: Kluwer Academic/Plenum Publishers; 1999. p 391–5.

47. Donabedian A, Wheeler JRC, Wyszewianski L. Quality, cost, and health: an integrative model. Med Care 1982;20:975–92.

48. Hermann RC, Finnerty M, Provost S, Palmer RH, Chan J, Lagodmos G, and others. Process measures for the assessment and improvement of quality of care for schizophrenia. Schizophr Bull 2002;28(1):95–104.

49. Goeree MD. Evaluation of programs for the treatment of schizophrenia: a health economic perspective. Ottawa: Health Canada; 1994. ISBN 0-662-22389-6: Cat. No. H39-302/1994E.

50. Srebnik D, Hendryx M, Stevenson J, Caverly S, Dyck DG, Cauce AM. Development of outcome indicators for monitoring the quality of public mental health care. Outcomes assessment in mental health treatment [special issue]. Psychiatr Serv 1998:13–9.

51. Russo J, Roy-Byrne P, Jaffe C, Ries R, Dagadakis C, Avery D. psychiatric status, quality of life, and level of care as predictors of outcomes of acute inpatient treatment. Outcomes assessment in mental health treatment [special issue]. Psychiatr Serv 1998:56–63.

52. Bhanji NH, Tempier R. Managing schizophrenia during the stable phase: is there consensus among practice guidelines? Can J Psychiatry 2002:47:76–80.

53. Finnerty M, Altmanberger R, Bopp J, Carpinello S, Docherty JP, Fisher W, and others. Using state administrative and pharmacy data bases to develop a clinical decision support tool for schizophrenia guidelines. Schizophr Bull 2002;28(1):85–94.

54. Buchanan RW, Kreyenbuhl J, Zito JM, Lehman A. The schizophrenia PORT pharmacological treatment recommendations: conformance and implications for symptoms and functional outcome. Schizophr Bull 2002;28(1):63–73.

55. Bassett A, Addington D. Canadian clinical practice guidelines for the treatment of schizophrenia. Can J Psychiatry 1998;43(Suppl 2, revised):25S–41S.

56. Erickson RC. Outcome studies in mental hospitals: a review. Psychol Bull 1975;82:519–40.

57. Bachrach LL. Assessment of outcomes in community support systems: results, problems, and limitations. Schizophr Bull 1982;8(1):39–60.

58. Dickerson FB. Assessing clinical outcomes: the community functioning of persons with serious mental illness. Outcomes assessment in mental health treatment [special issue]. Psychiatr Serv 1998:7–12.

59. Pincus HA, Zarin DA, West JC. Peering into the ‘black box’: measuring outcomes of managed care. Arch Gen Psychiatry 1996;53:870–7.

60. Bachrach LL. Planning high-quality services. Hosp Community Psychiatry 1991;42:268–9.

61. Goeree MD. Evaluation of programs for the treatment of schizophrenia: Part II, a review of selected programs in Canada. Ottawa (ON): Health Canada; 1996. ISBN 0-662-24140-1: Cat. No. H39-302/2-1996E.

62. Rosenblatt A, Attkisson CC. Assessing outcomes for sufferers of severe mental disorder: a conceptual framework and review. Eval Program Plann 1993;16:347–63.

63. Ogles BM, Lunnen KM. Assessing outcome in practice. J Ment Health 1996;5(1):35–45.

64. Dazord A. Assessing psychotherapy outcome: the state of the art. In: Guimon J, Sartorius N, editors. Manage or perish? The challenges of managed mental health care in Europe. New York (NY): Kluwer Academic/Plenum Publishers; 1999. p 381–9.

65. American Psychiatric Association: Diagnostic and statistical manual of mental disorders. 4th ed. Text revision. Washington (DC): American Psychiatric Association; 2000.

66. Bergner M, Bobbitt RA, Carter WB, Gilson BS. The Sickness Impact Profile: development and final revision of a health status measure. Med Care 1981;19:787–805

67. Hogan TP, Awad AG, Eastwood MR. A self-report scale predictive of drug compliance in schizophrenics: reliability and discriminative validity. Psychol Med 1983;13:177–83.

68. Kay SR, Fiszbein A, Opler LA. The Positive and Negative Syndrome Scale (PANSS) for schizophrenia. Schizophr Bull 1987;13:261–76.

69. American Psychiatric Association: Handbook of psychiatric measures. Washington (DC): American Psychiatric Association; 2000.

70. Tansella M, Thornicroft G. A conceptual framework for mental health services: the matrix model. Psychol Med 1998;28:503–8.

71. Clark RE, Bartels SJ, Mellman TA, Peacock WJ. Recent trends in antipsychotic combination therapy of schizophrenia and schizoaffective disorder: implications for state mental health policy. Schizophr Bull 2002;28(1):75–84.

72. Miller AL, Craig CS. Combination antipsychotics: pros, cons, and questions. Schizophr Bull 2002;28:105–9.

73. Montgomery S. Evidence-based prescribing or prescribing-based evidence? Abstract S0404. Int J Psychiatry Clin Pract 2002;6:229.

74. Pelc I, Joosten J, From L, Bergeret I, Ledoux Y, Tecco J. The Minimal Psychiatric Summary (MPS) and the current organization in mental health care. In: Guimon J, Sartorius N, editors. Manage or perish? The challenges of managed mental health care in Europe. New York: Kluwer Academic/Plenum Publishers; 1999. p 131–42.

75. Greist JH. The computer as clinician assistant: assessment made simple. Psychiatr Serv 1998;49:467–72.

76. Posternak MA, Zimmerman M, Soloman DA. Integrating outcomes research into clinical practice. Psychiatr Serv 2002;53:335–6.

Author(s)

Manuscript received October 2002, revised, and accepted February 2003.

1. Graduate Student, Department of Psychiatry, McGill University Health Centre, Community Mental Health Centre, Montreal, Quebec.

2. Associate Professor of Psychiatry, Department of Psychiatry, McGill University; Chief of Continuing Care Services, McGill University Health Centre, Montreal General Hospital, Montreal, Quebec.

Address for correspondence: Dr R Tempier, McGill University Health Centre, Montreal General Hospital, 1650 Cedar Ave, Montreal, QC H3G 1A4

e-mail: raymond.tempier@muhc.mcgill.ca

1 | 2