Canadian Psychiatric Association
 

Editorial Credits/ Crédits éditorials

Subscription Rates /Prix d'abonnements

Advertising Rates / Tarifs publicitaires (PDF)


Original Research Expression of Depressive Symptoms in a Nonclinical Brazilian Adolescent Sample
Clarice Gorenstein, PhD, Laura Andrade, MD, PhD, Elaine Zanolo, Rinaldo Artes, PhD

(PDF)

Posttraumatic Stress Disorder and General Psychopathology in Children and Adolescents Following a Wildfire Disaster
Brett M McDermott, MBBS, Cert Child Psych, FRANZCP, Erica M Lee, BA, Dip Psych, Marianne Judd, BSc Psych, MEd, Peter Gibbon, PhD

(PDF)

Individual Change in Methylphenidate Use in a National Sample of Children Aged 2 to 11 Years
Elisa Romano, PhD, Raymond H Baillargeon, PhD, Isabel Fortier, PhD, Hong-Xing Wu, MSc4, Philippe Robaey, MD, Mark Zoccolillo, MD, Richard E Tremblay, PhD

(PDF)

An Introduction to Economic Evaluation: What’s in a Name?
Jeffrey S Hoch, MA, PhD, Carolyn S Dewa, MPH, PhD

(PDF)


Recherche Originale *Méthadone et Syndrome d’Apnées du Sommeil
Philippe Durst, MD, Jérôme Palazzolo, MD, PhD, Jean-Pierre Peyrelong, MD, Michel Berger, MD, Michel Chalabreysse, MD, Michel Billiard, PhD, André Vialle, MD

(PDF)


Review Paper
Using Metaanalysis to Evaluate Evidence: Practical Tips and Traps

Raymond W Lam, MD, FRCPC, Sidney H Kennedy, MD, FRCPC

(PDF)


Brief Communication
Experimental Affective Symptoms in Panic Disorder Patients

Thea Overbeek, MD, PhD, Koen Schruers, MD, PhD, Ine Docters van Leeuwen, BSc, Tineke Klaassen, MD, PhD, Eric Griez, MA, MD, PhD

(PDF)


Book Reviews
(PDF)

The Treatment of Drinking
Problems: A Guide for the Helping Professions.

Review by
Nady el-Guebaly, MD, FRCPC


Bipolar Disorder: A Clinician’s Guide to Biological Treatments.
Review by
Review by: Rakesh Jain MD, MPH
Shailesh Jain, MD, MPH


Handbook of Female Psychopharmacology
Review by
Laura Calhoun, FRCPC


Sleep and Dreaming: Scientific Advances and Reconsiderations.
Review by
Alan Douglass, MD


Cognitive-Behavioral Treatment of Obesity. A Clinician’s Guide.
Review by
Hany Bissada, MD, FRCPC


Cognitive Therapy of Personality Disorders. Second Edition.
Review by
W John Livesley



Letters to the Editor
(PDF)

Hemorrhages During Escitalopram–Venlafaxine– Mirtazapine Combination Treatment of Depression

Re: Lorazepam-Induced Prolongation of the QT Interval in a Patient With Schizoaffective Disorder and Complete AV Block

Reply: Lorazepam-Induced Prolongation of the QT Interval in a Patient With Schizoaffective Disorder and Complete AV Block

Lithium-Associated Anencephaly

Aripiprazole-Induced Seizure

Prevalence of Bipolar Disorder and Major Depression Among Patients Seen in Primary and Secondary Care in Finland

The Need for More Community Nursing for Adults With Intellectual Disabilities and Mental Health Problems

Improvement in Tardive Dyskinesia With Aripiprazole Use


Original Research

An Introduction to Economic Evaluation: What’s in a Name?

Jeffrey S Hoch, MA, PhD1, Carolyn S Dewa, MPH, PhD2

 

Objective: This paper describes the main types of economic evaluation techniques.

Method: To examine the strengths and limitations of different types of economic evaluations, we used a hypothetical example to review the reasoning underlying each method and to illustrate when it is appropriate to use each method.

Results: The choice of economic evaluation method reflects a decision about what should represent “success” and how success should be valued. Measures of benefit and cost must be considered systematically and simultaneously. Claiming that a new treatment is cost-effective requires making a value judgment based on the personal beliefs of the claimant. Even when cost and effect data are objective, a verdict of cost-effective is subjective. The conclusions of an economic study can change significantly, depending on which patient outcome is used to measure success.

Conclusions: Clinicians must be sure that important patient outcomes are not excluded from economic evaluations. Economic evaluation is a process designed to produce an estimate rather than a decision. New treatment can be more costly and still be cost-effective (if the extra benefit is valued more than the extra cost to produce it). However, since economic evaluation does not explicitly consider a decision maker’s available budget, a new treatment can be deemed cost-effective but too expensive to approve.

(Can J Psychiatry 2005;50:159–166)

Click here for author affiliations. 

Clinical Implications

  • When advocating on behalf of patients that the extra benefit of new treatment is worth the extra cost, measures of benefit and cost must be considered simultaneously.

  • Proclaiming a new treatment is cost-effective involves making a value judgment that comes from the author and not the data. New treatment can be more costly and still be cost-effective.

  • Clinicians must be sure that important patient outcomes are not excluded from economic evaluations.

Limitations

  • The purpose of this paper is to help the reader appreciate the different types of economic evaluation; as a result, it does not cover in detail outcome or costing methodology.

  • Simple economic evaluations like cost-effectiveness and cost–utility analysis do not consider the decision maker’s budget. A decision maker might deem new treatment to be cost-effective but too expensive to approve.

  • The results from most economic evaluations are descriptive (that is, producing estimates) not prescriptive (that is, producing decisions). Health care decision making is complicated, regardless of whether it occurs at a patient or at a country level; economic evaluation must be viewed as just one piece of the process.

Key Words: cost-effectiveness, cost–benefit, teaching economic evaluation, health economics

Résumé : Une introduction à l’évaluation économique : derrière le nom

There has been unprecedented progress in health care during the past few decades. However, as Clare Booth Luce once quipped, “No good deed goes unpunished.” We face the myriad innovations before us with scarce resources (such as money, time, and personnel). Thus progress has also left us with difficult choices. Insatiable demands for new treatments and services compel society to make difficult choices.

How should the choices be made? It has been suggested that “methods such as ‘what we did last time,’ ‘gut feelings,’ and even ‘educated guesses’ are not always better than organized consideration of the factors involved in a decision to commit resources to one use instead of another” (1). As a result, economic evaluation methods have been developed to help inform difficult choices by simultaneously considering costs and consequences of treatment interventions.

Many of the methods’ names have become familiar—cost–benefit analysis (CBA), cost-effectiveness analysis (CEA), and cost–utility analysis (CUA). Unfortunately, their meanings have not. When is CUA preferred to CBA? Can something be more costly and still be cost-effective? It is clear from the names of these economic evaluations that cost plays a prominent role. However, it is not as widely appreciated that the defining aspect of an economic evaluation has nothing to do with cost; it is how patient outcomes are treated that distinguishes one method from another.

This paper describes the main types of economic evaluation techniques. We examine their strengths and limitations from theoretical and applied perspectives. Using a hypothetical example, we also review the reasoning underlying each method and consider how this can be used to decide when it is appropriate to use each method.

The Relativity of it All

Once an investigator decides to incorporate an economic evaluation into a study, an appropriate comparator must be identified. To use a manufacturing metaphor, the new treatment or intervention under study and its comparator(s) are competing factories with different production processes. When measuring the economic efficiency of each “factory,” costs must be compared, relative to some measure of “output.” In health services, the “output” is patient outcome or treatment effectiveness.

We use a hypothetical case as a vehicle for our discussion. Suppose a research team has been asked to evaluate a novel program of assertive community treatment (PACT) for homeless people with severe and persistent mental illness. In addition to the standard PACT team configuration (that is, a psychiatrist, nurses, social workers, a substance abuse specialist, a vocational specialist, and so forth), a specialized employment and housing (EH) component has been added. PACT-EH will be compared with an intensive case management (ICM) program that is currently serving a similar population of individuals.

Every type of economic evaluation seeks to simultaneously estimate the trade off associated with cost and effectiveness. To truly know whether we are getting “value for money” requires examining PACT-EH’s extra cost (ΔC) and extra effect (ΔE), relative to ICM. The estimate of extra cost is made from the difference in average cost estimates. That is,

                         ΔC = Average CostPACT-EH> – Average CostICM

Similarly, the estimate of the extra effect comes from difference in the average effect estimates:

                         ΔE = Average EffectPACT-EH – Average EffectICM

Economic evaluations such as CBA, CEA, and CUA differ only in how average effect is conceptualized.

The importance of the decision about how treatment consequences are considered cannot be overstated. The type of economic evaluation one conducts and how the results can be interpreted are both predicated on this decision.

Choosing the Economic Evaluation Method

The choice of economic evaluation method reflects a decision about what should represent “success” and how that success should be valued. Assume the research team has collected data for 2 health outcomes (HEALTH1 and HEALTH2), 1 housing outcome (HOUSING1), and 2 employment outcomes (EARNINGS1 and EARNINGS2). Is it reasonable to choose just one outcome as the consequence of PACT-EH? Should all outcomes be used? If so, how should they be combined? The answers to these questions are reflected in the economic evaluation method chosen.

Table 1 describes common types of economic evaluations from which an investigator might choose. The great variety has led to some confusion (2,3); articles in the medical literature do not always correctly identify the type of economic evaluation that was conducted (4). This is in spite of the many articles (5–16) and books (17–21) that have been written to introduce economic evaluation concepts. Often, the source of problems involves how patient outcomes or effects were handled in the analysis (2). The remainder of the paper discusses how the different economic evaluation methods treat effects, the underlying rationale and limitations for each approach, and how estimates can be influenced.

Table 1  Types of economic evaluation 

Name 

Measurement units
for effect(s)a 

Strengths 

Limitations 


Cost–benefit analysis (CBA) 

All effects measured in dollars. 

The net benefit (NB) is easy to interpret.  For example, a new treatment’s extra benefits are worth more than the extra costs when NB > 0. 

  • It is difficult to measure the value of all health outcomes in dollars. 
  • There may be moral objections about the impact of ability to pay in the process of valuing the effects. 

Cost–utility analysis (CUA) 

Two effects (quality and length of life) whose product  is taken as quality-adjusted life years (QALYs). 

Patient outcomes involving both quality and length of life can be incorporated in the analysis. 

In theory, the QALY measure is “universal,” so that very different programs evaluatedwith QALYs can be compared. 

  • QALY measures vary by method. 
  • QALY measures vary by respondent. 
  • Society may value a QALY for different patient groups differently (for example, is a QALY gained for sufferers of erectile dysfunction valued the same as a QALY gained for sufferers of severe mental illness?) 

Cost-effectiveness analysis (CEA) 

One effect measured in “natural units”. 

There is one outcome and it is measured in its “natural units.” 

  • Only one outcome will represent the effect of treatment; however, other outcomes may be relevant. 

Cost-minimization analysis (CMA) 

No effects measured. 

There is only a need to collect cost data. 

  • Few treatments have identical outcomes. 
  • Researchers would likely need to collect the effect data to verify the “equal effect”
    assumption. 

aAll costs are measured in dollars. 

Cost-Minimization Analysis

For the purposes of our example, suppose the average cost of PACT-EH is $55 000 and the average cost of ICM is $50 000 (Table 2). Thus ΔC = $5000 (that is, $55 000-$50 000). In cost-minimization analysis (CMA), we focus solely on differences in costs or DC. The choice of CMA implies that the investigator is certain there are no differences between PACT-EH and ICM in any of the patient outcomes. CMA answers the question, “Is the extra effectiveness worth the extra cost?” In this example, ΔC > 0 indicates that PACT-EH is relatively more expensive. A CMA cannot answer the question, “Is the extra effectiveness worth the extra cost?” because effectiveness is not examined (it is presumed to be equivalent).

Two health treatments that unequivocally produce equivalent patient outcomes are infrequently observed. Despite this fact, the “Which is cheaper” reasoning associated with CMA persists; some have suggested this may be especially pernicious in mental health (22). In situations where there may be a difference in effectiveness (that is, ΔE ¹ 0), CMA is inappropriate because it will provide only one-half the picture; the cheaper option may harm the patient. Instead, researchers should adopt an economic evaluation that allows the competing treatments or interventions to have different effectiveness (that is, ΔE ¹ 0); the simplest of these types of economic evaluations is CEA.

Cost-Effectiveness Analysis

To answer the question “Is the treatment cost-effective?” one must do the following: first, estimate the extra cost (ΔC); second, estimate the extra effect (ΔE); third, compute the extra cost per extra effect (ΔC/ΔE); and fourth, judge whether the trade off estimated in step 3 is worth it.

CMA is simply step 1. To the extent that patient outcomes associated with the treatments differ, it is unhelpful to focus health care discussions solely on cost. In the same way that cheaper is not always better, so too is cheaper not always cost-effective. A CMA is not a proper substitute for a CEA when there is an appreciated difference in patient outcomes.

CEA uses one outcome as a measure of “effect.” Since 2 of the primary objectives of PACT-EH are stable housing and employment, our CEA should examine at least one of these outcomes. Suppose we decide to focus on the effect variable “Days of stable housing” (HOUSING1). The extra gain (ΔE) would be expressed as the difference in stable housing associated with PACT-EH versus ICM (that is, the mean of HOUSING1 for PACT-EH minus the mean of HOUSING1 for ICM). The difference in average HOUSING1 for PACT-EH and ICM is 10 days (from Table 2, 310 – 300 = 10 days). Step 3 involves calculating the conventional CEA statistic known as the incremental cost-effectiveness ratio (ICER = ΔC/ΔE). The ICER describes the extra cost per one additional patient outcome. For our evaluation, the ICER is

                         ΔC/ΔE = $5000 per 10 extra days of stable housing

                              = $500 per 1 extra day of stable housing

This estimate indicates that, compared with ICM, PACT-EH costs an average of $500 additional dollars per one additional day of stable housing.

In many economic evaluations, the ICER indicates that a new treatment is relatively more costly (ΔC > 0) and relatively more effective (ΔE > 0) than usual care. In these situations, to label something cost-effective is to express a personal opinion about the ICER “price tag,” based on the amount one is willing to pay to get an extra unit of effect. Forming an opinion about whether the ICER “price tag” represents a good value (step 4 of a CEA) generally involves a value judgment. According to the $500 daily estimate, PACT-EH is cost-effective if one believes the extra effect is worth the extra cost. The ICER is an estimate, not a decision about the cost and effect trade off. Ultimately, the decision maker is left with the responsibility for step 4 (that is, deciding whether the ICER represents a good deal; for example, is an additional day of stable housing worth $500?).

Further, while there is usually little controversy about how the outcome is measured for a CEA (for example, community tenure measured in days), there may be some concern regarding which outcome was chosen to represent success. Indeed, there may be many measures that could be selected as a reasonable effect of new treatment. This is especially true for a complex program with more than one objective, such as PACT-EH, where many outcome measures could be used and would yield different ICERs.

For example, we could have easily chosen our employment measures as an outcome, defining a new variable (total earnings) from the sum of EARNINGS1 and EARNINGS2. Had we done that, we would have found that, compared with ICM, PACT-EH is associated with a $100 less in total earnings. In this case,

                                     ΔC/ΔE = $5000/–$100 = –$50

Notably, this ICER of –$50 means $50 spent for each additional dollar not earned, rather than $50 saved for each additional dollar earned. This difference in interpretation is important. Thus the separate parts of the ICER (that is, ΔC and ΔE) should always be reported. Likewise, although both scenarios result in a positive ICER, it is reasonable to expect that decision makers will care whether the treatment is more costly (ΔC > 0) and more effective (ΔE > 0) rather than less costly (ΔC < 0) and less effective (ΔE < 0). This can only be determined through a separate inspection of ΔC and ΔE. In addition, compared with the ICER calculated with HOUSING1 as an outcome, the ICER calculated with total earnings, makes PACT-EH appear less attractive. Thus different ICERs generated from different outcome measures can yield different results. When reading economic evaluations, it is important to remember that a study’s final conclusions may rest on which measure of effectiveness was chosen (for example, 23).

In general, the CEA approach offers a different perspective from only considering costs. With CMA, we know that PACT-EH is more expensive than ICM. With CEA, we also know about the trade off (that is, what additional effect is associated with the additional costs). However, we are still left with the question in step 4—is the trade off worth it? One way to incorporate the value judgments necessary to address step 4 is to use methods from CBA.

Cost–Benefit Analysis

There are 2 main characteristics that distinguish a CBA from a CEA. The first involves the number of patient outcomes considered as the new treatment’s effect. A CEA uses only one outcome as a proxy for a treatment’s effect, whereas a CBA typically considers a broad range of patient outcomes (such as life years saved, suffering averted, and so on) that together comprise the benefits produced by the intervention.

Table 2  Hypothetical results for the economic evaluation of a program of assertive community treatment, employment, and housing (PACT-EH) compared with an intensive case management (ICM) program 

Variables 

Symbol 

PACT-EH 

ICM 

Extra cost
(DC) 

Extra effect
(DE) 


Average cost 

$55 000 

$50 000 

$5000 

— 

Average effect 

 

       

Days of stable housing 

E1 

310 

300 

— 

10 

Total wagesa  

E2 

$2900 

$3000 

— 

–$100 

QALYs 

E3 

0.58 

0.48 

— 

0.10 

Incremental analysis 

 

       

ICER1  (using days of stable housing) 

DC/DE1 

$5000/10 = $500 

— 

— 

ICER2  (using total wages) 

DC/DE2 

$5000/–100 = –$50 

— 

— 

ICER3  (using QALYs) 

DC/DE3 

$5000/0.10 = $50 000 

— 

— 


aThe variables EARNINGS1 and EARNINGS2 (not shown)  record wages from noncompetitive employment and from competitive employment, respectively. The variable total wages is the sum of EARNINGS1 and EARNINGS2

The second defining characteristic of a CBA is that outcomes are valued in monetary units (for example, dollars). Conversely, for a CEA, the outcome is measured in natural units (such as days). For example, to conduct a CBA of PACT-EH, the monetary value of

                               HEALTH1 + HEALTH2 + HOUSING1 + EARNINGS1 + EARNINGS2

must be estimated. Suppose HEALTH1 measures the subject’s quality of life. A dollar value must be attached to each gain in quality of life. If quality of life were measured on a 7-point Likert scale, how much would society be willing to pay for a 1-increment improvement? Suppose HOUSING1 indicates the number of days of stable housing. What is the monetary value of each additional night a subject is not sleeping on the street? Decisions like these must be made for all treatment consequences. In this way, benefits are valued in dollars.

By subtracting the extra cost from the monetary value of the extra effects, CBA yields an estimate of the incremental net benefit (INB) of PACT-EH. The INB shows whether the extra cost of PACT-EH (ΔC) outweighs the value of the extra effect (ΔE). A negative INB means that the extra effect’s value does not justify the extra cost (that is, the additional benefits of PACT-EH are worth less than the additional costs, and the intervention is not economically attractive). This can be shown mathematically (for example, INB = ΔE ! ΔC, so if INB < 0, then ΔE !ΔC < 0 or ΔE < ΔC; that is, the extra costs are greater than the extra benefits).

Clearly, a challenge with “real world” CBA is getting the dollar value for the extra effect. While society might be able to attach a value to a new bridge, road, or airport, it may be more difficult to value a day without psychotic symptoms. Thus, not surprisingly, few CBAs have been done of mental health interventions (some exceptions include a seminal economic evaluation of PACT [24] and a CBA of nurse therapy for neuroses in primary care [25]). CUA could be considered a compromise between CBA (where many outcomes are considered) and CEA (where only one outcome is considered).

Cost–Utility Analysis

When an economic evaluation focuses on a single outcome measured in convenient, natural units (such as length of life measured in years), it is a CEA. However, when the economic evaluation focuses on quality adjusted life years (QALYs), the economic evaluation is frequently described as a CUA. The World Health Organization (WHO) uses disability adjusted life years (DALYs) to measure global burden of disease (26–29). Given the controversy surrounding DALYs (30–33) is beyond the scope of this paper and given the fact that most mental health economic evaluations analyses use QALYs rather than DALYs, we refer readers interested in calculating and presenting DALYs elsewhere (34–37).

The theoretical construction of a QALY is straightforward; the QALY is simply the product of quality of life and length of life. For example, imagine there are 2 people who are expected to live another 20 years. The first person is in perfect health (quality of life = 1.0), so she will experience 20 × 1.0 = 20 QALYs. The other person has a severe chronic illness, so he rates his quality of life as 0.5 and will experience only 20 × 0.5 = 10 QALYs over the next 20 years of life. Defining HEALTH1 as “quality of life” and HEALTH2 as “length of life,” each study participant’s QALY can be calculated as:

                                                         QALY = HEALTH1 × HEALTH2

Similar to CEA, the results of an economic evaluation using QALYs are reported with an ICER. For example, if PACT-EH costs more (ΔC = $5000) and provides more QALYs (ΔE = 0.1) (Table 2), then the ICER estimates the extra cost per extra QALY. In this case, the estimate suggests that PACT-EH costs $50 000 per additional QALY when compared with ICM. However, again we are left with the important but unanswered question, “Is the trade off worth it?”

Further, while there are various methods to calculate QALYs, there is no universally accepted standard. In fact, there is evidence that there are important differences in QALY estimates, depending on the methods used (37–39) and on whom the methods are used (for example, the patient, family, clinician, or general public) (40). The advantage of a CUA is that the cost per QALY for a new mental health treatment or intervention can be compared with other health care interventions’ cost per QALY “price tags.” Some people feel that a mental health treatment with an ICER estimate of $20 000 to $40 000 per QALY “compares favorably to those for a wide range of preventive and therapeutic services”(22, p 1643). However, to enjoy this advantage, there is a risk of using an erroneous measure to elicit incorrect values from the wrong people.

What to do With Results of the Economic Evaluation?

After reporting the results of an economic evaluation, investigators tend to speculate whether the trade off of ΔC for ΔE is worth it (that is, whether the treatment or intervention is cost-effective). In our example, as in most economic evaluations, there is no clear-cut answer. In fact, sometimes usual care provides better outcomes at higher costs, compared with new treatment (for example, 41). If we had a sense of the range of society’s willingness to pay for an additional QALY or day of stable housing, then we could compare this with our ICER “price tag.” This could be done using a regression framework that allows for a variety of willingness to pay values (for example, $20 000 per QALY, $50 000 per QALY, and $100 000 per QALY) (42). Often this speculative process is aided by appealing to previous funding decisions. For example, if the Ministry of Health (MOH) typically funds treatments/interventions/health technologies that are around $50 000 per additional QALY, perhaps they should fund PACT-EH as well. While this type of reasoning may help to provide context, some have argued that decisions made in this way will lead to uncontrolled growth in expenditures (43–45), in the same way that most people do not have enough money to buy from every store every item that is “on sale” even though all items are “sale-priced.”

Discussion and Conclusion

Borrowing from Dickens, we might describe this new century as the best of times and the worst. With respect to health care, we cannot afford to do all that we can do. Whether this is caused by lack of money, time, or labour, the fact remains that scarcity forces choice. The results of an economic evaluation can assist decision makers by providing estimates of how much we must give up (that is, the extra cost) for what we get (that is, the extra gain in patient outcome).

Strengths of an Economic Evaluation

One of the most useful aspects of an economic evaluation is the potential to evaluate costs and consequences systematically. It requires the researcher to clearly enumerate the costs and consequences being considered, as well as which methods will be used to analyze the data. Will costs be calculated using the human capital method or the friction cost method (46,47)? Will patient outcomes be determined using QALYs or DALYs? At the same time, this means that both cost and consequence data must be gathered (or estimated). This frequently requires program budgets to be released to the investigators. If it is a service program, workload measures will also be collected; this can create an additional burden on frontline staff and create resistance to the project (48). These barriers can be overcome, but they demand a greater investment to develop mutual trust on the part of both the research team and the service providers than might otherwise be needed. Another powerful argument in favour of doing an economic evaluation is that it can summarize costs and consequences simultaneously in one number (for example, in an ICER). However, it is important to remember that cheaper is not always better. Researchers should heed the caution that “there is little point in spending money on something that is cheap if it provides no benefits” (49, p 740).

Limitations of an Economic Evaluation

The primary limitation of summarizing cost and consequences in an ICER “price tag” is that decision makers, assuming they find the economic evaluation useful (50), must still decide whether the extra gain associated with the new treatment is worth the extra cost. Another key limitation of economic evaluation is that it does not explicitly consider the decision maker’s budget. As a result, a decision maker might deem new treatment cost-effective but too expensive to approve. A compromise to avoid this state of affairs happens in the pharmaceutical area, when products become listed as “limited use” on a formulary. To simplify exposition, the importance of statistics in economic evaluation has been omitted. Readers should be aware that economic estimates vary, and their uncertainty should be described using confidence intervals or with some other method (42).

Advocating for Better Care

To successfully advocate on behalf of patients that extra money spent for extra benefit may be worthwhile, measures of costs and benefits must be considered simultaneously. For example, mental health economic evaluations frequently include the costs of hospitalization that occur during the treatment period. However, to claim that hospitalization is bad simply because it adds cost ignores the potential benefits of hospitalization. Since economic evaluations consider both costs and consequences, it is impossible to determine a priori whether new treatment with less hospitalization will be more economically attractive. Some hospitalizations are likely to be beneficial (such as necessary hospitalization), and some are not (such as unnecessary hospitalization). In either case, a “good” or a “bad” hospitalization uses scarce health care resources; ignoring the consequences of the hospitalization ignores the reason for and the impact of the “good” and “bad” characterization (51). From the earlier discussion of CMA, it is clear that it is only proper to ignore outcomes when there are no differences between treatment options. Thus while additional psychiatric hospitalization may cost more (ΔC > 0), the additional gain in patient outcomes (ΔE) may be worthwhile. Clearly, this determination can only be made after ΔE is calculated and valued.

Using economic evaluation to analyze costs and consequences systematically and simultaneously, researchers can generate estimates (for example, $50 000 per extra QALY) upon which decision makers can render judgments (such as, “that’s not economically attractive”). The choice of economic evaluation (for example, CEA, CBA, or CVA) is inextricably linked to the choice of consequence(s) used to represent the effect(s) of treatment. In fact, a recent review of economic evaluations of mental health care interventions found “few good full economic evaluations . . . in mental health care” (2, p 161) and speculated that this might be related to complications involving measurement of effectiveness. At its core, economic evaluation is a process designed to produce an estimate rather than a decision, and it is likely that health care decisions will always be based on other factors in addition (perhaps) to economic efficiency. This brings us to that old saying, “There’s no such thing as a free lunch.” Economic evaluation helps us understand what we must trade off to have our lunch. Without examining trade-offs now, we may, in the future, lose our lunch.

Acknowledgements

This paper has benefited from valuable comments by David Streiner and an anonymous reviewer. Dr Hoch and Dr Dewa are grateful for the support provided by the Ontario Ministry of Health and Long-term Care Career Scientist Awards.


References

1. Drummond MF, O’Brien BJ, Torrance GW, Stoddart GL. Methods for the economic evaluation of health care programmes. 2nd ed. Oxford: Oxford University Press; 1997.

2. Evers SM, Van Wijk AS, Ament AJ. Economic evaluation of mental health care interventions. A review. Health Econ 1997;6:161–77.

3. Udvarhelyi IS, Colditz GA, Rai A, Epstein AM. Cost-effectiveness and cost–benefit analyses in the medical literature. Are the methods being used correctly? Ann Intern Med 1992;116:238–44.

4. Zarnke KB, Levine MA, O’Brien BJ. Cost–benefit analyses in the health-care literature: don’t judge a study by its label. J Clin Epidemiol 1997;50:813–22.

5. Weinstein MC, Stason WB Foundations of cost-effectiveness analysis for health and medical practices. N Engl J Med 1977;296:716–21.

6. Doubilet P, Weinstein MC, McNeil BJ. Use and misuse of the term cost effective in medicine. N Engl J Med 1986;314:253–6.

7. Eisenberg JM. Clinical economics: a guide to the economic analysis of clinical practices. JAMA 1989;262:2879–86.

8. Detsky AS, Naglie IG. A clinician’s guide to cost-effectiveness analysis. Ann Intern Med 1990;113(2):147–54.

9. Robinson R. Economic evaluation and health care: cost-effectiveness analysis. BMJ 1993a:307:793–5.

10. Robinson R. Economic evaluation and health care: cost–utility analysis. BMJ 1993b:307:859–62.

11. Robinson R. Economic evaluation and health care: cost–benefit analysis. BMJ 1993:307:924–6.

12. Smith AF, Brown GC. Understanding cost effectiveness: a detailed review. Br J Ophthalmol 2000;84:794–8.

13. Cunningham SJ. Economic evaluation of healthcare—is it important to us? Br Dent J 2000;188:250–4.

14. Cunningham SJ. An introduction to economic evaluation of health care. J Orthod 2001;28:246–50.

15. Goodacre S, McCabe C. An introduction to economic evaluation. Emerg Med J 2002;19:198–201.

16. Hollenbeak CS. Dillon PW. An introduction to economic evaluation for pediatric surgeons. Semin Pediatr Surg 2002;11(1):55–9.

17. Drummond MF, Stoddart GL, Torrance GW. Methods for the economic evaluation of health care programmes. 1st edition. Oxford University Press: Oxford; 1987.

18. Gold MR, Siegel JA, Russell LB, Weinstein MC, editors. Cost-effectiveness in health and medicine. New York: Oxford University Press; 1996.

19. Hargreaves W, Shumway M, Hu T-W, Cuffel B. Cost-outcome methods for mental health. San Diego: Academic Press; 1998.

20. Muennig P, Khan K. Designing and conducting cost-effectiveness analyses in medicine and health care. San Francisco: Jossey-Bass; 2002.

21. Petitti DB. Meta-analysis, decision analysis, and cost-effectiveness analysis: methods for quantitative synthesis in medicine. 2nd ed. New York: Oxford University Press; 2000.

22. Simon GE, Katon WJ, Von Korff M, Unutzer J, Lin EH, Walker EA, and others. Cost-effectiveness of a collaborative care program for primary care patients with persistent depression. Am J Psych 2001;158:1638–44.

23. Dixon L, Hoch J, Clark R, Bebout R, Drake R, McHugo G, and others. Cost-effectiveness of two vocational rehabilitation programs for persons with severe mental illness. Psychiatr Serv 2002;53:1118–24.

24. Weisbrod BA, Test MA, Stein LI. Alternatives to mental hospital treatment: economic cost–benefit analysis. Arch Gen Psych 1980;37:400–5.

25. Ginsberg G, Marks I, Waters H. Cost–benefit analysis of a controlled trial of nurse therapy for neuroses in primary care. Psychol Med 1984;14:683–90.

26. Ustun TB, Ayuso-Mateos JL, Chatterji S, Mathers C, Murray CJ. Global burden of depressive disorders in the year 2000. Br J Psychiatry 2004;184:386–92.

27. Murray CJ. Quantifying the burden of disease: the technical basis for disability-adjusted life years. Bull World Health Organ 1994;72:429–45.

28. Murray CJ, Acharya AK. Understanding DALYs (disbility-adjusted life years). J Health Econ 1997;16:703–30.

29. Murray CJ, Lopez AD. The utility of DALYs for public health policy and research: a reply. Bull World Health Organ 1997;75:377–81.

30. Anand S, Hanson K. Disability-adjusted life years: a critical review. J Health Econ 1997;16:685–702.

31. Sayers BM, Fliedner TM. The critique of DALYs: a counter-reply. Bull World Health Organ 1997;75:383–4.

32. Arnesen T, Nord E. The value of DALY life: problems with ethics and validity of disability adjusted life years. BMJ 1999;319:1423–5. Erratum in: BMJ 2000;320:1398.

33. Banerji D. Report of the WHO Commission on Macroeconomics and Health: a critique. Int J Health Serv 2002;32:733–54.

34. Homedes N. The Disability-Adjusted Life Year (DALY). Definition, measurement and potential use. World Bank. [Human Capital Development Working Paper 68]. Available: http://www.worldbank.org/html/extdr/hnp/ hddflash/workp/wp_00068.html. Accessed 2004 June 8.

35. Fox-Rushby JA, Hanson K. Calculating and presenting disability adjusted life years (DALYs) in cost-effectiveness analysis. Health Policy Plan 2001;16:326–31.

36. Bastian H. A consumer trip into the world of the DALY calculations: an Alice-in-Wonderland experience. Reprod Health Matters 2000;8:113–6.

37. Gold MR, Stevenson D, Fryback DG. HALYS and QALYS and DALYS, oh my: similarities and differences in summary measures of population Health. Annu Rev Public Health 2002;23:115–34.

38. Krabbe PFM, Essink-Bot ML, Bonsel GJ. The comparability and reliability of five health-state valuation methods. Soc Sci Medicine 1997;45:1641–52.

39. Arnesen T, Trommald M. Roughly right or precisely wrong? Systematic review of quality-of-life weights elicited with the time trade-off method. J Health Serv Res Policy 2004;9(1):43–50.

40.Shumway M. Preference weights for cost–outcome analyses of schizophrenia treatments: comparison of four stakeholder groups. Schizophr Bull 2003;29:257–66.

41. Fenton W, Hoch J, Herrell J, Mosher L, Dixon L. Cost and cost-effectiveness of hospital versus crisis residential care for patients with serious mental illness. Arch Gen Psychiatry 2002;594:357–64.

42. Hoch J, Briggs A, Willan A. Something old, something new, something borrowed, something BLUE: a framework for the marriage of health econometrics and cost-effectiveness analysis. Health Econ 2002;11:415–30.

43. Gafni A, Birch S. Guidelines for the adoption of new technologies: a prescription for uncontrolled growth in expenditures and how to avoid the problem. CMAJ 1993;148:913–7.

44. Donaldson C, Currie G, Mitton C. Cost effectiveness analysis in health care: contraindications. BMJ 2002;325:891–4.

45. Gafni A, Birch S. Inclusion of drugs in provincial drug benefit programs: should reasonable decisions lead to uncontrolled growth in expenditures? CMAJ 2003;168:849–51.

46. Brouwer WB, Koopmanschap MA. On the economic foundations of CEA. Ladies and gentlemen, take your positions! J Health Econ 2000;194:439–59.

47. Adam T, Koopmanschap MA. Cost-effectiveness analysis: can we reduce variability in costing methods? Int J Technol Assess Health Care 2003;19:407–20.

48. Dewa CS, Horgan S, Russell M, Keats J. What? Another rorm? The process of measuring and comparing service utilization in a community mental health program model. Evaluation and Program Planning 2001;24:239–47.

49. Sturm R. “Economic grand rounds: the myth of medical cost offset” Psych Services 2001;52:738–40.

50. Drummond M. Economic evaluation in health care: is it really useful or are we just kidding ourselves? Aust Econ Rev 2004;37:3–11.

51. O’Reilly RL. The seven deadly sins of mental health reform. Canadian Psychiatric Association Bulletin 2001;33:17–20.

Authors

Manuscript recieved March 2004, revised, and accepted January 2005.

1. Associate Professor, Department of Health Policy Management and Evaluation, University of Toronto, Toronto, Ontario; Adjunct Associate Professor, Department of Epidemiology and Biostatistics, Univeristy of Western Ontario, London, Ontario.

2. Associate Professor, Department of Psychiatry, Centre for Addiction and Mental Health, Health Systems Research and Consulting Unit, University of Toronto,Toronto, Ontario.

Address for correspondence: Dr J Hoch, Centre for Research on Inner City Health, University of Toronto, 30 Bond Street, Toronto, ON M5B 1W8

e-mail: hochj@smh.toronto.on.ca

1 | 2


CJP Archives in English | Archives RCP en français
Supplements and Position Paper Inserts |
Lignes directrices cliniques, énoncés de principe et communiqués
Author Index to 2001 | Index RCP des auteurs 2001
Author Index to 2002 | Index RCP des auteurs 2002
Author Index to 2003 | Index RCP des auteurs 2003
Subject Index to 2001 | Index RCP des sujets 2001
Subject Index to 2002 | Index RCP des sujets 2002
Subject Index to 2003 | Index RCP des sujets 2003
Information for Contributors | Information à l'intention des auteurs
Style Notes for Contributors
Subscription Rates | Prix d'abonnements
Advertising Rates | Tarifs publicitaires
CPA Home | Page d'accueil