 |
 |
|
|
|
Assessing the risk for violence among psychiatric patients is an important and complex clinical task. Consideration of a patient’s potential risk for harm to others or for violence is mandated in Canada (1) and the US (2) by mental health statutes in the context of involuntary civil commitment. Although the ability to attain reliable and valid risk assessments has been seriously doubted in the past (3,4), research beginning in the 1990s has provided reason to be more optimistic (5–10). Researchers have identified important risk factors (6,11,12), and they have developed integrative risk-assessment measures with predictive validities that are much greater than chance (7,9,13). Given that risk-assessment practice and research continues unabated, it is important to scrutinize the characteristics of research strategies that are used to identify risk factors and violence and to validate risk-assessment instruments. One of the key characteristics of risk-assessment research with potentially large influence on empirical findings is the method used to detect patient violence in instrument and risk factor validation research. The method of detecting and measuring violence ostensibly will affect the base rate and type of violence identified in a sample. This in turn can affect the predictive power of instruments, the strength of statistical association between risk factors and violence, which variables do and do not enter multivariate analyses, and so forth. Studies of patient violence tend to use one of several violence detection and measurement procedures. Some have relied solely on single archival sources, such as criminal records (14,15). Other studies have been able to rely on multiple archival sources (16). Still others have included data from archival sources as well as patient self-reports and reports of collaterals (7,17,18). Generally, the use of multiple measures of a variable of interest is recommended in any research context. Mulvey and others provided a cogent analysis of why this is the case in the measurement of patient violence specifically (19). They delineated the potential biases of 5 measurement sources for violent behaviour of psychiatric patients. Generally, the use of multiple measures can offset the biases inherent in any one procedure. There are informative data on the effect of using archival sources vs self-report and collateral report. In a prospective study of 714 psychiatric patients, Mulvey and colleagues reported that using file information alone to detect postrelease community violence gave a 12% 6-month base rate of violence, compared with 47% when patient and collateral interviews also were used (19). Similarly, Steadman and others, in the large-scale MacArthur study of mental illness and violence among over 1100 civil psychiatric patients, reported a 1-year base rate of 4.5% using agency records alone, compared with 27.5% when patient and collateral interviews also were included (18). Most research on the violence of psychiatric patients relies on archival sources because of the resource-intensive nature of including patient and collateral reports as part of the violence measurement procedure. The potential shortcomings of using archival sources alone vs more comprehensive approaches that also include collateral or self-report procedures are fairly well understood because of the research by Mulvey and others (19) and Steadman and others (18). However, much less is known about the effect of using different sources of archival data alone (for example, criminal records, general hospital admissions data, and psychiatric hospital admissions data). Given that the use of archival means of detecting and measuring patient violence is the most prevalent methodology used in research on this topic, it is important to understand how different types of archival data sources might affect important outcomes, such as the base rate of violence detected, the nature of violence detected, and the accuracy of risk assessment. The current study reports the base rate of different types of violence and the accuracy of risk assessments among a sample of 193 Canadian civil psychiatric patients using 3 different measurement sources of archival data: criminal records, tertiary psychiatric hospital records, and general hospital psychiatric unit records. This is essentially a study of concordance of information sources and the impact of these different sources on base rates and predictive accuracy. MethodParticipants The patient population has been described in detail elsewhere (16); key characteristics are summarized here. Most participants were men (n = 117; 61%), white (n = 152; 79%), single (n = 132; 68%), childless (n = 137; 71%), unemployed at admission (n = 180; 93%), and had less than high school education (n = 107; 55%). The most common Axis I discharge diagnosis was schizophrenia (n = 85; 44%), followed by affective disorders (n = 31; 16%) and schizoaffective disorder (n = 27; 14%). Most patients did not have an Axis II diagnosis (n = 111; 58%). Most participants had a history of hospitalization for psychiatric reasons (n = 184; 95%), had past substance use problems (n = 145; 75%), and were using substances at the time of admission (n = 98; 51%). In terms of past criminal and violent behaviour, again, most had previously been arrested or convicted of criminal offences (n = 123; 64%). Many had been arrested or convicted of violent criminal offences (n = 78; 40%). Procedures Trained graduate student research assistants collected information on a variety of variables from patient files, most of which is not relevant to the present study. Research assistants who collected file information were blind to whether patients had violent outcomes; that is, the coders did not have access to information from any subsequent readmissions to the psychiatric hospital (nor did they include any outcome information from subsequent admissions to general hospitals or any criminal involvement). Research assistants who collected outcome data were blind to hospital and predictor information. The community follow-up period averaged 626.48 (SD = 220.19) days. Measures Definition of Violence. Violence was defined broadly in a manner consistent with past research on psychiatric patients (13,16,20) to include any actual, attempted, or threatened harm to others. A demarcation between physical violence and nonphysical violence also was made on the basis of whether the incident involved physical contact (for example, use of the hands, other body parts, or a weapon). As such, 3 categories of violence were used: physical violence, nonphysical violence, and “any” violence (a combination of the former 2). It is likely that the “any” category, which is just an overall index of violence, includes discrepant acts such as serious physical violence as well as less serious violence such as threats. This is an inherent feature of an inclusive category that is based on a broad definition. This also is why we did not solely use such a category but included the more specific categories of physical and nonphysical violence to distinguish between types of violence of differing severities. Risk-Assessment Predictor Measures. Two instruments that have been used with civil psychiatric patients in previous research to predict violence were used to evaluate the impact of sources of outcome data on predictive accuracy of decisions made about patients (7,16,18). The HCR-20 violence risk assessment scheme (13) was developed in Canada as a broadband instrument with applicability to civil psychiatric, forensic psychiatric, and correctional samples. It contains 20 risk factors that are rated by clinicians and spans 3 subscales, which give the instrument its abbreviation: Historical (focusing on relatively fixed, static aspects of the individual’s past), Clinical (focusing on recent mental, attitudinal, and behavioral features), and Risk Management (focusing on future adjustment and situational factors). Reviews indicate that it has good interrater reliability and predictive validity in these samples (6,10,21,22). It has been used in the present sample to predict violence, with positive results and good interrater reliability (16). The previous research was a detailed investigation of the predictive validity of the HCR-20, using the combined rather than separate data sources for outcome. This differs from the present investigation, which focuses on how this predictive accuracy might change as a function of using separate outcome sources. The Hare Psychopathy Checklist: Screening Version (PCL:SV; 23) is a 12-item measure of psychopathic personality, a construct related to the DSM-IV’s antisocial personality disorder. Beyond the behavioural characteristics identified in the DSM-IV, it includes a greater emphasis on the personality features of the construct (that is, grandiosity, callousness, lack of empathy, lack of guilt, and lack of remorse). The PCL:SV has been found to predict violence in various settings, including among civil psychiatric patients (16,24). It was used successfully in the present sample to predict violence and has acceptable interrater reliability (16). ResultsEffect of Measurement Source on Observed Base Rate of Violence
We were interested in the degree of overlap between sources of violence data. That is, what common cases would the 3 procedures identify? Results suggest that, while there was some overlap among sources, it was far from complete. Table 2 presents 3 cross-tabulations: criminal records × general hospital records, criminal records × psychiatric hospital records, and general hospital records × psychiatric hospital records. These data show that, by relying on any one type of archival source only, we would have missed important violence data and would have grossly underestimated the base rate of violence. For instance, only 9/19 patients who were arrested for violent offences also were violent in the context of a rehospitalization to a general hospital. Similarly, only 3/19 patients who were arrested for violent offences were returned to the psychiatric hospital with violence as part of the problem. Conversely, of the 23 people who were detected to be violent as part of return to the psychiatric hospital, only 3 also were violent according to criminal records. Even general vs psychiatric hospital records resulted in very different classifications of patients as violent or nonviolent. Of the 52 people detected as violent as part of admissions to the psychiatric units of general hospitals, only 12 also were violent in the context of admission to the psychiatric hospital.
We also computed correlations between the numbers of violent incidents detected under each violence-detection source. Results are presented in Table 3 for physical (above the diagonal) and nonphysical (below the diagonal) violence. As is evident, correlations are near zero and nonsignificant for physical violence, despite reasonable statistical power. For nonphysical violence, 2 of the 3 correlations are significant although small in magnitude. These correlation coefficients suggest that each of the sources of violence detection functions quite differently from the others in terms of the number of violent incidents detected.
Effect of Measurement Source for Violence Data on Predictor–Outcome Indexes Table 4 shows the areas under the curve (AUC) of the ROC for the HCR-20 and PCL:SV under combined and separate data sources. The AUC is an index of predictive accuracy of the predictor. It ranges from 0 (perfect negative prediction) through 0.50 (chance prediction) to 1.0 (perfect positive prediction). It is interpreted as the probability that a randomly chosen, actually violent person will score higher on the predictor than a randomly chosen, actually nonviolent person.
There was substantial variability in the relation between the predictors (HCR-20 and PCL:SV) and violence measured by the different sources. Generally, there is less variability for the HCR-20 than for the PCL:SV. Nonetheless, the AUC of the HCR-20 for any violence varies from 0.72 to 0.80 and is 0.76 using the combined sources of measurement for violence. It varies from 0.68 to 0.75 for physical violence (again, 0.76 using combined sources). The variability is largest for nonphysical violence as a function of outcome source (0.71 to 0.91). Depending on the outcome source used, then, a researcher might conclude that the HCR-20 relates to violence with moderate-to-large AUC values. For the PCL:SV, the variability as a function of outcome source is larger than for the HCR-20. The AUC values are routinely near chance (0.55 to 0.60) across types of violence when psychiatric records are used in isolation and are routinely large across violence types (0.78 to 0.83) when criminal records are used. General hospital records produced AUC values between these extremes. Using combined data, the AUC values typically are moderate in strength. Thus, the utility of using the PCL:SV alone as a predictor varies from uncorrelated to strongly correlated with violence in this sample, depending upon the outcome source of violence considered. The most reliable result, for both the PCL:SV and the HCR-20 (and any other predictor that would be used), is the one based upon the combined data sources. DiscussionPsychiatric patient violence and attendant risk assessment are important clinical and legal topics. In recent years, there have been more empirical investigations into these topics (7,16–18,20,24). All studies use different methodologies to some degree, and different methodological approaches can have effects on research findings and clinical practice based on such research. One of the dimensions along which research studies vary is outcome-measurement procedure. Although something is known about the underestimation that occurs through the use of agency records alone relative to these sources in addition to self-reports and collateral reports, because of recent American research (18,19), much less is known about the effect of different archival sources of violence. However, in theory, this is one methodological facet of studies that could have a large impact on substantive findings, such as estimated violence base rate and the accuracy of violence prediction. This study investigated the impact on substantive findings of violence base rate and accuracy of risk assessment as a function of methodological variations in measuring violence. The base rate of postrelease community violence of 193 involuntarily committed psychiatric patients, as well as the predictive accuracy of the HCR-20 violence risk assessment scheme (13) and the PCL:SV (23), was calculated under 4 outcome measurement conditions: criminal record data, general hospital psychiatric unit records, psychiatric hospital records, and a combination of the 3 preceding single sources. Only patients who had been released to the community were included in this study. This could be considered a limiting factor in terms of patient characteristics. However, we were primarily concerned with community follow-up and violence after release, and hence this strategy made the most sense. Findings clearly showed that the base rate of violence differed substantially, depending on the outcome data source used. Similarly, the predictive accuracy of the HCR-20 and PCL:SV varied across outcome measurement procedures, especially for the PCL:SV. In fact, accuracy varied so much for the PCL:SV that one could reasonably conclude that psychopathy is not correlated with violence (if relying solely on psychiatric hospital records) or that it is a strong correlate of violence (if relying solely on criminal records) in the sample. Using the more stable estimate of violence, based on a combination of the 3 individual sources, the best estimate of the predictive strength is between these 2 other estimates. Implications for Science The present findings have implications for the state of research as well. That is, it would be worth systematically summarizing, in metaanalytic fashion, research on psychiatric patient violence taking into account outcome measurement methodology. This would permit the evaluation of violence correlates and risk-assessment instruments across studies after accounting for any biasing impact that outcome procedures might have. Implications for Clinical Practice Funding and SupportThis research was supported in part by a grant to the second author by the British Columbia Health Research Foundation. References1. Robertson GB. Mental disability and the law in Canada. 2nd ed. Toronto: Carswell; 1994. 2. Melton GB, Petrila J, Poythress NG, Slobogin C. Psychological evaluations for the courts: a handbook for mental health professionals and lawyers. 2nd ed. New York: Guilford; 1997. 3. Ennis BJ, Litwack TR. Psychiatry and the presumption of expertise: flipping coins in the courtroom. California Law Review 1974;62:693–752. 4. Monahan J. Predicting violent behavior: an assessment of clinical techniques. Beverly Hills (CA): Sage; 1981. 5. Borum R. Improving the clinical practice of violence risk assessment: technology, guidelines, and training. Am Psychol 1996;51:945–56. 6. Douglas KS, Webster CD. Predicting violence in mentally and personality disordered individuals. In: Roesch R, Hart SD, Ogloff JRP, editors. Psychology and law: the state of the discipline. New York: Plenum; 1999. p 175–239. 7. Monahan J, Steadman HJ, Silver E, Appelbaum PS, Robbins PC, Mulvey EP, and others. Rethinking risk assessment: the MacArthur study of mental disorder and violence. New York: Oxford University Press; 2001. 8. Mossman D. Assessing predictions of violence: being accurate about accuracy. J Consult Clin Psychol 1994;62:783–92. 9. Quinsey VL, Harris GT, Rice GT, Cormier CA. Violent offenders: appraising and managing risk. Washington (DC): American Psychological Association; 1998. 10. Otto RK. Assessing and managing violence risk in outpatient settings. J Clin Psychiatry 2000;56:1239–62. 11. Bonta J, Law M, Hanson RK. The prediction of criminal and violent recidivism among mentally disordered offenders: a meta-analysis. Psychol Bull 1998;123:123–42. 12. Monahan J, Steadman HJ, editors. Violence and mental disorder: developments in risk assessment. Chicago (IL): University of Chicago Press; 1994. 13. Webster CD, Douglas KS, Eaves D, Hart SD. HCR-20: assessing risk for violence. Version 2. Burnaby (BC): Mental Health, Law, and Policy Institute, Simon Fraser University; 1997. 14. Steadman HJ, Halfon A. The Baxstrom patients: backgrounds and outcomes. Semin Psychiatry 1971;3:376–86. 15. Lindqvist P, Allebeck P. Schizophrenia and crime: a longitudinal follow-up of 644 schizophrenics in Stockholm. Br J Psychiatry 1990;157:345–50. 16. Douglas KS, Ogloff JRP, Nicholls TL, Grant I. Assessing risk for violence among psychiatric patients: the HCR-20 violence risk assessment scheme and the Psychopathy Checklist: Screening Version. J Consult Clin Psychol 1999;67:917–30. 17. Lidz CW, Mulvey EP, Gardner W. The accuracy of predictions of violence to others. JAMA 1993;269:1007–111. 18. Steadman HJ, Mulvey E, Monahan J, Robbins PC, Appelbaum PS, Grisso T, and others. Violence by people discharged from acute psychiatric inpatient facilities and by others in the same neighborhoods. Arch Gen Psychiatry 1998;55:393–401. 19. Mulvey EP, Shaw E, Lidz CW. Why use multiple sources in research on patient violence in the community? Criminal Behavior and Mental Health 1994;4:253–8. 20. McNiel DE, Binder RL. Clinical assessment of the risk of violence among psychiatric inpatients. Am J Psychiatry 1991;148:1317–21. 21. Mossman D. Evaluating violence risk ‘by the book’: a review of HCR-20: assessing risk for violence. Version 2 and the Manual for the Sexual Violence Risk - 20. Behav Sci Law 2000;18:781–9. 22. Witt PH. A practitioner’s view of risk assessment: the HCR-20 and SVR-20. Behav Sci Law 2000;18:791–8. 23. Hart SD, Cox DN, Hare RD. The Hare Psychopathy Checklist: Screening Version (PCL:SV). Toronto: Multi-Health Systems; 1995. 24. Skeem, JL, Mulvey, EP. Psychopathy and community violence among civil psychiatric patients: results from the MacArthur Violence Risk Assessment Study. J Consult Clin Psychol 2001;69:358–74. AuthorsManuscript received July 2002, revised, and accepted August 2003. 1. Assistant Professor, Department of Mental Health Law & Policy, Louis de la Parte Florida Mental Health Institute, University of South Florida, Tampa, Florida; Guest Professor of Applied Criminology at Mid Sweden University, Sundsvall, Sweden. 2. Foundation Professor of Clinical Forensic Psychology, School of Psychology, Psychiatry, and Psychological Medicine, Monash University, Clayton, Victoria, Australia; Director of Psychological Services, Victorian Institute of Forensic Mental Health (Forensicare), Thomas Embling Hospital, Fairfield, Victoria, Australia. Address for correspondence: Dr KS Douglas, Department of Mental Health Law & Policy, Louis de la Parte Florida Mental Health Institute, University of South Florida, 13301 Bruce B Downs Boulevard, Tampa, FL 33612 e-mail: kdouglas@fmhi.usf.edu
1 | 2
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||