 |
 |
|
|
|
Residency training in psychiatry is meant to enable candidates to practise competently as consultant psychiatrists (1). Preparation for the certification exam represents the culmination of residents’ training, where knowledge is consolidated and interview skills are fine-tuned. In this way, exam preparation is intended to further prepare candidates for practice and to ensure that all successful candidates have relatively uniform abilities relating to the core competencies in psychiatry. It is not currently known which experiences best prepare candidates for their certification exam, outside of those suggested by the CanMEDS 2000 guidelines (1,2) and by the prescribed rotations of residency. Each year, as psychiatry residents prepare for their Royal College certification exam in psychiatry, both candidates and their supervisors voice uncertainties regarding how to best prepare for the exam (3,4). Beyond knowing that observed interviews (5–7), study groups (8), and a diverse knowledge base are helpful, candidates and their supervisors are left guessing the best way to prepare for the exam and rely instead on personal opinion, exam “folklore,” and chance learning experiences. In retrospect, candidates have reported that more opportunities for taking mock oral exams, for learning examination-passing skills, and for knowing clearer study guidelines would have been helpful (9,10). However, research on this subject is up to 20 years old and may not be entirely applicable to the current exam format. We speculated that candidates preparing for the 2002 certification exam would be especially anxious, knowing that they would be the first candidates to experience the new exam format, comprising a written exam in May 2002 and an oral exam in June 2002, with the usual long case and also 4 new case vignette stations. We thought it would be timely to survey Canadian university affiliated psychiatry residents as they prepared for the new examination to determine what experiences should be emphasized in preparing for the exam. Since then, the exam has been further modified, so that residents will need to successfully pass 2 long cases in the year prior to the exam, while the oral component consists solely of case vignettes. Prior evaluation of those candidates who have passed and failed their certification exam in psychiatry suggests that the candidates most likely to pass are from English-language medical schools, have graduated in the top two-thirds of their class, are aged under 30 years, had entered psychiatric training soon after graduating from medical school, and had uninterrupted psychiatric training (11). Residents enrolled in the 2-year child and adolescent program also did not fare as well as general psychiatric residents (12). We collected subjective reports from clinicians who helped prepare psychiatry residents for their examinations as well as reports from current or past Royal College examiners. These reports suggest that candidates who failed their examinations had a less secure knowledge base, less frequent exposure to mock oral exams, poorer ability to consolidate pertinent information from the history and mental status exam, problems presenting information or answering oral exam questions, and difficulties managing performance anxiety. In addition, it has been speculated that residency experience in research may indirectly assist exam preparation by increasing residents’ knowledge base and their ability to consolidate information. We hypothesized that a survey of residents’ experiences of exam preparation would help to confirm or disconfirm the opinions of residents, clinicians, and current or past Royal College examiners regarding how best to prepare for the certification exam in psychiatry. Such a survey would also assist in updating the literature base for the new examination format and would provide the basis for appropriate curriculum changes. Before and after the certification exam, we attempted to measure which psychiatric residency experiences best prepared residents to pass their exam. Specifically, we measured whether residents with more frequent and diverse “Royal College style” observed interview experiences (that is, experiences wherein the format mimics that of the exam’s long case scenario, with 2 examiners, a 50-minute interview, 10 minutes preparation, and a 1-hour question and answer session), with primarily adult psychiatry training, more confidence in their knowledge base, with research experience, and with management strategies for exam anxiety were better prepared to pass their certification exam. We also hypothesized that psychiatry residents with the above-noted experiences would be better prepared to pass. MethodsThe University of Calgary’s Research Ethics Board approved the research protocol prior to surveys being mailed. We sent an anonymous survey to all final year (PGY5) Canadian university–affiliated psychiatry residents 6 months before and immediately after their certification exam in psychiatry. The survey was sent via courier to each residency program director’s secretary, who distributed them to applicable residents. The research protocol and ethics approval was provided to any residency program director who requested them. The directors made the final decision regarding whether their residents would participate in the study, and the residents voluntarily decided whether to complete the survey. Because informed consent was not feasible via a mail-out survey, a letter of invitation outlining the purpose of the research accompanied each survey. The letter also provided contact numbers for the primary investigator and for internal awards at the University of Calgary, in case there were any questions or concerns about participants’ rights. Surveys were completed and sealed in provided envelopes and were either returned to the program director’s secretary, who then forwarded them to the authors, or sent directly by the residents. To ensure residents’ privacy, most programs did not allow their residents to be contacted; thus nonresponders could not be followed up. We aimed for a response rate of 50% or more. To help increase the response rate, we attached a movie coupon to all surveys. To guarantee anonymity, no identifying information was included in the surveys, and prior to the investigators’ scrutiny of the survey, the secretary removed the initial letter of invitation, which included each resident’s address (when available). Surveys asked residents to subjectively report on observed interview experiences, on the relative importance of other training experiences (for example, on-call experiences, teaching sessions, and psychotherapy), on their confidence in their knowledge base and perceived preparedness, on research experience, and on management strategies for exam anxiety. ResultsWe sent anonymous random surveys to 68 final-year psychiatry residents who were attending universities across Canada and who accepted inclusion in the research study. There were 44 respondents (64.7%) to the 68 surveys sent out 6 months prior to the Royal College certification exam. Six months later, following the exam, 44 surveys were sent to the initial respondents, and 29 residents replied (65.9%). A total of 68% of survey 1 respondents and 69% of survey 2 respondents were women. Notably, 100% of survey 2 respondents had passed the FRCPC exam. Residents were asked to rank the importance of several exam preparation methods prior to and following the certification exam (Figure 1). Observed interviews were considered to be the most valuable preparation experience. The average number of informal psychiatric observed interviews (that is, with only one examiner or with a format not mimicking the certification exam’s long case) and formal Royal College style observed interviews performed in the 6 months prior to the exam (survey 2) were 16.2 and 7.3, respectively. The average number of informal observed interviews was double that of formal interviews for both surveys. On average, the total number of observed interviews conducted in the 6 months prior to the exam doubled from the previous year. An average of 6.7 (range 0 to 20) interviews were observed by a Royal College examiner in the 6 months prior to the exam. Respondents consistently commented that more formal Royal College style observed interviews, specifically with past or present Royal College examiners, would be most beneficial. Further, these interviews should be performed on a regular basis throughout the 4 final years of psychiatric training. It was frequently noted in the comment section of the surveys that many residents did not have regular observed interviews in year 3 (that is, the subspecialty year) and year 4 (the elective year), which varied among universities. Most residents (61% from survey 1 and 76% from survey 2) responded that 2 examiners should provide feedback and most (88% and 82% from surveys 1 and 2, respectively) said that 3 or more points about each observed interview performed should be discussed. On average, respondents thought that being exposed to 7 or 8 examiners in the year prior to the exam was ideal. While residents reported that observed interviews were most imperative in preparing for the exam, there was no correlation between the number of interviews and the feelings of preparedness.   Reading time was considered to be the second most vital factor in preparing for the Royal College exam. On average, residents spent 16.2 hours weekly (range 2 to 50 hours) reading 6 months prior to the exam (survey 2), which is almost double that in the year prior to survey 1. The importance of reading time was ranked higher on survey 2, after candidates had completed their exams. This was consistent with written comments stating that “more reading time during residency would be beneficial.” Both before and after the exam, in descending order of ranked importance, the following reading material was identified as most useful for preparation: Kaplan and Sadock’s Synopsis of Psychiatry (13) (82% on survey 1, and 93% on survey 2); the DSM-IV-TR (14) (39% on survey 1, and 34% on survey 2); notes from the Ottawa Review Course (25% on survey 1, 41% on survey 2) and the Canadian Psychiatric Association (CPA) and American Psychiatric Association (APA) clinical practice guidelines (25% on survey 1, and 38% on survey 2). Residents reported that preparation and training for the second half of the exam’s oral component was of paramount importance but a weak area in psychiatric training. Written comments included the following:
Managing performance anxiety was ranked as very important both pre- and postexam. However, only 41% of survey respondents replied that they had discussed anxiety management skills during their residency training. In descending rank order, the most frequent stress-reduction techniques used by residents were exercise; caffeine reduction or elimination; rest, relaxation, and (or) leisure; support from family and (or) friends; and medications, such as propranolol. Other stress- reduction methods cited for the day of the exam (and for immediate days preceding the exam) included cognitive-behavioural techniques or positive self-talk; distraction; meditation; massage; sleep aids (for example, zopiclone or valerian root extracts); and finally, familiarizing oneself with the exam location. Respondents emphasized that if any pharmaceutical agents are to be used, a trial of the medication should be tested in a comparable setting well before the exam. Residents’ had an average number of 2.2 hours weekly (range 0 to 10 hours) of formal teaching in the year prior to the exam (survey 1) and 1.4 hours weekly (range 0 to 6 hours) 6 months prior to the exam (survey 2). On survey 1, residents reported an average of 2.4 informal teaching hours weekly (range 0.2 to 10 hours), and on survey 2, they reported 3.4 hours weekly (range 0 to 15 hours). When residents were asked whether they felt any teaching or experience that they did not receive would have been useful to their exam preparation (in the last 6 months), the most common response was “More formal teaching on formulation, presentation skills, and approach to case vignettes.” Of residents, 48% were actively involved in research over the course of their training. Of those, 80% presented and 40% published their research. Respondents, however, did not identify their research as being valuable to their exam preparation. Figure 3 illustrates how residents felt that confidence in their psychiatric knowledge base contributed to a feeling of preparedness for the exam. In survey 1, residents ranked in order of importance the following training attributes as critical for the oral component of the exam: observed interviews, presenting cases and learning better presentation strategies, time management strategies, knowledge of diagnostic criteria, clinical experience, and learning better screening questions and (or) rapport-building techniques. In survey 2, after the certification exam, respondents considered clinical experience to be most important.  DiscussionA major limitation of the study was the small sample size, owing to some residency programs’ choice not to participate in the research. However, representation from residency programs was garnered from British Columbia, Alberta, Saskatchewan, Manitoba, Ontario, Quebec, Nova Scotia, and Newfoundland, which hopefully represents Canadian psychiatry residency training programs in general. In addition, the small overall number of PGY5 psychiatry residents in Canada further restricted the data analysis. Another limitation was that all survey 2 respondents had passed their certification exam; therefore, we were unable to cross-tabulate differences between the experiences of candidates who had successfully completed their exam and those who had not. This prevented the research hypothesis from being specifically tested. Thus the findings were limited to being descriptive in nature only. Despite the limitations, several key observations emerged from the study. Observed interviews, especially with Royal College examiners, were identified as the most essential experience in preparing for the certification exam. However, there was no correlation between the number of interviews and overall feelings of preparedness. This potentially suggests that the frequency of observed interviews may relate to other factors. Availability, candidate anxiety, and the quality or form of feedback may be equally as important. In addition, candidates recommended exposure to 2 examiners at one time, exposure to a total of 7 or 8 examiners overall in their training, and feedback on 3 or more specific interviewing points per interview. The last point is interesting, because we would not expect candidates to be able to integrate this many feedback points. Future research might specifically evaluate how feedback from an observed interview is best delivered (that is, written and [or] spoken), how many points (both positive and negative) are optimally brought forward, and how many of these are integrated by the candidate. General clinical experience, specific case vignettes or problem solving scenarios, training in formulation and presentation skills, and learning anxiety management techniques were also identified as factors that enabled successful exam completion. Hopefully, general clinical experience is obtained via the core residency rotations and by candidates and program directors together identifying areas wherein they require greater experience and seeking these areas out. Case vignettes, formulation and presentation skills, and anxiety management generally do not occur in a formalized fashion but rather on an individualized basis, despite being identified as important (9,10). Because candidates identify these skills as being of great importance, they should be made a formal part of the curriculum, be it in the form of lectures, a review course, or as part of regular mock oral examinations. Not surprisingly, the more confident residents were in their knowledge base, the more prepared they felt for the exam. Though we did not specifically inquire about candidates’ desire for more reading time, their request for such may relate to the reading being delayed until later in the training or to its being focused on other subjects, to the exclusion of core psychiatric knowledge. The request might also be anxiety-driven. Reading up on cases from basic psychiatric texts and the guidelines prepared by the CPA and the APA on an ongoing basis should be emphasized to residents during each rotation. Though residents did not identify their research experience as being helpful to their exam preparation, they may not have identified the applicability of such experience to their future practice, much like their view of on-call experiences. RecommendationsDespite the study limitations, some overall recommendations can be made. To best prepare residents for successful completion of the Royal College certification exam (and presumably for practice), residency programs should incorporate the following: 1. Observed interviews by past and present Royal College examiners throughout all years of residency training on a scheduled basis (for example, at least twice yearly). 2. Formal training for all final-year psychiatry residents in conducting the oral component of the certification exam, in presentation and formulation techniques, and in strategies for responding to questions in the long case and case vignettes. 3. A review of anxiety-management strategies, specifically, with all final-year psychiatry residents in the months preceding their certification exam. Funding and SupportThe research was supported by an unrestricted grant from the University of Calgary, Department of Psychiatry. AcknowledgementsWe thank Herta Fidler and Dr Shawn Currie for reviewing and reworking the survey items prior to their being sent. We also thank all the residents and programs that participated in the study. References1. Societal Needs Working Group. CanMEDS 2000 project. Skills for the new millennium. Ann R Coll Physicians Surg Can 1996;29:206–16. 2. Martin L, Saperson K, Maddigan B. Residency training: challenges and opportunities in preparing trainees for the 21st century. Can J Psychiatry 2003;48:225–31. 3. McLean P, Fleming J. Evaluation of clinical competence in psychiatry by the Royal College Examination. Can Psychiatr Assoc J 1978;23:521–9. 4. Tohen M, Frankenburg FR. Preparation for the certification examination. Can J Psychiatry 1983;28:471–4. 5. Raskin DE. Mini-boards: a means of evaluating psychiatry residents. Am J Psychiatry 1972;128:116–7. 6. McCormick WO. A practice oral examination rating scale inter-observer reliability. Can J Psychiatry 1981;26:236–9. 7. Leichner P, Sisler GC, Harper D. A study of the reliability of the clinical examination in psychiatry. Can J Psychiatry 1984;29:394–7. 8. Persad E, Garfinkel P. Practising psychiatrists’ views of the certification examination in psychiatry. Can J Psychiatry 1979;24:303–7. 9. Leichner PP. The certification examinations: viewpoints of past candidates. Can Psychiatr Assoc J 1978;23:531–40. 10. Leichner PP. The certification examination: viewpoints of fourth-year residents. Can Psychiatr Assoc J 1977;22:149–53. 11. Lowy FH, Dongier M. The Canadian certification examination in psychiatry. II: who passes and who fails. Can J Psychiatry 1979;24:284–92. 12. Garfinkel BD, Golombek H, Garfinkel PE. Training in child and adolescent psychiatry: selection, academic performance and outcome. Can Psychiatr Assoc J 1976;21:151–5. 13. Sadock BJ, Sadock VA, editors. Kaplan & Sadock’s Synopsis of Psychiatry. 9th ed. Philadelphia (PA): Lippincott Williams & Wilkins; 2003. 14. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th ed. Text Revision. Washington (DC): American Psychiatric Association; 2003. Author(s)Manuscript received September 2003, revised, and accepted June 2004. 1. Assistant Professor, University of Calgary, Department of Psychiatry, Calgary, Alberta. 2. Psychiatry Resident, University of Saskatchewan, Saskatoon, Saskatchewan. 3. Assistant Professor and Residency Program Director, University of Calgary, Department of Psychiatry, Calgary, Alberta. Address for correspondence: Dr D Crockford, C203, 1403-29 Street NW, Foothills Hospital, Calgary, AB T2N 2T9
1 | 2
|
||||||