Modifying the conjoint examination to match the new developments in training and assessment

C S Y Chan 陳兆儀

The Conjoint Examination is at its 15th year in 2001. Since its creation, continual refinements have been made to serve the purpose of fairly and accurately assessing who is competent to be passed as a qualified family physician and a Fellow of the College.1

These changes include: the creation of the Physical Examination (PE) segment in 1993 in response to detected deficiencies in physical examination techniques of candidates, the merging of the Case Commentary segment with the Orals segment in 1999 to allow candidates to have two instead of one chance to present and demonstrate his/her care of patients to the examiners. The Modified Essay Questions (MEQ) segment has changed from written format to computerised format and back again to written, and from the unfolding of a single case to a series of unrelated questions.

The Diagnostic Interview (DI) segment initially began with the use of real patients for interview and physical examination, and then switched to the use of role-playing examiner for standardisation. The long DI was set to encourage candidates to take a comprehensive view of patient problems. However, with time, some candidates have turned it into a purposeless fishing expedition of just asking any and every question that can be asked rather than for problem solving. Hence the long DI case was cancelled in 1999 and three short cases were used to distinguish candidates who can appropriately focus on the relevant biopsychosocial issues for the specific problems of the patients.

The language medium of the Management Interview (MI) segment changed from Cantonese to English in 1992 as required by the Royal Australian College of General Practitioners (RACGP), and then one of the cases was changed back to a Cantonese option in 1997 to suit the needs of local doctors and patients.

Thus the examination is a dynamic instrument that responds to the needs, strength and weakness of the candidates, the profession and the community and matches with the new developments in teaching, learning and assessment.

The new redeveloped examination of the RACGP

Since 1999, the RACGP has changed its Fellowship Examination to a more valid format, structure and process.2 Instead of running the examination in a number of segments, it has been changed to a two-segment written examination and a clinical objective structured clinical examination (OSCE).3,4 The OSCE consists of about 14 stations of short and long consultations. Each station may concentrate on some specific tasks but is not strictly divided into DI, MI and PE stations. Instead of a purely criterion-based rating form using numeric scores, they have also employed categorical assessments of a number of descriptive performance domains of consultation tasks, correlated them with the global overall performance, and used standard setting procedures5 to set the cut-off score, instead of fixing the 65% as pass mark. Moreover, candidates are required to pass the whole examination. If they fail, they have to repeat the whole examination, both written and clinical.

The RACGP has found the change beneficial, in terms of improving the validity and efficiency of the examination. The OSCE reflects what family physicians do in real life, seeing a variety of patients with different problems and for various reasons and requiring a variety of approaches to solve their problems. Candidates also accepted the change. The Hong Kong College of Family Physicians, as a partner of the Conjoint Examination, must consider whether we should implement similar changes.

In addition, we have to modify our Examination system to cater for the major changes in our vocational training system: the addition of part-time training, and the recent and projected increases in training posts.

What are the main changes proposed?

Since we have already been conducting the clinical examination in an OSCE format from 1999 onwards, the proposed change in format will not be drastic. We shall just increase the variety and the total number of stations. The major change is how a candidate may pass or fail the examination.

We propose to hold the written examination in May/ June and the clinical OSCE in October/November each year. The separation of the two enables candidates to concentrate on one section at a time. Candidates will be required to pass the entire written examination in one sitting. The OSCE can only be taken after satisfactory completion of the written examination. Again a candidate has to pass the entire clinical examination in one sitting. If one fails the written, the entire written examination has to be repeated. If one fails the OSCE, the whole OSCE has to be repeated.

We propose this change mainly because the whole examination is constructed under a comprehensive matrix of content and performance domains that encompass the required knowledge, attitude and skills of a family doctor. By allowing candidates to retain certain segments, the casemix that they are exposed to, when they repeat the examination, becomes quite limited. They may thus fail a repeat segment because they did not perform satisfactorily in a small number of cases. With the new system, candidates have a better chance of being presented with a comprehensive set of case content and requirements, and do not pass or fail based on only 2-3 cases or only 2-3 examiner teams.

The second reason for not allowing candidates to retain segments is that the part-time training route allows a candidate up to 10 years to finish the training. Even for trainees working in the Hospital Authority, some may have a delay in finding community-based training positions after they finish their hospital-based training. There will be confusion if we allow candidates to retain segment results as trainees may finish their training in varying lengths of time. Allowing retention of results up to 8-10 years when medicine changes so quickly defeats the purpose of the assessment. As a compromise, we suggest allowing retention, for up to three years only, of the successful result of the new comprehensive written examination.

What are the reasons for the change?

The following list summarises the reasons for proposing the changes:

1. RACGP, our counterpart, has changed its examination system.2
2. RACGP is a progressive College which uses current best educational evidence to develop valid and realistic assessment tools.6,7
3. The new examination is more comprehensive.
4. The new examination is more valid and matches the requirements of real life practice and the goals of the College examination.
5. It is fairer for candidates to be passed and failed on a large number of stations rather than by a limited number of cases and examiners.
6. Time and administrative efficiency: timetabling will be simplified if every candidate rotates through the same number of stations. Candidates do not have to wait for hours during the OSCE just to re-sit 1 to 2 segments of the examination.
7. The new rating schedules and standard setting procedures allow a small degree of flexibility, such as expert judgment of the examiners and norm referencing, which is better than the use of an absolute pass mark of 65%.

We would like to hear your views

The Conjoint Examination Committee is still deliberating about the proposed changes. We would like to invite members to submit their views, especially members who will be affected by such changes in examination policy, e.g. those who have not yet taken but are planning to take the examination, and vocational trainees. Please do so within the next two months. Talk to our committee members or send your views in writing. We may hold a forum for discussion at a later date.

However, in order not to delay changes and facilitate a smooth transition to any new examination system, candidates who start sitting for the examination for the first time this year can only retain their scores (65% or above in any segment) for three years (up to 2004), and those who start sitting for the examination in 2002 can only retain it for two years, and so on, until 2004, by which time all those who started sitting for the examination before 2001 with the provision that a pass mark can be retained for four years, would have finished the process. If we do not run a parallel new examination before the change, we shall change to a new system latest by 2005.

Please be assured that we shall not make any hasty changes until we have had thorough discussion with our members, and allow adequate time for planning and preparation to switch to the new system.