Performance and views of examiners in the Applied Knowledge Test for the nMRCGP licensing examination

Siriwardena, Aloysius Niroshan and Dixon, Hilton and Blow, Carol and Irish, Bill and Milne, Paul (2009) Performance and views of examiners in the Applied Knowledge Test for the nMRCGP licensing examination. British Journal of General Practice, 59 (559). pp. 110-113. ISSN 0960-1643

Documents
Performance and views of examiners in the Applied Knowledge Test for the nMRCGP licensing examination
Study involved in validating the AKT of the nMRCGP licensing examination for UK general practitioners
[img]
[Download]
Request a copy
[img] PDF
Siriwardena_Examiner_AKT_evaluation_BJGP_2009.pdf
Restricted to Repository staff only until 2099.

134kB

Official URL: http://dx.doi.org/10.3399/bjgp09X395111

Abstract

BACKGROUND: A new computer-based Applied Knowledge Test (AKT) has been developed for the licensing examination for general practice administered by the Royal College of General Practitioners. AIM: The aim of this evaluation was to assess the acceptability, feasibility, and validity of the test as well as its transfer to a computerised format at local test centres. DESIGN OF STUDY: Computer-based test and postal questionnaire. PARTICIPANTS AND SETTING: Panel of examiners, Membership of the Royal College of General Practitioners (MRCGP) examination, UK. METHOD: Self-administered postal questionnaires were sent to examiners not involved with the development of the test after completing it. Their performance scores were compared with those of candidates. RESULTS: The majority of participants (80.9%) were satisfied with the new computer-based test. Responses relating to content and attitudes to the test were also positive overall, but some problems with content were highlighted. Fewer examiners (61.9%) were positive about the physical comfort of the test centre, including seating, heating, and lighting. Examiners had significantly higher scores (mean 83.3%, range 69 to 93%, 95% confidence interval [CI] = 81.9 to 84.7%) than 'real' candidates (mean 75.0%, range 45 to 94%, 95% CI = 74.6 to 75.5%), who subsequently took an identical test. CONCLUSION: The new computer-based licensing test (the AKT) was found to be acceptable to the majority of examiners. The pass-fail standard, determined by routine methods including an Angoff procedure, was supported by the higher success rate of examiners compared with candidates. The use of selected groups to assess high-stakes (licensing) examinations can be useful for assessing test validity

Item Type:Article
Additional Information:BACKGROUND: A new computer-based Applied Knowledge Test (AKT) has been developed for the licensing examination for general practice administered by the Royal College of General Practitioners. AIM: The aim of this evaluation was to assess the acceptability, feasibility, and validity of the test as well as its transfer to a computerised format at local test centres. DESIGN OF STUDY: Computer-based test and postal questionnaire. PARTICIPANTS AND SETTING: Panel of examiners, Membership of the Royal College of General Practitioners (MRCGP) examination, UK. METHOD: Self-administered postal questionnaires were sent to examiners not involved with the development of the test after completing it. Their performance scores were compared with those of candidates. RESULTS: The majority of participants (80.9%) were satisfied with the new computer-based test. Responses relating to content and attitudes to the test were also positive overall, but some problems with content were highlighted. Fewer examiners (61.9%) were positive about the physical comfort of the test centre, including seating, heating, and lighting. Examiners had significantly higher scores (mean 83.3%, range 69 to 93%, 95% confidence interval [CI] = 81.9 to 84.7%) than 'real' candidates (mean 75.0%, range 45 to 94%, 95% CI = 74.6 to 75.5%), who subsequently took an identical test. CONCLUSION: The new computer-based licensing test (the AKT) was found to be acceptable to the majority of examiners. The pass-fail standard, determined by routine methods including an Angoff procedure, was supported by the higher success rate of examiners compared with candidates. The use of selected groups to assess high-stakes (licensing) examinations can be useful for assessing test validity
Keywords:assessment, general practice, validity, reliability and validity, attitudes, questionnaire survey, performance, quantitative performance evaluation
Subjects:A Medicine and Dentistry > A300 Clinical Medicine
X Education > X300 Academic studies in Education
Divisions:College of Social Science > School of Health & Social Care
ID Code:2054
Deposited By: Niro Siriwardena
Deposited On:11 Nov 2009 11:30
Last Modified:13 Mar 2013 08:33

Repository Staff Only: item control page