Milne, Paul, Dixon, Hilton, Blow, Carol and Siriwardena, A. Niroshan (2012) Assuring the quality of an applied knowledge assessment for licensing purposes (Membership of the Royal College of General Practitioners, MRCGP) in UK general practice. In: 15th Ottawa Conference on the Assessment of Competence in Medicine and the Healthcare Professions, 9-13 March 2012, Kuala Lumpur Convention Centre, Malaysia.
Full content URL: http://www.ottawaconference.org/
| Documents |
| ||||
![[img]](http://eprints.lincoln.ac.uk/style/images/fileicons/application_pdf.png)
|
PDF
AKTKLfinal.pdf - Presentation Available under License Creative Commons Attribution Non-commercial No Derivatives. 569kB |
| Item Type: | Conference or Workshop contribution (Presentation) |
|---|---|
| Item Status: | Live Archive |
Abstract
Background
The Applied Knowledge Test (AKT) forms part of the tripos of the MRCGP exam. It is a computer-delivered licensing assessment of applied knowledge. This presentation demonstrates how the test quality is reviewed.
Summary of work
From the RCGP curriculum blueprint, the construction of test items is referenced to high quality evidence, peer reviewed and critically appraised before addition to the question bank. A standard process for question selection, proof reading and feedback to candidates supports its validity. New question formats include free text answers which enhance discrimination. Standard setting follows the modified Angoff process.
Summary of results
The test shows a high Cronbach’s alpha (0.88-0.92). Pre-trialling of questions, common to many postgraduate medical examinations, is deemed unnecessary with an alpha-coefficient for new questions in the most recent test > 0.8 (0.91 when adjusted for the test length). Computer delivered testing enables detailed analysis of patterns of timing and omission of items by candidates.
Conclusions
Systematic processes in test construction and computer delivery has enhanced continuous quality assurance of the AKT.
Take-home messages
Test performance relies on meticulous attention to detail in item writing and test construction. Use of un-trialled new questions enables reliable testing of emerging and changing knowledge.
| Additional Information: | Background The Applied Knowledge Test (AKT) forms part of the tripos of the MRCGP exam. It is a computer-delivered licensing assessment of applied knowledge. This presentation demonstrates how the test quality is reviewed. Summary of work From the RCGP curriculum blueprint, the construction of test items is referenced to high quality evidence, peer reviewed and critically appraised before addition to the question bank. A standard process for question selection, proof reading and feedback to candidates supports its validity. New question formats include free text answers which enhance discrimination. Standard setting follows the modified Angoff process. Summary of results The test shows a high Cronbach’s alpha (0.88-0.92). Pre-trialling of questions, common to many postgraduate medical examinations, is deemed unnecessary with an alpha-coefficient for new questions in the most recent test > 0.8 (0.91 when adjusted for the test length). Computer delivered testing enables detailed analysis of patterns of timing and omission of items by candidates. Conclusions Systematic processes in test construction and computer delivery has enhanced continuous quality assurance of the AKT. Take-home messages Test performance relies on meticulous attention to detail in item writing and test construction. Use of un-trialled new questions enables reliable testing of emerging and changing knowledge. |
|---|---|
| Keywords: | primary care, general practice, assessment, knowledge, examination, medical licensing |
| Subjects: | A Medicine and Dentistry > A300 Clinical Medicine |
| Divisions: | College of Social Science > School of Health & Social Care |
| ID Code: | 5053 |
| Deposited On: | 10 Apr 2012 17:25 |
Repository Staff Only: item control page