Quantitative versus qualitative notions of validity: investigating the validity of PSYCHLOPS as a patient outcome measure in trials of insomnia and sleep problems

Davy, Zowie and Quinn, Casey and Wilson, Helen and Togher, Fiona and Siriwardena, A Niroshan (2011) Quantitative versus qualitative notions of validity: investigating the validity of PSYCHLOPS as a patient outcome measure in trials of insomnia and sleep problems. In: 40th Annual Scientific Meeting of the SAPC, 6-8 July 2011, University of Bristol, Bristol .

Documents
Quantitative versus qualitative notions of validity: investigating the validity of PSYCHLOPS as a patient outcome measure in trials of insomnia and sleep problems
Presentation
[img]
[Download]
[img]
Preview
PDF
SAPCBristol2011Psychlops.pdf - Whole Document
Available under License Creative Commons Attribution Non-commercial No Derivatives.

1MB

Official URL: http://www.sapc.ac.uk/

Abstract

Introduction
PSYCHLOPS (‘Psychological Outcome Profiles’)is a novel patient-reported outcome measure developed by Ashworth and colleagues that allows clients to evaluate their progress by measuring severity scores for self-defined problems at the start of, during and after therapy. PSYCHLOPS was administered in a study, ‘Resources for Effective Sleep Treatment (REST)’, designed to assess the feasibility of training primary care clinicians (GPs and nurses) to deliver ‘problem focused therapy’ for insomnia to improve sleep outcomes in adults with insomnia. We aimed in this sub-study to assess both qualitative and quantitative notions of validity of PSYCHLOPS in sleep studies.
Methods
PSYCHLOPS was administered by a practice nurse to trial participants in two intervention primary care sites (participants offered ‘problem focused therapy’) and two control sites (participants offered sleep hygiene information) as part of a cluster randomized feasibility study. Other predetermined outcome measures including the Pittsburgh Sleep Quality Index (PSQI), Insomnia Severity Index (ISI), Beck Depression Inventory (BDI) and sleep diaries. We investigated qualitative and quantitative markers of validity including content validity (nature of response to items), internal validity (reliability), criterion validity (strength of relationship with a related variable) and construct validity (strength of relationship with an underlying variable).
Results
A qualitative analysis of test content showed a representative sample of the behavior domain expected in relation to sleep problems including sleep, sleep-related problems and underlying psychosocial and physical problems such as anxiety, worry and arthritis. A positive, statistically significant correlation was found between PSYCHLOPS and ISI (Kendall’s tau = 0.47, p<0.001) but not between PSYCHLOPS and PSQI (Kendall’s tau = 0.13, p=0.24) suggesting partial criterion validity with regards to insomnia impact. We also found a positive correlation between PSYCHLOPS and BDI and (Kendall’s tau = 0.47, p<0.001) indicating construct validity.
Conclusion
PSYCHLOPS demonstrated some aspects of quantitative validity supporting its wider use in this study setting. However, applying qualitative notions of validity uncovered a number of assumptions and response biases that should be acknowledged in future studies using the tool. Qualitative analysis added an additional dimension to the assessment of validity which should be considered in other validation studies.

Item Type:Conference or Workshop Item (Presentation)
Additional Information:Introduction PSYCHLOPS (‘Psychological Outcome Profiles’)is a novel patient-reported outcome measure developed by Ashworth and colleagues that allows clients to evaluate their progress by measuring severity scores for self-defined problems at the start of, during and after therapy. PSYCHLOPS was administered in a study, ‘Resources for Effective Sleep Treatment (REST)’, designed to assess the feasibility of training primary care clinicians (GPs and nurses) to deliver ‘problem focused therapy’ for insomnia to improve sleep outcomes in adults with insomnia. We aimed in this sub-study to assess both qualitative and quantitative notions of validity of PSYCHLOPS in sleep studies. Methods PSYCHLOPS was administered by a practice nurse to trial participants in two intervention primary care sites (participants offered ‘problem focused therapy’) and two control sites (participants offered sleep hygiene information) as part of a cluster randomized feasibility study. Other predetermined outcome measures including the Pittsburgh Sleep Quality Index (PSQI), Insomnia Severity Index (ISI), Beck Depression Inventory (BDI) and sleep diaries. We investigated qualitative and quantitative markers of validity including content validity (nature of response to items), internal validity (reliability), criterion validity (strength of relationship with a related variable) and construct validity (strength of relationship with an underlying variable). Results A qualitative analysis of test content showed a representative sample of the behavior domain expected in relation to sleep problems including sleep, sleep-related problems and underlying psychosocial and physical problems such as anxiety, worry and arthritis. A positive, statistically significant correlation was found between PSYCHLOPS and ISI (Kendall’s tau = 0.47, p<0.001) but not between PSYCHLOPS and PSQI (Kendall’s tau = 0.13, p=0.24) suggesting partial criterion validity with regards to insomnia impact. We also found a positive correlation between PSYCHLOPS and BDI and (Kendall’s tau = 0.47, p<0.001) indicating construct validity. Conclusion PSYCHLOPS demonstrated some aspects of quantitative validity supporting its wider use in this study setting. However, applying qualitative notions of validity uncovered a number of assumptions and response biases that should be acknowledged in future studies using the tool. Qualitative analysis added an additional dimension to the assessment of validity which should be considered in other validation studies.
Keywords:general practice, primary care, Patient Reported Outcome Measure, validity, qualitative
Subjects:B Subjects allied to Medicine > B710 Community Nursing
A Medicine and Dentistry > A300 Clinical Medicine
Divisions:College of Social Science > School of Health & Social Care
ID Code:4574
Deposited By: Niro Siriwardena
Deposited On:13 Jul 2011 07:33
Last Modified:28 Aug 2014 13:01

Repository Staff Only: item control page