Abstract

Introduction
PSYCHLOPS (‘Psychological Outcome Profiles’)is a novel patient-reported outcome measure developed by Ashworth and colleagues that allows clients to evaluate their progress by measuring severity scores for self-defined problems at the start of, during and after therapy. PSYCHLOPS was administered in a study, ‘Resources for Effective Sleep Treatment (REST)’, designed to assess the feasibility of training primary care clinicians (GPs and nurses) to deliver ‘problem focused therapy’ for insomnia to improve sleep outcomes in adults with insomnia. We aimed in this sub-study to assess both qualitative and quantitative notions of validity of PSYCHLOPS in sleep studies.
Methods
PSYCHLOPS was administered by a practice nurse to trial participants in two intervention primary care sites (participants offered ‘problem focused therapy’) and two control sites (participants offered sleep hygiene information) as part of a cluster randomized feasibility study. Other predetermined outcome measures including the Pittsburgh Sleep Quality Index (PSQI), Insomnia Severity Index (ISI), Beck Depression Inventory (BDI) and sleep diaries. We investigated qualitative and quantitative markers of validity including content validity (nature of response to items), internal validity (reliability), criterion validity (strength of relationship with a related variable) and construct validity (strength of relationship with an underlying variable).
Results
A qualitative analysis of test content showed a representative sample of the behavior domain expected in relation to sleep problems including sleep, sleep-related problems and underlying psychosocial and physical problems such as anxiety, worry and arthritis. A positive, statistically significant correlation was found between PSYCHLOPS and ISI (Kendall’s tau = 0.47, p<0.001) but not between PSYCHLOPS and PSQI (Kendall’s tau = 0.13, p=0.24) suggesting partial criterion validity with regards to insomnia impact. We also found a positive correlation between PSYCHLOPS and BDI and (Kendall’s tau = 0.47, p<0.001) indicating construct validity.
Conclusion
PSYCHLOPS demonstrated some aspects of quantitative validity supporting its wider use in this study setting. However, applying qualitative notions of validity uncovered a number of assumptions and response biases that should be acknowledged in future studies using the tool. Qualitative analysis added an additional dimension to the assessment of validity which should be considered in other validation studies.