Measurement of symptom change following web-based psychotherapy: statistical characteristics and analytical methods for measuring and interpreting change

Eyal Karin*, Blake F. Dear, Gillian Z. Heller, Milena Gandy, Nickolai Titov

*Corresponding author for this work

    Research output: Contribution to journalArticlepeer-review

    33 Citations (Scopus)
    38 Downloads (Pure)


    Background: Accurate measurement of treatment-related change is a key part of psychotherapy research and the investigation of treatment efficacy. For this reason, the ability to measure change with accurate and valid methods is critical for psychotherapy. Objective: The aims of this study were to (1) explore the underlying characteristics of depressive symptom change, measured with the nine-item Patient Health Questionnaire (PHQ-9), following psychotherapy, and (2) compare the suitability of different ways to measure and interpret symptom change. A treatment sample of Web-based psychotherapy participants (n=1098) and a waitlist sample (n=96) were used to (1) explore the statistical characteristics of depressive symptom change, and (2) compare the suitability of two common types of change functions: Linear and proportional change. Methods: These objectives were explored using hypotheses that tested (1) the relationship between baseline symptoms and the rate of change, (2) the shape of symptom score distribution following treatment, and (3) measurement error associated with linear and proportional measurement models. Results: Findings demonstrated that (1) individuals with severe depressive baseline symptoms had greater reductions in symptom scores than individuals with mild baseline symptoms (11.4 vs 3.7); however, as a percentage measurement, change remained similar across individuals with mild, moderate, or severe baseline symptoms (50%-55%); (2) positive skewness was observed in PHQ-9 score distributions following treatment; and (3) models that measured symptom change as a proportional function resulted in greater model fit and reduced measurement error (<30%). Conclusions: This study suggests that symptom scales, sharing an implicit feature of score bounding, are associated with a proportional function of change. Selecting statistics that overlook this proportional change (eg, Cohen d) is problematic and leads to (1) artificially increased estimates of change with higher baseline symptoms, (2) increased measurement error, and (3) confounded estimates of treatment efficacy and clinical change. Implications, limitations, and idiosyncrasies from these results are discussed.

    Original languageEnglish
    Article numbere10200
    Pages (from-to)1-13
    Number of pages13
    JournalJMIR Mental Health
    Issue number3
    Publication statusPublished - Jul 2018

    Bibliographical note

    Copyright the Author(s) 2018. Version archived for private and non-commercial use with the permission of the author/s and according to publisher conditions. For further rights please contact the publisher.


    • clinical measurement
    • treatment evaluation
    • symptom change
    • symptom scales
    • psychotherapeutic change


    Dive into the research topics of 'Measurement of symptom change following web-based psychotherapy: statistical characteristics and analytical methods for measuring and interpreting change'. Together they form a unique fingerprint.

    Cite this