Reliability of the PEDro scale for rating quality of randomized controlled trials

Christopher G. Maher; Catherine Sherrington; Robert D. Herbert; Anne M. Moseley; Mark Elkins

Reliability of the PEDro scale for rating quality of randomized controlled trials

Christopher G. Maher^*, Catherine Sherrington, Robert D. Herbert, Anne M. Moseley, Mark Elkins

^*Corresponding author for this work

Research output: Contribution to journal › Article › peer-review

3568 Citations (Scopus)

Abstract

Background and Purpose. Assessment of the quality of randomized controlled trials (RCTs) is common practice in systematic reviews. However, the reliability of data obtained with most quality assessment scales has not been established. This report describes 2 studies designed to investigate the reliability of data obtained with the Physiotherapy Evidence Database (PEDro) scale developed to rate the quality of RCTs evaluating physical therapist interventions. Method. In the first study, 11 raters independently rated 25 RCTs randomly selected from the PEDro database. In the second study, 2 raters rated 120 RCTs randomly selected from the PEDro database, and disagreements were resolved by a third rater; this generated a set of individual rater and consensus ratings. The process was repeated by independent raters to create a second set of individual and consensus ratings. Reliability of ratings of PEDro scale items was calculated using multirater kappas, and reliability of the total (summed) score was calculated using intraclass correlation coefficients (ICC [1,1]). Results. The kappa value for each of the 11 items ranged from .36 to .80 for individual assessors and from .50 to .79 for consensus ratings generated by groups of 2 or 3 raters. The ICC for the total score was .56 (95% confidence interval=.47-.65) for ratings by individuals, and the ICC for consensus ratings was .68 (95% confidence interval=.57-.76). Discussion and Conclusion. The reliability of ratings of PEDro scale items varied from "fair" to "substantial," and the reliability of the total PEDro score was "fair" to "good." [Maher CG, Sherrington C, Herbert RD, et al. Reliability of the PEDro scale for rating quality of randomized controlled trials.

Original language	English
Pages (from-to)	713-721
Number of pages	9
Journal	Physical Therapy
Volume	83
Issue number	8
Publication status	Published - 1 Aug 2003
Externally published	Yes

Keywords

Evidence-based medicine
Meta-analysis
Physical therapy
Randomized controlled trials

Cite this

@article{b92f8bcc751e486fb268a60967004dce,

title = "Reliability of the PEDro scale for rating quality of randomized controlled trials",

abstract = "Background and Purpose. Assessment of the quality of randomized controlled trials (RCTs) is common practice in systematic reviews. However, the reliability of data obtained with most quality assessment scales has not been established. This report describes 2 studies designed to investigate the reliability of data obtained with the Physiotherapy Evidence Database (PEDro) scale developed to rate the quality of RCTs evaluating physical therapist interventions. Method. In the first study, 11 raters independently rated 25 RCTs randomly selected from the PEDro database. In the second study, 2 raters rated 120 RCTs randomly selected from the PEDro database, and disagreements were resolved by a third rater; this generated a set of individual rater and consensus ratings. The process was repeated by independent raters to create a second set of individual and consensus ratings. Reliability of ratings of PEDro scale items was calculated using multirater kappas, and reliability of the total (summed) score was calculated using intraclass correlation coefficients (ICC [1,1]). Results. The kappa value for each of the 11 items ranged from .36 to .80 for individual assessors and from .50 to .79 for consensus ratings generated by groups of 2 or 3 raters. The ICC for the total score was .56 (95% confidence interval=.47-.65) for ratings by individuals, and the ICC for consensus ratings was .68 (95% confidence interval=.57-.76). Discussion and Conclusion. The reliability of ratings of PEDro scale items varied from {"}fair{"} to {"}substantial,{"} and the reliability of the total PEDro score was {"}fair{"} to {"}good.{"} [Maher CG, Sherrington C, Herbert RD, et al. Reliability of the PEDro scale for rating quality of randomized controlled trials.",

keywords = "Evidence-based medicine, Meta-analysis, Physical therapy, Randomized controlled trials",

author = "Maher, {Christopher G.} and Catherine Sherrington and Herbert, {Robert D.} and Moseley, {Anne M.} and Mark Elkins",

year = "2003",

month = aug,

day = "1",

language = "English",

volume = "83",

pages = "713--721",

journal = "Physical Therapy",

issn = "0031-9023",

publisher = "American Physical Therapy Association",

number = "8",

}

TY - JOUR

T1 - Reliability of the PEDro scale for rating quality of randomized controlled trials

AU - Maher, Christopher G.

AU - Sherrington, Catherine

AU - Herbert, Robert D.

AU - Moseley, Anne M.

AU - Elkins, Mark

PY - 2003/8/1

Y1 - 2003/8/1

N2 - Background and Purpose. Assessment of the quality of randomized controlled trials (RCTs) is common practice in systematic reviews. However, the reliability of data obtained with most quality assessment scales has not been established. This report describes 2 studies designed to investigate the reliability of data obtained with the Physiotherapy Evidence Database (PEDro) scale developed to rate the quality of RCTs evaluating physical therapist interventions. Method. In the first study, 11 raters independently rated 25 RCTs randomly selected from the PEDro database. In the second study, 2 raters rated 120 RCTs randomly selected from the PEDro database, and disagreements were resolved by a third rater; this generated a set of individual rater and consensus ratings. The process was repeated by independent raters to create a second set of individual and consensus ratings. Reliability of ratings of PEDro scale items was calculated using multirater kappas, and reliability of the total (summed) score was calculated using intraclass correlation coefficients (ICC [1,1]). Results. The kappa value for each of the 11 items ranged from .36 to .80 for individual assessors and from .50 to .79 for consensus ratings generated by groups of 2 or 3 raters. The ICC for the total score was .56 (95% confidence interval=.47-.65) for ratings by individuals, and the ICC for consensus ratings was .68 (95% confidence interval=.57-.76). Discussion and Conclusion. The reliability of ratings of PEDro scale items varied from "fair" to "substantial," and the reliability of the total PEDro score was "fair" to "good." [Maher CG, Sherrington C, Herbert RD, et al. Reliability of the PEDro scale for rating quality of randomized controlled trials.

AB - Background and Purpose. Assessment of the quality of randomized controlled trials (RCTs) is common practice in systematic reviews. However, the reliability of data obtained with most quality assessment scales has not been established. This report describes 2 studies designed to investigate the reliability of data obtained with the Physiotherapy Evidence Database (PEDro) scale developed to rate the quality of RCTs evaluating physical therapist interventions. Method. In the first study, 11 raters independently rated 25 RCTs randomly selected from the PEDro database. In the second study, 2 raters rated 120 RCTs randomly selected from the PEDro database, and disagreements were resolved by a third rater; this generated a set of individual rater and consensus ratings. The process was repeated by independent raters to create a second set of individual and consensus ratings. Reliability of ratings of PEDro scale items was calculated using multirater kappas, and reliability of the total (summed) score was calculated using intraclass correlation coefficients (ICC [1,1]). Results. The kappa value for each of the 11 items ranged from .36 to .80 for individual assessors and from .50 to .79 for consensus ratings generated by groups of 2 or 3 raters. The ICC for the total score was .56 (95% confidence interval=.47-.65) for ratings by individuals, and the ICC for consensus ratings was .68 (95% confidence interval=.57-.76). Discussion and Conclusion. The reliability of ratings of PEDro scale items varied from "fair" to "substantial," and the reliability of the total PEDro score was "fair" to "good." [Maher CG, Sherrington C, Herbert RD, et al. Reliability of the PEDro scale for rating quality of randomized controlled trials.

KW - Evidence-based medicine

KW - Meta-analysis

KW - Physical therapy

KW - Randomized controlled trials

UR - http://www.scopus.com/inward/record.url?scp=0141447463&partnerID=8YFLogxK

M3 - Article

C2 - 12882612

AN - SCOPUS:0141447463

SN - 0031-9023

VL - 83

SP - 713

EP - 721

JO - Physical Therapy

JF - Physical Therapy

IS - 8

ER -

Reliability of the PEDro scale for rating quality of randomized controlled trials

Abstract

Keywords

Other files and links

Fingerprint

Cite this