TY - JOUR
T1 - Sensitivity analyses assessing the impact of early stopping on systematic reviews
T2 - recommendations for interpreting guidelines
AU - Marschner, Ian C.
AU - Askie, Lisa M.
AU - Schou, I. Manjula
PY - 2020/3
Y1 - 2020/3
N2 - The CONSORT Statement says that data-driven early stopping of a clinical trial is likely to weaken the inferences that can be drawn from the trial. The GRADE guidelines go further, saying that early stopping is a study limitation that carries the risk of bias, and recommending sensitivity analyses in which trials stopped early are omitted from evidence synthesis. Despite extensive debate in the literature over these issues, the existence of clear recommendations in high profile guidelines makes it inevitable that systematic reviewers will consider sensitivity analyses investigating the impact of early stopping. The purpose of this article is to assess methodologies for conducting such sensitivity analyses, and to make recommendations about how the guidelines should be interpreted. We begin with a clarifying overview of the impacts of early stopping on treatment effect estimation in single studies and meta-analyses. We then warn against naive approaches for conducting sensitivity analyses, including simply omitting trials stopped early from meta-analyses. This approach underestimates treatment effects, which may have serious implications if cost-effectiveness analyses determine whether treatments are made widely available. Instead, we discuss two unbiased approaches to sensitivity analysis, one of which is straightforward but statistically inefficient, and the other of which achieves greater statistical efficiency by making use of recent methodological developments in the analysis of clinical trials. We end with recommendations for interpreting: (a) the CONSORT Statement on reporting of reasons for early stopping, and (b) the GRADE guidelines on sensitivity analyses assessing the impact of early stopping.
AB - The CONSORT Statement says that data-driven early stopping of a clinical trial is likely to weaken the inferences that can be drawn from the trial. The GRADE guidelines go further, saying that early stopping is a study limitation that carries the risk of bias, and recommending sensitivity analyses in which trials stopped early are omitted from evidence synthesis. Despite extensive debate in the literature over these issues, the existence of clear recommendations in high profile guidelines makes it inevitable that systematic reviewers will consider sensitivity analyses investigating the impact of early stopping. The purpose of this article is to assess methodologies for conducting such sensitivity analyses, and to make recommendations about how the guidelines should be interpreted. We begin with a clarifying overview of the impacts of early stopping on treatment effect estimation in single studies and meta-analyses. We then warn against naive approaches for conducting sensitivity analyses, including simply omitting trials stopped early from meta-analyses. This approach underestimates treatment effects, which may have serious implications if cost-effectiveness analyses determine whether treatments are made widely available. Instead, we discuss two unbiased approaches to sensitivity analysis, one of which is straightforward but statistically inefficient, and the other of which achieves greater statistical efficiency by making use of recent methodological developments in the analysis of clinical trials. We end with recommendations for interpreting: (a) the CONSORT Statement on reporting of reasons for early stopping, and (b) the GRADE guidelines on sensitivity analyses assessing the impact of early stopping.
KW - early stopping
KW - interim analysis
KW - meta-analysis
KW - randomized controlled trial
KW - sensitivity analysis
UR - http://www.scopus.com/inward/record.url?scp=85079045404&partnerID=8YFLogxK
UR - http://purl.org/au-research/grants/nhmrc/1150467
U2 - 10.1002/jrsm.1394
DO - 10.1002/jrsm.1394
M3 - Article
C2 - 31901013
AN - SCOPUS:85079045404
SN - 1759-2879
VL - 11
SP - 287
EP - 300
JO - Research Synthesis Methods
JF - Research Synthesis Methods
IS - 2
ER -