Do systematic review updates target questions where evidence accumulates faster?

OBJECTIVE: There are several methods available for determining if a systematic review needs to be updated, but little is known about whether reviewers prioritise clinical questions with new evidence. Our aim was to determine whether systematic reviews were targeted for updating after relevant trials were published.

METHODS: Systematic reviews published in the Cochrane Database of Systematic Reviews in 2010 were selected if they included at least one clinical trial; updated before December 1 2016; performed a new search; and did not change populations, interventions, outcomes, or comparators. Using the updated set of trials to define the most recent evidence base, we retrospectively quantified the accumulation of new evidence between the search dates of the reviews and their updates. Recording trial publication dates (and trial completion dates where registration information was available), the ongoing completeness of a review was determined by the number of participants included in the review as a proportion of the total number of participants available as new evidence was published. We determined whether reviews with a signal of new evidence (≤ 90% completeness within a year of the search date) were updated faster (using time between search dates) than reviews without a signal.

RESULTS: From 773 articles published in 2010, 53 systematic reviews were sampled for analysis. The median update time was 41 months (IQR 35-60). For 55% (29/53) of the reviews, no new trials were added in the update. Within a year of search date, the reviews covered between 72.4% and 100% of the published trial participants. Clinical questions (with ≤ 90% completeness) were not targeted for update (N=12, median 49 months) earlier than those without (N=41, median 40 months); p=0.017 in a Wilcoxon rank sum test. The 14 reviews with complete registration information covered a median of 93.9% (IQR 83.8%-100%) of completed trial participants at the search date, and 85.8% (IQR 73.8%-96.0%) within a year of search.

CONCLUSION: Updates to Cochrane systematic reviews mostly found no new evidence, and updates were not targeted at questions where new evidence was published. Methods for automatically monitoring registrations and publications may help to prioritise systematic review updates.