Computer modeling of diabetes and its transparency: A report on the Eighth Mount Hood Challenge

Andrew J. Palmer, Lei Si, Michelle Tew, Xinyang Hua, Michael S. Willis, Christian Asseburg, Philip McEwan, José Leal, Alastair Gray, Volker Foos, Mark Lamotte, Talitha Feenstra, Patrick J. O’Connor, Michael Brandle, Harry J. Smolen, James C. Gahn, William J. Valentine, Richard F. Pollock, Penny Breeze, Alan Brennan & 8 others Daniel Pollard, Wen Ye, William H. Herman, Deanna J. Isaman, Shihchen Kuo, Neda Laiteerapong, An Tran-Duy, Philip M. Clarke

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Objectives: The Eighth Mount Hood Challenge (held in St. Gallen, Switzerland, in September 2016) evaluated the transparency of model input documentation from two published health economics studies and developed guidelines for improving transparency in the reporting of input data underlying model-based economic analyses in diabetes.
Methods: Participating modeling groups were asked to reproduce the results of two published studies using the input data described in those articles. Gaps in input data were filled with assumptions reported by the modeling groups. Goodness of fit between the results reported in the target studies and the groups’ replicated outputs was evaluated using the slope of linear regression line and the coefficient of determination (R2). After a general discussion of the results, a diabetes-specific checklist for the transparency of model input was developed.
Results: Seven groups participated in the transparency challenge. The reporting of key model input parameters in the two studies, including the baseline characteristics of simulated patients, treatment effect and treatment intensification threshold assumptions, treatment effect evolution, prediction of complications and costs data, was inadequately transparent (and often missing altogether). Not surprisingly, goodness of fit was better for the study that reported its input data with more transparency. To improve the transparency in diabetes modeling, the Diabetes Modeling Input Checklist listing the minimal input data required for reproducibility in most diabetes modeling applications was developed.
Conclusions: Transparency of diabetes model inputs is important to the reproducibility and credibility of simulation results. In the Eighth Mount Hood Challenge, the Diabetes Modeling Input Checklist was developed with the goal of improving the transparency of input data reporting and reproducibility of diabetes simulation model results.
LanguageEnglish
Pages724-731
Number of pages8
JournalValue in Health
Volume21
Issue number6
Early online date9 Apr 2018
DOIs
Publication statusPublished - Jun 2018
Externally publishedYes

Fingerprint

Checklist
Research Design
Economic Models
Switzerland
Documentation
Linear Models
Therapeutics
Economics
Guidelines
Costs and Cost Analysis
Health

Keywords

  • computer modeling
  • diabetes
  • Mount Hood Challenge
  • transparency

Cite this

Palmer, A. J., Si, L., Tew, M., Hua, X., Willis, M. S., Asseburg, C., ... Clarke, P. M. (2018). Computer modeling of diabetes and its transparency: A report on the Eighth Mount Hood Challenge. Value in Health, 21(6), 724-731. https://doi.org/10.1016/j.jval.2018.02.002
Palmer, Andrew J. ; Si, Lei ; Tew, Michelle ; Hua, Xinyang ; Willis, Michael S. ; Asseburg, Christian ; McEwan, Philip ; Leal, José ; Gray, Alastair ; Foos, Volker ; Lamotte, Mark ; Feenstra, Talitha ; O’Connor, Patrick J. ; Brandle, Michael ; Smolen, Harry J. ; Gahn, James C. ; Valentine, William J. ; Pollock, Richard F. ; Breeze, Penny ; Brennan, Alan ; Pollard, Daniel ; Ye, Wen ; Herman, William H. ; Isaman, Deanna J. ; Kuo, Shihchen ; Laiteerapong, Neda ; Tran-Duy, An ; Clarke, Philip M. / Computer modeling of diabetes and its transparency : A report on the Eighth Mount Hood Challenge. In: Value in Health. 2018 ; Vol. 21, No. 6. pp. 724-731.
@article{891870f5432647babeebf6960ad5f59f,
title = "Computer modeling of diabetes and its transparency: A report on the Eighth Mount Hood Challenge",
abstract = "Objectives: The Eighth Mount Hood Challenge (held in St. Gallen, Switzerland, in September 2016) evaluated the transparency of model input documentation from two published health economics studies and developed guidelines for improving transparency in the reporting of input data underlying model-based economic analyses in diabetes.Methods: Participating modeling groups were asked to reproduce the results of two published studies using the input data described in those articles. Gaps in input data were filled with assumptions reported by the modeling groups. Goodness of fit between the results reported in the target studies and the groups’ replicated outputs was evaluated using the slope of linear regression line and the coefficient of determination (R2). After a general discussion of the results, a diabetes-specific checklist for the transparency of model input was developed.Results: Seven groups participated in the transparency challenge. The reporting of key model input parameters in the two studies, including the baseline characteristics of simulated patients, treatment effect and treatment intensification threshold assumptions, treatment effect evolution, prediction of complications and costs data, was inadequately transparent (and often missing altogether). Not surprisingly, goodness of fit was better for the study that reported its input data with more transparency. To improve the transparency in diabetes modeling, the Diabetes Modeling Input Checklist listing the minimal input data required for reproducibility in most diabetes modeling applications was developed.Conclusions: Transparency of diabetes model inputs is important to the reproducibility and credibility of simulation results. In the Eighth Mount Hood Challenge, the Diabetes Modeling Input Checklist was developed with the goal of improving the transparency of input data reporting and reproducibility of diabetes simulation model results.",
keywords = "computer modeling, diabetes, Mount Hood Challenge, transparency",
author = "Palmer, {Andrew J.} and Lei Si and Michelle Tew and Xinyang Hua and Willis, {Michael S.} and Christian Asseburg and Philip McEwan and Jos{\'e} Leal and Alastair Gray and Volker Foos and Mark Lamotte and Talitha Feenstra and O’Connor, {Patrick J.} and Michael Brandle and Smolen, {Harry J.} and Gahn, {James C.} and Valentine, {William J.} and Pollock, {Richard F.} and Penny Breeze and Alan Brennan and Daniel Pollard and Wen Ye and Herman, {William H.} and Isaman, {Deanna J.} and Shihchen Kuo and Neda Laiteerapong and An Tran-Duy and Clarke, {Philip M.}",
year = "2018",
month = "6",
doi = "10.1016/j.jval.2018.02.002",
language = "English",
volume = "21",
pages = "724--731",
journal = "Value in Health",
issn = "1098-3015",
publisher = "Blackwell Science",
number = "6",

}

Palmer, AJ, Si, L, Tew, M, Hua, X, Willis, MS, Asseburg, C, McEwan, P, Leal, J, Gray, A, Foos, V, Lamotte, M, Feenstra, T, O’Connor, PJ, Brandle, M, Smolen, HJ, Gahn, JC, Valentine, WJ, Pollock, RF, Breeze, P, Brennan, A, Pollard, D, Ye, W, Herman, WH, Isaman, DJ, Kuo, S, Laiteerapong, N, Tran-Duy, A & Clarke, PM 2018, 'Computer modeling of diabetes and its transparency: A report on the Eighth Mount Hood Challenge', Value in Health, vol. 21, no. 6, pp. 724-731. https://doi.org/10.1016/j.jval.2018.02.002

Computer modeling of diabetes and its transparency : A report on the Eighth Mount Hood Challenge. / Palmer, Andrew J.; Si, Lei; Tew, Michelle; Hua, Xinyang; Willis, Michael S.; Asseburg, Christian; McEwan, Philip; Leal, José; Gray, Alastair; Foos, Volker; Lamotte, Mark; Feenstra, Talitha; O’Connor, Patrick J.; Brandle, Michael; Smolen, Harry J.; Gahn, James C.; Valentine, William J.; Pollock, Richard F.; Breeze, Penny; Brennan, Alan; Pollard, Daniel; Ye, Wen; Herman, William H.; Isaman, Deanna J.; Kuo, Shihchen; Laiteerapong, Neda; Tran-Duy, An; Clarke, Philip M.

In: Value in Health, Vol. 21, No. 6, 06.2018, p. 724-731.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Computer modeling of diabetes and its transparency

T2 - Value in Health

AU - Palmer, Andrew J.

AU - Si, Lei

AU - Tew, Michelle

AU - Hua, Xinyang

AU - Willis, Michael S.

AU - Asseburg, Christian

AU - McEwan, Philip

AU - Leal, José

AU - Gray, Alastair

AU - Foos, Volker

AU - Lamotte, Mark

AU - Feenstra, Talitha

AU - O’Connor, Patrick J.

AU - Brandle, Michael

AU - Smolen, Harry J.

AU - Gahn, James C.

AU - Valentine, William J.

AU - Pollock, Richard F.

AU - Breeze, Penny

AU - Brennan, Alan

AU - Pollard, Daniel

AU - Ye, Wen

AU - Herman, William H.

AU - Isaman, Deanna J.

AU - Kuo, Shihchen

AU - Laiteerapong, Neda

AU - Tran-Duy, An

AU - Clarke, Philip M.

PY - 2018/6

Y1 - 2018/6

N2 - Objectives: The Eighth Mount Hood Challenge (held in St. Gallen, Switzerland, in September 2016) evaluated the transparency of model input documentation from two published health economics studies and developed guidelines for improving transparency in the reporting of input data underlying model-based economic analyses in diabetes.Methods: Participating modeling groups were asked to reproduce the results of two published studies using the input data described in those articles. Gaps in input data were filled with assumptions reported by the modeling groups. Goodness of fit between the results reported in the target studies and the groups’ replicated outputs was evaluated using the slope of linear regression line and the coefficient of determination (R2). After a general discussion of the results, a diabetes-specific checklist for the transparency of model input was developed.Results: Seven groups participated in the transparency challenge. The reporting of key model input parameters in the two studies, including the baseline characteristics of simulated patients, treatment effect and treatment intensification threshold assumptions, treatment effect evolution, prediction of complications and costs data, was inadequately transparent (and often missing altogether). Not surprisingly, goodness of fit was better for the study that reported its input data with more transparency. To improve the transparency in diabetes modeling, the Diabetes Modeling Input Checklist listing the minimal input data required for reproducibility in most diabetes modeling applications was developed.Conclusions: Transparency of diabetes model inputs is important to the reproducibility and credibility of simulation results. In the Eighth Mount Hood Challenge, the Diabetes Modeling Input Checklist was developed with the goal of improving the transparency of input data reporting and reproducibility of diabetes simulation model results.

AB - Objectives: The Eighth Mount Hood Challenge (held in St. Gallen, Switzerland, in September 2016) evaluated the transparency of model input documentation from two published health economics studies and developed guidelines for improving transparency in the reporting of input data underlying model-based economic analyses in diabetes.Methods: Participating modeling groups were asked to reproduce the results of two published studies using the input data described in those articles. Gaps in input data were filled with assumptions reported by the modeling groups. Goodness of fit between the results reported in the target studies and the groups’ replicated outputs was evaluated using the slope of linear regression line and the coefficient of determination (R2). After a general discussion of the results, a diabetes-specific checklist for the transparency of model input was developed.Results: Seven groups participated in the transparency challenge. The reporting of key model input parameters in the two studies, including the baseline characteristics of simulated patients, treatment effect and treatment intensification threshold assumptions, treatment effect evolution, prediction of complications and costs data, was inadequately transparent (and often missing altogether). Not surprisingly, goodness of fit was better for the study that reported its input data with more transparency. To improve the transparency in diabetes modeling, the Diabetes Modeling Input Checklist listing the minimal input data required for reproducibility in most diabetes modeling applications was developed.Conclusions: Transparency of diabetes model inputs is important to the reproducibility and credibility of simulation results. In the Eighth Mount Hood Challenge, the Diabetes Modeling Input Checklist was developed with the goal of improving the transparency of input data reporting and reproducibility of diabetes simulation model results.

KW - computer modeling

KW - diabetes

KW - Mount Hood Challenge

KW - transparency

UR - http://www.scopus.com/inward/record.url?scp=85045039038&partnerID=8YFLogxK

U2 - 10.1016/j.jval.2018.02.002

DO - 10.1016/j.jval.2018.02.002

M3 - Article

VL - 21

SP - 724

EP - 731

JO - Value in Health

JF - Value in Health

SN - 1098-3015

IS - 6

ER -