OpenLB
Open Library of Bioscience
Advanced Search
Research synthesis methods
Nov, 2023;14 (6): 794-806.

Improved methods to construct prediction intervals for network meta-analysis.

Hisashi Noma, Yasuyuki Hamura, Shonosuke Sugasawa, Toshi A Furukawa
Author Information
  1. Hisashi Noma: Department of Data Science, The Institute of Statistical Mathematics, Tokyo, Japan. ORCID
  2. Yasuyuki Hamura: Graduate School of Economics, Kyoto University, Kyoto, Japan.
  3. Shonosuke Sugasawa: Faculty of Economics, Keio University, Tokyo, Japan.
  4. Toshi A Furukawa: Departments of Health Promotion and Human Behavior, Kyoto University Graduate School of Medicine/School of Public Health, Kyoto, Japan. ORCID
PMID: 37399809 DOI: 10.1002/jrsm.1651

Abstract

Network meta-analysis has played an important role in evidence-based medicine for assessing the comparative effectiveness of multiple available treatments. The prediction interval has been one of the standard outputs in recent network meta-analysis as an effective measure that enables simultaneous assessment of uncertainties in treatment effects and heterogeneity among studies. To construct the prediction interval, a large-sample approximating method based on the t-distribution has generally been applied in practice; however, recent studies have shown that similar t-approximation methods for conventional pairwise meta-analyses can substantially underestimate the uncertainty under realistic situations. In this article, we performed simulation studies to assess the validity of the current standard method for network meta-analysis, and we show that its validity can also be violated under realistic situations. To address the invalidity issue, we developed two new methods to construct more accurate prediction intervals through bootstrap and Kenward-Roger-type adjustment. In simulation experiments, the two proposed methods exhibited better coverage performance and generally provided wider prediction intervals than the ordinary t-approximation. We also developed an R package, PINMA (https://cran.r-project.org/web/packages/PINMA/), to perform the proposed methods using simple commands. We illustrate the effectiveness of the proposed methods through applications to two real network meta-analyses.

Keywords

Kenward-Roger-type adjustment bootstrap higher-order approximation network meta-analysis prediction interval

References

  1. Salanti G. Indirect and mixed-treatment comparison, network, or multiple-treatments meta-analysis: many names, many benefits, many concerns for the next generation evidence synthesis tool. Res Synth Methods. 2012;3(2):80-97.
  2. Salanti G, Higgins JP, Ades AE, Ioannidis JP. Evaluation of networks of randomized trials. Stat Methods Med Res. 2008;17(3):279-301.
  3. Higgins JPT, Thompson SG, Spiegelhalter DJ. A re-evaluation of random-effects meta-analysis. J R Stat Soc A. 2009;172(1):137-159.
  4. Riley RD, Higgins JPT, Deeks JJ. Interpretation of random effects meta-analyses. BMJ. 2011;342:d549.
  5. IntHout J, Ioannidis JP, Rovers MM, Goeman JJ. Plea for routinely presenting prediction intervals in meta-analysis. BMJ Open. 2016;6(7):e010247.
  6. Veroniki AA, Jackson D, Bender R, et al. Methods to calculate uncertainty in the estimated overall effect size from a random-effects meta-analysis. Res Synth Methods. 2019;10(1):23-43.
  7. Chaimani A, Salanti G. Visualizing assumptions and results in network meta-analysis: the network graphs package. Stata J. 2015;15(4):905-920.
  8. Follmann DA, Proschan MA. Valid inference in random effects meta-analysis. Biometrics. 1999;55(3):732-737.
  9. Partlett C, Riley RD. Random effects meta-analysis: coverage performance of 95% confidence and prediction intervals following REML estimation. Stat Med. 2017;36(2):301-317.
  10. Nagashima K, Noma H, Furukawa TA. Prediction intervals for random-effects meta-analysis: a confidence distribution approach. Stat Methods Med Res. 2019;28(6):1689-1702.
  11. Hamaguchi Y, Noma H, Nagashima K, Yamada T, Furukawa TA. Frequentist performances of Bayesian prediction intervals for random-effects meta-analysis. Biom J. 2021;63(2):394-405.
  12. Noma H, Nagashima K, Furukawa TA. Permutation inference methods for multivariate meta-analysis. Biometrics. 2020;76(1):337-347.
  13. Noma H, Nagashima K, Maruo K, Gosho M, Furukawa TA. Bartlett-type corrections and bootstrap adjustments of likelihood-based inference methods for network meta-analysis. Stat Med. 2018;37(7):1178-1190.
  14. Sugasawa S, Noma H. A unified method for improved inference in random effects meta-analysis. Biostatistics. 2019;22(1):114-130.
  15. Chatterjee S, Lachat P, Li H. Parametric bootstrap approximation to the distribution of eblup and related prediction intervals in linear mixed models. Ann Stat. 2008;36(3):1221-1245.
  16. Lahiri P. On the impact of bootstrap in survey sampling and small area estimation. Stat Sci. 2003;18(2):199-210.
  17. Saegusa T, Sugasawa S, Lahiri P. Parametric bootstrap confidence intervals for the multivariate Fay-Herriot model. J Surv Stat Meth. 2022;10(1):115-130.
  18. Kenward MG, Roger JH. Small sample inference for fixed effects from restricted maximum likelihood. Biometrics. 1997;53(3):983-997.
  19. Noma H, Hamura Y, Gosho M, Furukawa TA. Kenward-Roger-type corrections for inference methods of network meta-analysis and meta-regression. Res Synth Methods. 2023. doi:10.1002/jrsm.1652
  20. Wu HY, Huang JW, Lin HJ, et al. Comparative effectiveness of renin-angiotensin system blockers and other antihypertensive drugs in patients with diabetes: systematic review and bayesian network meta-analysis. BMJ. 2013;347:f6008.
  21. Siontis GC, Mavridis D, Greenwood JP, et al. Outcomes of non-invasive diagnostic modalities for the detection of coronary artery disease: network meta-analysis of diagnostic randomised controlled trials. BMJ. 2018;360:k504.
  22. White IR. Network meta-analysis. Stata J. 2015;15(4):951-985.
  23. White IR, Barrett JK, Jackson D, Higgins JP. Consistency and inconsistency in network meta-analysis: model estimation using multivariate meta-regression. Res Synth Methods. 2012;3(2):111-125.
  24. Wei Y, Higgins JP. Estimating within-study covariances in multivariate meta-analysis with multiple outcomes. Stat Med. 2013;32(7):1191-1205.
  25. Trikalinos T, Olkin I. A method for the meta-analysis of mutually exclusive binary outcomes. Stat Med. 2008;27:4279-4300.
  26. Higgins JP, Whitehead A. Borrowing strength from external trials in a meta-analysis. Stat Med. 1996;15(24):2733-2749.
  27. Lu G, Ades AE. Modeling between-trial variance structure in mixed treatment comparisons. Biostatistics. 2009;10(4):792-805.
  28. White IR. Multivariate random-effects meta-regression: updates to mvmeta. Stata J. 2011;11(2):255-270.
  29. Jackson D, White IR, Riley RD. A matrix-based method of moments for fitting the multivariate random effects model for meta-analysis and meta-regression. Biom J. 2013;55(2):231-245.
  30. Jackson D, White IR, Thompson SG. Extending DerSimonian and Laird's methodology to perform multivariate random effects meta-analyses. Stat Med. 2010;29(12):1282-1297.
  31. Kackar AN, Harville DA. Approximations for standard errors of estimators of fixed and random effects in mixed linear models. J Am Stat Assoc. 1984;79(388):853-862.
  32. Balduzzi S, Rücker G, Nikolakopoulou A, et al. Netmeta: an R package for network meta-analysis using frequentist methods. J Stat Softw. 2023;106:1-40.
  33. Friede T, Rover C, Wandel S, Neuenschwander B. Meta-analysis of few small studies in orphan diseases. Res Synth Methods. 2017;8(1):79-91.
  34. Noma H. Confidence intervals for a random-effects meta-analysis based on Bartlett-type corrections. Stat Med. 2011;30(28):3304-3312.
  35. Knapp G, Hartung J. Improved tests for a random effects meta-regression with a single covariate. Stat Med. 2003;22(17):2693-2710.
  36. Viechtbauer W, Lopez-Lopez JA, Sanchez-Meca J, Marin-Martinez F. A comparison of procedures to test for moderators in mixed-effects meta-regression models. Psychol Methods. 2015;20(3):360-374.
  37. Morey RD, Hoekstra R, Rouder JN, Lee MD, Wagenmakers EJ. The fallacy of placing confidence in confidence intervals. Psychon Bull Rev. 2016;23(1):103-123.
  38. Wiksten A, Rucker G, Schwarzer G. Hartung-Knapp method is not always conservative compared with fixed-effect meta-analysis. Stat Med. 2016;35(15):2503-2515.

Grants

  1. JP22H03554/Japan Society for the Promotion of Science
  2. JP22K19688/Japan Society for the Promotion of Science

MeSH Term

Network Meta-Analysis
Computer Simulation
Meta-Analysis Journal Article
Article has an altmetric score of 4

Word Cloud

Created with Highcharts 10.0.0predictionmethodsmeta-analysisnetworkintervalstudiesconstructtwointervalsproposedeffectivenessstandardrecentmethodgenerallyt-approximationmeta-analysescanrealisticsituationssimulationvalidityalsodevelopedbootstrapKenward-Roger-typeadjustmentNetworkplayedimportantroleevidence-basedmedicineassessingcomparativemultipleavailabletreatmentsoneoutputseffectivemeasureenablessimultaneousassessmentuncertaintiestreatmenteffectsheterogeneityamonglarge-sampleapproximatingbasedt-distributionappliedpracticehowevershownsimilarconventionalpairwisesubstantiallyunderestimateuncertaintyarticleperformedassesscurrentshowviolatedaddressinvalidityissuenewaccurateexperimentsexhibitedbettercoverageperformanceprovidedwiderordinaryRpackagePINMAhttps://cranr-projectorg/web/packages/PINMA/performusingsimplecommandsillustrateapplicationsrealImprovedhigher-orderapproximation

Similar Articles

Cited By