Abstract
Meta-analysis of binary outcome data faces often a situation where studies with a rare event are part of the set of studies to be considered. These studies have low occurrence of event counts to the extreme that no events occur in one or both groups to be compared. This raises issues how to estimate validly the summary risk or rate ratio across studies. A preferred choice is the Mantel–Haenszel estimator, which is still defined in the situation of zero studies unless all studies have zeros in one of the groups to be compared. For this situation, a modified Mantel–Haenszel estimator is suggested and shown to perform well by means of simulation work. Also, confidence interval estimation is discussed and evaluated in a simulation study. In a second part, heterogeneity of relative risk across studies is investigated with a new chi-square type statistic which is based on a conditional binomial distribution where the conditioning is on the event margin for each study. This is necessary as the conventional Q-statistic is undefined in the occurrence of zero studies. The null-distribution of the proposed Q-statistic is obtained by means of a parametric bootstrap as a chi-square approximation is not valid for rare events meta-analysis, as bootstrapping of the null-distribution shows. In addition, for the effect heterogeneity situation, confidence interval estimation is considered using a nonparametric bootstrap procedure. The proposed techniques are illustrated at hand of three meta-analytic data sets.
Funding source: Deutsche Forschungsgemeinschaft
Award Identifier / Grant number: HO1286/16-1 b
Acknowledgment
The authors would like to thank the Editor and reviewers for all suggestions which lead to considerable improvements of the paper. This study was supported by Bualuang ASEAN Chair Professor Fund.
-
Author contributions: All the authors have accepted responsibility for the entire content of this submitted manuscript and approved submission.
-
Research funding: This work was supported under Grant HO1286/16-1 by the German Research Foundation (DFG) and Bualuang ASEAN Chair Professor Fund, Agreement Number TUBC 02/2022.
-
Conflict of interest statement: The authors declare no conflicts of interest regarding this article.
References
1. Sutton, AJ, Higgins, JPT. Recent developments in meta-analysis. Stat Med 2008;27:625–50. https://doi.org/10.1002/sim.2934.Search in Google Scholar PubMed
2. Borenstein, M, Hedges, LV, Higgins, JP, Rothstein, HR. Introduction to meta-analysis. Chichester: John Wiley & Sons; 2009.10.1002/9780470743386Search in Google Scholar
3. Schulze, R, Holling, H, Böhning, D. Meta-analysis: new developments and applications in medical and social sciences. Göttingen: Hogrefe & Huber; 2003.Search in Google Scholar
4. Seide, SE, Röver, C, Friede, T. Likelihood-based random-effects meta-analysis with few studies: empirical and simulation studies. BMC Med Res Methodol 2019;19:1–14. https://doi.org/10.1186/s12874-018-0618-3.Search in Google Scholar PubMed PubMed Central
5. Stijnen, T, Hamza, TH, Ozdemir, P. Random effects meta-analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data. Stat Med 2010;29:3046–67. https://doi.org/10.1002/sim.4040.Search in Google Scholar PubMed
6. Böhning, D, Mylona, K, Kimber, A. Meta-analysis of clinical trials with rare events. Biom J 2015;57:633–48. https://doi.org/10.1002/bimj.201400184.Search in Google Scholar PubMed
7. Bhaumik, DK, Amatya, A, Normand, ST, Greenhouse, J, Kaizar, E, Neelon, B, et al.. Meta-analysis of rare binary adverse event data. J Am Stat Assoc 2012;107:555–67. https://doi.org/10.1080/01621459.2012.664484.Search in Google Scholar PubMed PubMed Central
8. Gart, JJ, Pettigrew, HM, Thomas, DG. The effect of bias, variance estimation, skewness and kurtosis of the empirical logit on weighted least squares analyses. Biometrika 1985;72:179–90. https://doi.org/10.1093/biomet/72.1.179.Search in Google Scholar
9. Sweeting, MJ, Sutton, AJ, Lambert, PC. What to add to nothing? use and avoidance of continuity corrections in meta-analysis of sparse data. Stat Med 2004;23:1351–75. https://doi.org/10.1002/sim.1761.Search in Google Scholar PubMed
10. Bradburn, MJ, Deeks, JJ, Berlin, JA, Localio, AR. Much ado about nothing: a comparison of the performance of meta-analytical methods with rare events. Stat Med 2007;26:53–77. https://doi.org/10.1002/sim.2528.Search in Google Scholar PubMed
11. Cai, T, Parast, L, Ryan, L. Meta-analysis for rare events. Stat Med 2010;29:2078–89. https://doi.org/10.1002/sim.3964.Search in Google Scholar PubMed PubMed Central
12. Efthimiou, O. Practical guide to the meta-analysis of rare events. Evid Base Ment Health 2018;21:72–6. https://doi.org/10.1136/eb-2018-102911.Search in Google Scholar PubMed
13. Jackson, D, White, IR. When should meta-analysis avoid making hidden normality assumptions? Biom J 2018;60:1040–58. https://doi.org/10.1002/bimj.201800071.Search in Google Scholar PubMed PubMed Central
14. Jiang, T, Cao, B, Shan, G. Accurate confidence intervals for risk difference in meta-analysis with rare events. BMC Med Res Methodol 2020;20:1–10. https://doi.org/10.1186/s12874-020-00954-8.Search in Google Scholar PubMed PubMed Central
15. Keus, F, Wetterslev, J, Gluud, C, Gooszen, HG, van Laarhoven, CJHM. Robustness assessments are needed to reduce bias in meta-analyses that include zero-event randomized trials. Am J Gastroenterol 2009;104:546–51. https://doi.org/10.1038/ajg.2008.22.Search in Google Scholar PubMed
16. Kuss, O. Statistical methods for meta-analyses including information from studies without any events - add nothing to nothing and succeed nevertheless. Stat Med 2015;34:1097–116. https://doi.org/10.1002/sim.6383.Search in Google Scholar PubMed
17. Wan, X, Wang, W, Liu, J, Tong, T. Estimating the sample mean and standard deviation from the sample size, median, range and/or interquartile range. BMC Med Res Methodol 2014;14:1–13. https://doi.org/10.1186/1471-2288-14-135.Search in Google Scholar PubMed PubMed Central
18. Liu, D, Liu, RY, Xie, M. Exact meta-analysis approach for discrete data and its application to 2 × 2 tables with rare events. Am Stat Assoc 2014;109:1450–65. https://doi.org/10.1080/01621459.2014.946318.Search in Google Scholar PubMed PubMed Central
19. Wang, X, Liu, C, Dorris, K, Fouladi, M, Song, S. Random-effects meta-analysis for systematic reviews of phase I clinical trials: rare events and missing data. Res Synth Methods 2017;8:124–35. https://doi.org/10.1002/jrsm.1209.Search in Google Scholar PubMed PubMed Central
20. Yang, G, Liu, D, Wang, J, Xie, M. Meta-analysis framework for exact inferences with application to the analysis of rare events. Biometrics 2016;72:1378–86. https://doi.org/10.1111/biom.12497.Search in Google Scholar PubMed
21. Piaget-Rossel, R, Taffé, P. Meta-analysis of rare events under the assumption of a homogeneous treatment effect. Biom J 2019;61:1557–74. https://doi.org/10.1002/bimj.201800381.Search in Google Scholar PubMed
22. Beisemann, M, Doebler, P, Holling, H. Comparison of random-effects meta-analysis models for the relative risk in the case of rare events: a simulation study. Biom J 2020;62:1597–630. https://doi.org/10.1002/bimj.201900379.Search in Google Scholar PubMed
23. Günhan, BK, Röver, C, Friede, C. Random-effects meta-analysis of few studies involving rare events. Res Synth Methods 2020;11:74–90. https://doi.org/10.1002/jrsm.1370.Search in Google Scholar PubMed
24. Böhning, D, Holling, H, Böhning, W, Sangnawakij, P. Investigating heterogeneity in meta-analysis of studies with rare events. Metron 2021;79:259–72. https://doi.org/10.1007/s40300-021-00211-y.Search in Google Scholar
25. Sangnawakij, P, Böhning, D, Holling, H. On the exact null-distribution of a test for homogeneity of the risk ratio in meta-analysis of studies with rare events. J Stat Comput Simulat 2021;91:420–34. https://doi.org/10.1080/00949655.2020.1815200.Search in Google Scholar
26. Holling, H, Jansen, K, Böhning, W, Böhning, D, Martin, S, Sangnawakij, P. Estimation of effect heterogeneity in rare events meta-analysis. Psychometrika 2022;87:1–22.10.1007/s11336-021-09835-5Search in Google Scholar PubMed PubMed Central
27. Mantel, N, Haenszel, W. Statistical aspects of the analysis of data from retrospective studies of disease. J Natl Cancer Inst 1959;22:719–48.Search in Google Scholar
28. Cochran, WG. The combination of estimates from different experiments. Biometrics 1954;10:101–29. https://doi.org/10.2307/3001666.Search in Google Scholar
29. Rothman, KJ, Greenland, S, Lash, TL. Modern epidemiology. Philadelphia: Lippincott Williams & Wilkins; 2008.Search in Google Scholar
30. Casella, G, Berger, RL. Statistical inference. Duxbury: Pacific Grove; 2002.Search in Google Scholar
31. Fieller, EC. The biological standardization of Insulin. J R Stat Soc Series B 1940;7:1–64. https://doi.org/10.2307/2983630.Search in Google Scholar
32. R Core Team. R: a language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing; 2020.Search in Google Scholar
33. Böhning, D, Sangnawakij, P. The identity of two meta-analytic likelihoods and the ignorability of double-zero studies. Biostatistics 2021;22:890–6. https://doi.org/10.1093/biostatistics/kxaa004.Search in Google Scholar PubMed
34. Efron, B. Bootstrap methods: another look at the jackknife. Ann Stat 1979;7:1–26. https://doi.org/10.1214/aos/1176344552.Search in Google Scholar
35. Crowley, P. Interventions for preventing or improving the outcome of delivery at or beyond term. Cochrane Database Syst Rev 2000;2:CD000170. https://doi.org/10.1002/14651858.CD000170.Search in Google Scholar PubMed
36. Nissen, SE, Wolski, K. Rosiglitazone revisited: an updated meta-analysis of risk for myocardial infarction and cardiovascular mortality. Arch Intern Med 2010;170:1191–201. https://doi.org/10.1001/archinternmed.2010.207.Search in Google Scholar PubMed
37. Chu, DK, Akl, EA, Duda, S, Solo, K, Yaacoub, S, Schünemann, HJ. Physical distancing, face masks, and eye protection to prevent person-to-person transmission of SARS-CoV-2 and COVID-19: a systematic review and meta-analysis. Lancet 2020;395:1973–87. https://doi.org/10.1016/s0140-6736(20)31142-9.Search in Google Scholar
Supplementary Material
The online version of this article offers supplementary material (https://doi.org/10.1515/ijb-2021-0087).
© 2022 Walter de Gruyter GmbH, Berlin/Boston
