Methodological Quality and Publication Bias in Observational Studies on Risk of Rupture of Unruptured Intracranial Aneurysms
To the Editor:
We would like to comment on some aspects of the very interesting updated meta-analysis on risk of rupture of unruptured intracranial aneurysms by Wermer and colleagues.1
First, with regard to the assessment of methodological quality of the included studies, the authors have rated studies as “good” when they fulfilled 3 criteria with regard to design, completeness of follow-up and certainty of diagnosis of subarachnoid hemorrhage. Several other studies have stressed that accounting for confounders such as size and location of the aneurysm and inclusion of intracavernous aneurysms with regard to rupture risk is also an important quality criterion,2,3 not used, however, by Wermer et al. The authors found no significant difference in rupture risk between studies with limited versus high quality (high quality studies had a relative rupture risk of 0.8 with a 95% CI of 0.6 to 1.1 compared with limited quality studies). However, the extent of quality assessment of the included studies may have influenced the results of this meta-analysis.
Second, the authors did not mention or study the influence of publication bias on their results. We made a funnel plot (Figure) based on the numbers of subarachnoid hemorrhage cases and patient-years-at-risk provided by the authors in their Table 1. This plot shows an asymmetrical distribution of rupture rates, with larger rupture rates in studies including less patient-years-at-risk. The most likely cause of this overrepresentation of small studies with high rupture rates is publication bias. One other possible explanation is incomplete follow-up of patients without rupture, and patients with aneurysm rupture coming to notice automatically because of their clinical condition. Based on the data provided by Wermer and colleagues, it is unclear whether the funnel plot asymmetry is caused by publication bias or by this overestimation of rupture rate in smaller studies of limited quality.4
With their updated review, Wermer et al have made an important contribution by providing essential data for clinical decision-making in patients with unruptured intracranial aneurysms. However, the potential influence of biased data introduced by either publication bias or heterogeneity in methodological quality deserves further study.
Wermer MJH, Van der Schaaf IC, Algra A, Rinkel GJE. Risk of rupture of unruptured intracranial aneurysms in relation to patient and aneurysm characteristics: an updated meta-analysis. Stroke. 2007; 38: 1404–1410.
Tooth L, Ware R, Bain C, Purdie DM, Dobson A. Quality of reporting of observational longitudinal research. Am J Epidemiol. 2005; 161: 280–288.
Sterne JAC, Egger M, Smith GD. Investigating and dealing with publication and other biases in meta-analysis. BMJ. 2001; 323: 101–105.