Abstract
Background and Purpose—Knowing characteristic adverse events (AEs) and their incidence among patients participating in acute stroke trials may assist interpretation of future studies. We aimed to develop an online tool to inform stroke trial safety.
Methods—We identified relevant AEs from patients within the Virtual International Stroke Trials Archive (VISTA), using receiver operating characteristic principles. We modeled their incidence on patient age, baseline National Institutes of Health Stroke Scale, and comorbidities using binary logistic regression. Models with an R2 >5% were deemed powerful enough to predict expected AE incidences and were included. The calculator was developed using programs R and Visual Studios.
Results—Forty-eight of the most common AEs were identified and incorporated into the IschAEmic Stroke Calculator. The calculator, publicly available at http://www.vistacollaboration.org calculates the expected incidence of AEs or groups of AEs in a trial cohort and where possible compares them with the observed incidence.
Conclusions—The IschAEmic Stroke Calculator is an open access resource to support safety interpretation within acute stroke trials. Prediction of AEs with higher likelihood of occurrence may direct preventive clinical measures.
Introduction
Adverse events (AE) are a known burden to researchers and patients.1 Their reporting can lead to the inflation of trial costs and, importantly, a lower sensitivity for genuine serious unexpected treatment-related AEs.1 Second, these AEs have an impact on patient outcomes.1
Various studies demonstrate that these AEs are predictable.1–3 This information could be useful to trials, as it can identify at-risk patients and perhaps prevent the AE from occurring. However, there is still no tool available to date.
We aimed to develop a resource to calculate the likely incidence of important and common AEs in acute ischemic stroke trials based on the trial population’s characteristics. It is hoped that this resource can objectively evaluate a trial’s safety.
Methods
Data Source
We sought data from patients with an acute ischemic stroke, who had participated in the placebo-arm of randomized clinical trials and whose information was stored in the Virtual International Stroke Trials Archive (VISTA). Separate ethical approval was not needed because anonymized historical patient data were used.
Variables of interest were patient age, sex, body mass index, baseline glucose (mmol/L), blood pressure (mm Hg), estimated glomerular filtration rate (mL/min/1.73 m2), smoking status, National Institutes of Health Stroke Scale, hemispheric lateralization, thrombolysis treatment, and comorbidities. We also collected AEs, coded using the Medical Dictionary for Regulatory Activities system.
Statistical Analysis
Identification of Common AEs
We used a receiver operating characteristic analysis to define a concise, yet inclusive list of AEs. This was examined by plotting the cumulative proportion of AE occurrences versus the proportion of types of AEs included, with the point on the curve closest to (0,1) defining our cutoff. In a further step, we used factor analysis to group correlated AEs.
Modeling of AEs
We used backward stepwise binary logistic regression to model the incidence of the AEs and groups of AEs as a function of the previously mentioned variables. Models with an R2 >5% were deemed powerful enough to predict expected AE incidences and defined the final list of AEs to be incorporated in the online tool.
Statistical analysis was performed using SPSS v21.0 (IBM Corp., Armonk, NY).
Development of the IschAEmic Stroke Calculator
The IschAEmic Stroke Calculator has 2 aims. It should compare the baseline demographics of a user’s trial-specific population with that of a pooled population from VISTA. It should also calculate the expected incidence of selected AEs and groups of AEs and compare it with the observed incidence. We used R v3.1.0 (R Foundation for Statistical Computing, Vienna, Austria) and Visual Studios v2012 (Microsoft Corp., Seattle, WA).
We produced the background algorithm with R. The code uses a 2-sample t test for numeric data and a 2-sample test for proportions for categorical data to compare the baseline demographics. It then uses transformed versions of the logistic regression equations to calculate the expected incidence of AEs and uses a 2-sample t test to compare it with the observed incidence. Results are expressed as 95% confidence intervals and P values with statistical significance assessed at the conventional P<0.05.
Visual Studios was used to develop the graphical user interface. It queries the user’s input to the database, held in the R code before presenting the results. Our aims were to develop a user-friendly input window and a clear output window with results in tables and graphs.
Results
Data from 5775 placebo-treated patients were available for analysis. Their mean age was 69.3±12.3 years, and 53.8% were male. The median National Institutes of Health Stroke Scale was 13 (interquartile range, 9–18).
Important AEs and Groups of AEs
Our receiver operating characteristic analysis showed that the first 133 AEs with the highest incidences accounted for 82.7% of all occurrences, defining our list of common AEs (Figure 1). Factor analysis identified 19 groups of strongly correlated AEs from that list (Table I in the online-only Data Supplement). The largest group had 6 and the smallest groups had 2 AEs.
Receiver operating characteristic analysis to identify the common AEs to be used in the IschAEmic Stroke Calculator. The secondary curve identified the point on the first curve closest to the point (0,1). It showed that the 133 types of AEs with the highest incidence were responsible for 82.7% of all occurrences. AE indicates adverse event.
Final List of AEs and Their Models
We included 2199 patients with complete prognostic factor data to develop the AE functions. Of the 133 models for the individual AEs, 48 had an R2 >5% and were used in the online tool (Table II in the online-only Data Supplement). Subsequently, we had 67 functions to calculate the incidence of 48 individual common AEs and 19 groups of AEs.
IschAEmic Stroke Calculator
The online tool uses a digital interface to run the background R code. It is publically available at www.vistacollaboration.org.
When the user opens the resource they are shown a window with options to input summarized patient demographic data and observed AE incidences (Figure 2). Inputs include the mean and standard deviation of numeric variables and the proportion of patients who fall into a specific category for categorical variables.
The input window of the IschAEmic Stroke Calculator showing a various options to insert variables. A, Basic clinical information panel. B, Observed adverse event incidences panel.
By pressing the button Run Analysis the calculator queries the user’s data to the R code. To compare baseline demographics, the user’s data are substituted into the appropriate function (eg, 2-sample t tests) that already stores the corresponding VISTA value. To calculate the expected incidence of AEs, the R code substitutes the means and proportions of predictive variables into the transformed regression equations, iterating through the 48 AEs and 19 groups of AEs.
Results are displayed in both tabular and graphical formats, divided into numeric and categorical results for baseline demographic and system classes for AE incidence comparisons (Figure 3). Significant differences are highlighted in red and results can also be saved where required.
The output windows of the IschAEmic Stroke Calculator, with icons for tabular and graphical results of both baseline demographic and adverse event incidence comparisons. A and B, The comparison of baseline demographics between a mock user’s trial population and that from a pooled population from VISTA. C and D, The expected and observed incidences of cardiovascular adverse events in a mock user’s trial.
Discussion
We developed the IschAEmic Stroke Calculator as an open access resource to evaluate a stroke trial’s safety profile. The calculator compares the baseline demographics of the user’s trial population with that of a pooled population from VISTA. Significant differences may indicate potential bias in their patient selection. It then calculates the expected incidence of AEs or groups of AEs in the trial cohort. AEs with an unusually high incidence can be targeted with measures to reduce their likelihood of occurrence.
The strength of the study includes the high quality of the data, ensured through the strict entry criteria needed to be fulfilled by participating trials in VISTA.4 Additionally, the resource is open access. However, the highest R2=14.3%, suggesting that there is still a lot of variability that cannot be explained by the models. Furthermore, the tool can only be applied to trial cohorts, not individual patients. This may be the most reasonable and effective way to assess the safety profile of large multicenter stroke trials.
Conclusions
The IschAEmic Stroke Calculator is an open access platform to evaluate the safety profile of acute ischemic stroke trials. Its use can identify high-risk trial populations that may benefit from additional treatment measures.
Appendix
VISTA-Acute Steering Committee K.R. Lees (Chair), A. Alexandrov, P.M. Bath, E. Bluhmki, N. Bornstein, L. Claesson, S.M. Davis, G. Donnan, H.C. Diener, M. Fisher, B. Gregson, J. Grotta, W. Hacke, M.G. Hennerici, M. Hommel, M. Kaste, P. Lyden, J. Marler, K. Muir, R. Sacco, A. Shuaib, P. Teal, N.G. Wahlgren, S. Warach, and C. Weimar.
Acknowledgments
We would like to thank all VISTA collaborators. K.R.L. conceived and supervised the project. K.H. performed the analyses and drafted the initial manuscript. R.L.M. provided guidance in the use of the programming software. K.H., A.H.A.-R., P.D.L., E.B., and K.R.L. were involved in reviewing and reporting of the work. All authors approved the final version. Members of the VISTA-Acute Steering Committee approved the study plan in advance and the final manuscript.
Disclosures
K.R.L. discloses an institutional conflict of interest whereby the University of Glasgow holds grants for clinical trials from the National Institutes of Health, European Union, and commercial organizations.
Footnotes
* A list of all the VISTA Collaborators is given in the Appendix.
Guest Editor for this article was Natan M. Bornstein, MD.
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.115.011930/-/DC1.
- Received October 22, 2015.
- Accepted November 18, 2015.
- © 2016 American Heart Association, Inc.
Jump to
This Issue
Article Tools
Share this Article
- Online Tool to Improve Stratification of Adverse Events in Stroke Clinical Trials
