| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Stroke. 2006;37:123.)
© 2006 American Heart Association, Inc.
Original Contributions |
From the University of California, San Francisco (J.S.E., T.K., J.R., S.C.J.); Kaiser Permanente San Francisco, California (L.F.).
Correspondence to S. Claiborne Johnston, MD, PhD, UCSF Department of Neurology, Box 0114, 505 Parnassus Ave., M-798, San Francisco, CA 94143. E-mail Clay.Johnston{at}ucsfmedctr.org
| Abstract |
|---|
|
|
|---|
Methods We performed a meta-analysis of all randomized, controlled trials of
300 subjects that were designed to evaluate the efficacy of a medical intervention for the treatment of acute ischemic stroke. Data about trial recruitment, organization, and inclusion/exclusion criteria were abstracted independently by 2 reviewers who applied predefined criteria. Recruitment efficiency was defined as the number of subjects enrolled per study center per month of recruitment.
Results Of 32 trials meeting inclusion criteria, the average recruitment efficiency was 0.79 subjects per center per month (range 0.08 to 3.7). Recruitment efficiency did not vary by geographic region (P=0.36), but trials conducted in 1 country had more efficient recruitment than international studies (P=0.03), and recruitment efficiency declined with each percentage increase in the total number of study centers (P=0.002). The primary study entry criteria that predicted reduced recruitment efficiency were the maximum allowable time from stroke to study enrollment (P=0.002) and the exclusion of mild strokes (P=0.009). Trials with a treatment window >6 hours had approximately double the recruitment rates of trials that used treatment windows
6 hours (1.03 versus 0.52 patients per center per month).
Conclusion Recruitment rates for acute stroke trials are influenced by organizational structure and study entry criteria. Characterizing predictors of recruitment may help optimize future trial design.
Key Words: randomized controlled trials stroke, acute
| Introduction |
|---|
|
|
|---|
Trial feasibility and costs are impacted by 2 crucial factors: the number of subjects who need to be enrolled and the duration of time that it will take to enroll them. Although sample size requirements are generally fixed by statistical calculations, the duration of acute stroke trials depends primarily on sample size, the number of centers involved in recruitment, and the efficiency with which those centers recruit participants. Although the trial investigator can influence trial duration by determining the number of sites and the stringency of inclusion criteria, the impact of these variables on recruitment rates is unknown.
We hypothesized that trial organization and entry criteria would impact the efficiency of recruitment as defined by the number of participants enrolled per study center per month of recruitment. We tested this hypothesis in a meta-analysis examining predictors of efficient recruitment in large acute stroke trials completed during the last 15 years.
| Methods |
|---|
|
|
|---|
Two abstractors independently applied inclusion and exclusion criteria to identified studies and abstracted predefined data elements. A third reviewer adjudicated disagreements. Included studies were randomized-controlled trials with >300 participants in which the primary objective was to evaluate the efficacy of a medical intervention for the treatment of acute ischemic stroke. When trials did not report entry criteria, the duration of the enrollment period, or the number of study centers that participated in recruitment, we contacted the study author to obtain this information directly. In addition to the recruitment data, abstractors recorded the following information about a trial: (1) medication type (eg, thrombolytic, anticoagulant, or neuroprotectant), (2) the route of drug administration, (3) the countries where the study centers were located, and (4) the specific inclusion and exclusion criteria of the trial.
Statistical Methods
The primary outcome variable for analyses was an aggregate measure of recruitment efficiency defined as the number of participants enrolled per study center per month of recruitment. We considered the following variables as potential predictors of this outcome: number of centers used in the trial, geography of centers (North American only versus European only versus other countries only versus multiple regions), drug class (antithrombotic, thrombolytic, or neuroprotectant), route of drug administration (intravenous only versus oral or subcutaneous), maximum time from stroke to study enrollment, and trial year. We also used the California Acute Stroke Pilot Registry (CASPR), a large cohort study of acute stroke in California hospitals,6 to create an inclusiveness index defined as the percentage of acute stroke patients who would have been eligible for the trial based on the following trial criteria: minimum and maximum allowed age, systolic and diastolic blood pressure entry criteria, maximum allowable time from stroke to enrollment, whether minor strokes were excluded (estimated by National Institutes of Health Stroke Scale
5), and whether the study required the presence of atrial fibrillation.
Changes in trial recruitment parameters over time were evaluated by the Spearman rank correlation coefficient. Potential predictors of recruitment efficiency were evaluated using linear regression. Because the relationship between recruitment efficiency and the total number of study centers was nonlinear, we used a log transformation of study centers in the regression models, and therefore the estimated coefficient is interpreted as the average change in recruitment efficiency for each percentage change in the number of study centers.7 Although the number of centers contributes to the calculation of recruitment efficiency, we included centers as a predictor in the multivariable models to control for confounding if trials with restrictive entry criteria used high numbers of centers to meet enrollment goals. To limit colinearity between predictors in the multivariable models, we selected the following variables a priori for the initial model: number of centers, recruitment geography, treatment class (thrombolytic versus anticoagulant or neuroprotectant), and inclusiveness index. The final model was determined using backward, stepwise elimination to select variables that remained associated with the outcome at P<0.2. Using the multivariable model results, we calculated an expected recruitment for each study and used these results to identify studies that achieved recruitment efficiency above that predicted by our model. For the purposes of these analyses, it was assumed that the trials that were identified were a random sample of the total population of trials that could theoretically have been performed during the study period. All P values were based on 2-sided hypothesis tests. Statistical analyses were performed using STATA (version 8).
| Results |
|---|
|
|
|---|
|
When trial recruitment parameters were analyzed over time, the total size of trials (rs=0.22; P=0.23) and the duration of the enrollment period (rs=0.13; P=0.46) appeared to remain fairly constant during the study period. The number of study centers used for recruitment, however, increased over time (rs=0.44; P=0.01), and the recruitment efficiency of trials showed a trend toward decline over time (rs=0.30; P=0.096; Figure).
|
The regional base for trial recruitment did not appear to have a major impact on recruitment efficiency. Although trials (n=4) that recruited subjects exclusively in non-North American and non-European countries had the highest recruitment efficiency (1.36 subjects per center per month) and North American trials had the lowest efficiency (0.57 subjects per center per month), all differences by region could be explained by chance (P=0.36; Table 2). Furthermore, there was no linear relationship between recruitment efficiency and the total number of countries where subjects were enrolled (P=0.64). However, trials in which recruitment occurred in 1 county (n=14) had more efficient recruitment than trials that recruited subjects in multiple countries (1.13 versus 0.54 subjects per center per month; P=0.03). Additionally, recruitment efficiency declined with each percentage increase in the number of centers used in a study (P=0.002); trials that used a total number of centers above the median recruited subjects at 0.54 subjects per center per month compared with 1.05 subjects per center per month in trials that used below the median number of centers.
|
Several variables hypothesized to correlate with the stringency of trial entry criteria or the complexities of the intervention were associated with differences in recruitment efficiency (Table 2). Trials of thrombolytics (average recruitment 0.31 subjects per center per month) were less efficient in recruitment than trials of neuroprotectants and anticoagulants (combined average recruitment 0.95 subjects per center per month; P=0.04). There was no significant difference in recruitment rates between trials of anticoagulants (n=7) and neuroprotectants (n=17; P=0.29). Trials of medications that were administered orally or subcutaneously (n=12) had greater recruitment efficiency than trials of intravenously administered medications (1.26 versus 0.51 subjects per center per month; P=0.006). After adjustment for medication class, a trend toward increased recruitment efficiency was still present for oral and subcutaneous medications (P=0.08).
When inclusion and exclusion criteria were analyzed using CASPR registry data, between 6% and 69% of all acute ischemic stroke admissions would have been eligible for the trials considered in this analysis. Trials whose entry criteria placed them above the median inclusiveness recruited more efficiently than trials below the median (1.06 versus 0.52 subjects per center per month; P=0.047). When the entry criteria were considered individually, there was a highly significant relationship between recruitment efficiency and maximum allowed time from symptom onset to enrollment (P=0.002). Trials that allowed subjects to be enrolled up to 48 hours after stroke (n=9) achieved recruitment rates that on average were
3 times the rate of trials that required subject enrollment within 6 hours from stroke (n=15; 1.47 versus 0.52 subjects per center per month; P=0.007). Additionally, trials that excluded mild strokes (n=23) had lower recruitment efficiency than trials that did not specifically exclude subjects with mild symptoms (n=9; 0.57 versus 1.35; P=0.009).
In the multivariable analysis, 2 variables remained associated with recruitment efficiency in the final model: the number of trial centers (log transformed; P<0.001) and the summary measure of trial inclusiveness (P=0.01; model adjusted R2=0.39). We then calculated, for each trial, the expected recruitment efficiency based on these 2 parameters. Trials that achieved recruitment efficiency that was most above the predicted value based on the multivariable model are listed in Table 3. The high recruitment rates in these trials could not be explained by our estimate of the inclusiveness of their entry criteria or the number of recruitment centers used in the trial.
|
| Discussion |
|---|
|
|
|---|
40% of the variability in subject recruitment efficiency across the included trials. Although the magnitude of these associations makes them relevant to trial design, other factors, potentially related to site motivation and competency, are also likely to be critical in determining the efficiency with which randomized stroke trials are executed. Identifying these other factors may be crucial in accelerating future trials. The 2 primary predictors of recruitment efficiency in the multivariable analysis (inclusiveness of trial entry criteria and the number of study centers) have different potential implications for optimizing trial design. Given the fundamental importance of time to treatment in demonstrating the benefits of tissue plasminogen activator in acute ischemic stroke treatment, it is unlikely that trial investigators will change their target treatment window simply to improve recruitment. However, full knowledge of the impact of this parameter on recruitment rates is important in determining the expected duration of the trial and in choosing the number of centers necessary to complete it in a timely fashion. Given recent evidence documenting benefits of thrombolysis in subjects initially considered to have minor strokes,40 the inclusion of such subjects in stroke trials may enhance recruitment without a major threat to the power of a study to demonstrate efficacy. The greater recruitment efficiency observed in smaller studies may have been attributable to more careful selection of study centers or more personal investment on the part of the center investigators. Further study, particularly of center-specific recruitment results, is needed to identify the characteristics of high-performing centers and to guide center selection in future trials.
Trial experts recently advocated for performing large, simple randomized trials without complex measurements of baseline variables.41,42 The 2 largest trials included in this analysis, International Stroke Trial (IST) and Chinese Acute Stroke Trial (CAST), used this approach, and inclusion and exclusion decisions apart from time to treatment were left almost entirely to the discretion of the enrolling physician. Most other trials in this analysis enumerated numerous inclusion and exclusion criteria, most of which had no appreciable impact on our summary measure of trial inclusiveness but may have added to difficulty in enrolling patients. IST and CAST achieved recruitment efficiency that was above that predicted by our model, suggesting that their broadly inclusive trial entry criteria may provide additional benefits to recruitment efficiency beyond that which can be explained by the predictors considered in this analysis. Furthermore, such an approach is likely to improve the generalizability of the study findings.
The most reliable variable abstracted for this analysis that was specifically related to subject inclusion and exclusion was the maximum allowable time to treatment. All trials reported this variable, and it is likely that the criterion was applied reasonably uniformly across sites. Our ability to generalize about other specific inclusion and exclusion criteria is limited both by the sample size of our analysis, variable definitions used across studies, and the possibility that not all inclusion and exclusion criteria were reported in trial publications. Because only 1 trial included in this analysis required MRI imaging in all participants, we could not assess the impact on recruitment rates of using more demanding ancillary tests when compared with standard head computed tomography. Our analysis did not adjust for the effects of competing trials and assumed that all study centers were active from the beginning of the trial, 2 factors that would likely result in an underestimation of actual recruitment efficiency but that would not be expected to bias analyses of individual predictors. Furthermore, our estimates of recruitment efficiency are not generalizeable to small trials in which recruitment may be more easily coordinated than in the large, multicenter trials considered here.
Our results support the idea that trial design is a discipline amenable to guidance from scientific analysis as opposed simply to tradition and expert opinion. Given the high societal costs associated with the performance, and potentially the lack of performance, of well-designed randomized stroke trials, additional work in this area is needed to improve future trials and minimize tradeoffs between trial feasibility and scientific validity.
Received July 14, 2005; accepted August 3, 2005.
| References |
|---|
|
|
|---|
This article has been cited by other articles:
![]() |
S. T. Crowley, G. M. Chertow, J. Vitale, T. O'Connor, J. Zhang, R. M.H. Schein, D. Choudhury, K. Finkel, A. Vijayan, E. Paganini, et al. Lessons for Successful Study Enrollment from the Veterans Affairs/National Institutes of Health Acute Renal Failure Trial Network Study Clin. J. Am. Soc. Nephrol., July 1, 2008; 3(4): 955 - 961. [Abstract] [Full Text] [PDF] |
||||
![]() |
S. Blanton, D. M Morris, M. G Prettyman, K. McCulloch, S. Redmond, K. E Light, and S. L Wolf Lessons Learned in Participant Recruitment and Retention: The EXCITE Trial Physical Therapy, November 1, 2006; 86(11): 1520 - 1533. [Abstract] [Full Text] [PDF] |
||||
| |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Stroke Home | Subscriptions | Archives | Feedback | Authors | Help | AHA Journals Home | Search Copyright © 2006 American Heart Association, Inc. All rights reserved. Unauthorized use prohibited. |