Early Versus Late Assessment of Stroke Outcome
What is the best timing for outcome assessment in patients with severe stroke in randomized controlled trials; early assessment at 3 months versus later at 6 to 12 months?
Early Assessment Is Better
Kennedy R. Lees
The purpose of a clinical trial is usually to assess whether and to what extent a treatment improves outcome and to identify the type and frequency of any associated risks. The question that is posed here refers only to acute trials, in which enrollment occurs within hours after stroke onset and treatment is exhibited and probably completed within at most a few days. We can reasonably assume that the treatment exerts its beneficial effect only while it is being administered and that any adverse effects are similarly acute in onset.
Patients with acute stroke, and especially patients with severe stroke, are typically elderly and have elevated risk of cardiovascular disease; indeed, stroke survivors are more likely to die from cardiovascular disease than recurrent stroke. Long-term follow-up will dilute effects of treatment with other events that are part of the natural history of the condition. Any benefit may be revealed reasonably quickly, but with extended follow-up, there is an inevitable convergence of outcomes between treatment groups as complications of age, and associated risk factors come into play. We must, therefore, look at the natural history of outcomes after stroke, especially among patients with initially severe stroke, to see when recovery typically reaches its plateau at the group level.
Data from the Virtual International Stroke Trials Archive, 2015 (Dr Rachael MacIsaac, personal communication) on the distribution of outcomes across 1 year assessed by modified Rankin Scale (mRS) among 159 ischemic stroke patients with admission National Institutes of Health Stroke Scale score of ≥20 indicate that the proportions of patients with good outcomes—mRS scores of 0, 1, 2, 3, or 4—each rises between 1 week and 3 months after stroke but falls progressively through 6 months to 1 year. The proportion of patients who are bedbound falls abruptly between 1 week and 3 months but drops by only 2% to 3% during the next 9 months. Twenty percent of severely affected patients die between 1 week and 3 months after stroke, and thereafter, ≈1.5% die each month. Thus, the recovery that we can expect to detect on a population basis has been completed by 3 months. Although some individual patients may continue to improve thereafter, their numbers are more than offset by deaths and deteriorations among others.
Clinical trials are analyzed on the basis of group effects. For a clinical trial that uses mRS distribution or any usual dichotomization or variant of mRS as its primary outcome, there is no value in extending following up beyond 3 months. Where the value arises from an extension of follow-up is in demonstrating durability of effect and in assessing health economics. If a treatment limits the proportion of patients who have severe disability and if we assume that late deaths after stroke are mainly concentrated among the most disabled survivors, then it may allow a greater survival benefit to be detected through extended follow-up. This has limited value, when there is already general acceptance that bedbound survival should be considered an equally unattractive outcome to death1: we already know the effect of our treatment by 3 months. It may be preferable to increase the sample size to improve trial power than to rely on extended follow-up to rescue an underpowered trial. Long-term effects can readily be collected as a secondary, descriptive measure.
Taking 2 examples where patients with severe stroke have been selectively enrolled, the hemicraniectomy trials and the thrombectomy trials, neither needed extended follow-up to demonstrate its benefit. Although the final distribution of functional outcomes changes slightly between 6 months and 1 year in the hemicraniectomy trials, the survival benefit was not only fully evident at 6 months; the Kaplan–Meier curve shows that it was evident within the first days.2 Recent thrombectomy trials have enrolled patients with severe stroke and had no difficulty in demonstrating benefit by 3 months; indeed, benefit was evident even with early assessment.3,4 Risks, limited as they were, are concentrated entirely within the first hours.
Although there is no advantage to extending follow-up for the primary outcome, there may be costs associated with this. First, additional visits place a burden on patients, carers, and staff. Second, losses to follow-up will increase. Third, the accuracy with which assessments are recorded may diminish because of familiarity and prejudice from earlier findings. Fourth, extension of a study by ≥3 months extends the period over which the whole-study team must be funded, generating a substantial financial penalty. Fifth, extension of follow-up delays the readout of the trial result, postponing announcement of benefit (or harm) and depriving all patients of optimal therapy for the same period: this is unethical.
Further arguments favor retaining 3 months as the primary end point for trials in patients with severe stroke.5 A 3-month outcome has been applied in numerous trials to date involving a diverse range of treatment approaches. It facilitates comparisons among these options and assessment of potential combination effects if a common end point is available.6,7 It would be confusing if trials of mild stroke were to use an early outcome, moderate stroke an intermediate outcome, and severe stroke a late outcome.
However, the choice of best end point should not be based on stroke severity but rather on the nature of the intervention and its biological effect. Reperfusion strategies, hemorrhage removal, decompression, neuroprotection, and stroke unit care each has a short-term intervention period and largely immediate effect. Only for treatment approaches that carry a prevention element later assessment would be preferable.
Late Assessment Is Better
Joseph P. Broderick
The scientific rationale for timing of end points depends on what you want to measure and how you plan to use the information to decide on the efficacy, safety, and cost-effectiveness of a therapy. For a phase I or II safety trial of acute stroke, your primary safety end point may be during hospitalization or within the first month after an administered therapy. For a stroke prevention trial in which end points may be rare in the first weeks, outcomes are collected over years to demonstrate differences between treatments. For acute treatment trials of ischemic and hemorrhagic stroke, the timing for the outcome should ideally both maximize the observation of a potential benefit between the tested therapy and the standard of care and insure the sustainability of this benefit and its cost-effectiveness.
Severely affected patients often require a longer time to improve on measures of disability, such as the mRS score, and observed differences in outcome between treatment groups may not be maximal at 3 months. Two recent trials of severe ischemic and hemorrhagic strokes provide examples. In the moderately severe stroke cohort (National Institutes of Health Stroke Scale score, 8–19) of (Interventional Management of Stroke) IMS III,8 there was no benefit for endovascular treatment at any time point during 12 months (2% difference in outcomes in mRS scores of 0–2). Both groups in this cohort not only had the greatest improvement in the mRS score during the first 3 months but also had continued smaller improvement in the distribution of the mRS score during 12 months. In contrast, in the severe stroke cohort (National Institutes of Health Stroke Scale score, ≥20), the proportion of participants with an mRS score 0 to 2 at 3 months treated with endovascular therapy was 7% greater than those treated with intravenous tissue-type plasminogen activator alone (nonsignificant) but increased to 14% greater at 6 months, 15% at 9 months, and 14% at 12 months (P<0.05). Analysis of the distribution of the mRS score for the 2 treatment groups also reflected the increasing separation in outcomes between the 2 treatment groups after 3 months.
The Minimally Invasive Surgery Plus rt-PA for ICH Evacuation Phase II (MISTIE II) trial, a randomized minimally invasive surgery trial of intracerebral hemorrhage, also found that the differences in outcome between the active and standard therapy arms became greater during the course of 12 months when compared with traditional 3-month outcomes.9,10 As a result, the design of the ongoing phase III MISTIE III trial has the primary end point at 6 months. Substantial recovery beyond 3 months has been noted in patients with poor grade subarachnoid hemorrhage11 and with major upper extremity deficits from ischemic stroke.12
Although primary outcome measures at 6 to 12 months may provide greater ability to detect treatment differences in patients with severe stroke, another excellent reason to use a later time point is demonstration of sustainability. The direct effects of a therapy may become less detectable as new medical illnesses and life events affect the life of a study participant the further one gets from the index stroke. However, demonstrating a persistent benefit for >3 months is important on judging the effect of a therapy over time in the population and is key for accurate cost-effectiveness analyses. The publication of the sustained benefit of tissue-type plasminogen activator at 12 months by the National Institute of Neurological Disorders and Stroke (NINDS) trial investigators provided strong evidence of the sustained effect and cost-effectiveness of this therapy,13,14 when compared with complex modeling of 3-month outcomes in recent endovascular trials during 20 years with many underlying assumptions.15 The need for later outcomes to determine sustainability and cost-effectiveness is true not only for severe strokes but also for mild and moderate strokes.
In summary, for severe strokes, a time point later than 3 months may be the most sensitive measure to detect potential treatment effects in phase III trials and also provides the best and most accurate data for sustainability and cost-effectiveness.
Rebuttal by Dr Lees
The essence of the difference between Dr Broderick’s stance and my own lies in 2 points: experience across past trials and arguments around practical considerations. Dr Broderick quotes 2 trials in which an outcome difference between treated and control groups expanded after 3 months. Such examples may occur, but I am reluctant to accept a subgroup chosen post hoc from the IMS III trial when the trial itself was closed for futility.8 Similar approaches have previously informed us that being born under a certain star sign confers considerable advantage in terms of treatment benefit. Validation is necessary. The data from MISTIE II trial are also highly selected: whereas the 11% treatment difference at 6 months was based on 52 treated and 38 control patients, the 14% difference at 12 months was based on only 23 versus 25 patients.9,10 These examples are not a robust basis for trial design.
The second issue concerns practicality, ethics, patient retention, etc, that were not considered by Dr Broderick. Within the MISTIE II trial example, 6 of 89 patients (7%) due for 6-month follow-up had already been lost from the trial. An intent-to-treat analysis will be diluted by such losses; a conservative analysis would negate any incremental advantage of longer observation. We agree that demonstrating sustainability is desirable, but I submit that the primary end point of the trial may still be at 3 months, with unrestricted follow-up by electronic data capture thereafter, thus avoiding expense and delay in revealing a practice-changing result.
Comments by Drs Selim and Molina
Little did we know when we posed the question about the optimal timing for outcome assessment post stroke to our guests that it would fuel such a fiery debate! A seemingly simple question at first glance aimed to distinguish between 2 important aspects of poststroke recovery—the speed of recovery and the full extent of it. Drs Lees and Broderick seem to focus on the latter and cite different studies to support their viewpoints. Dr Lees argues that there is no value in assessing outcome beyond 3 months and that the only added value from extending follow-up is to demonstrate durability of effect and to assess health economics. He argues that extending the assessment of outcome beyond 3 months is cost-ineffective and unethical, and it should be reserved for preventive interventions. Dr. Broderick, on the other hand, argues that assessment of outcome at a later than 3-month time point may be the most sensitive measure to detect potential treatment effects and provides the most accurate data for sustainability and cost-effectiveness. Although we agree with Dr Lees that subgroup analyses from IMS III and MISTIE II trials should be interpreted with caution and that thrombectomy and hemicraniectomy trials are markedly different, the data still suggest that delayed recovery past 3 months is not that uncommon. It is intuitive that patients who reach their near-full extent of recovery by 3 months will continue to do well at later time points, but the reverse is not true. Failure to achieve full recovery by 3 months does not necessarily negate the possibility for later recovery. Needless to say that delayed recovery in some patients is still worthwhile to determine the overall benefit of an intervention and could have important implications from a patient, family, and provider perspectives. Dr Lees’s concern that convergence of outcomes with extended follow-up as a result of intervening death and comorbidities could dilute treatment effects is valid, but these variables can be minimized with appropriate randomization schema. We concur that the timing of primary end point assessment should be tailored according to stroke severity, comorbidities of the target population, and expected effects of the intervention.
Rebuttal by Dr Broderick
Dr Lees references the DECIMAL (Decompressive Craniectomy in Malignant Middle Cerebral Artery Infarcts) hemicraniectomy trial, which used 6 and 12 months, not 3 months, as the primary time points to measure functional outcome. There was no difference in the primary outcome of an mRS score of ≤3 between the 2 groups at 6 months (25% of the surgical group and 23% of the medical group). Yet at 1 year, 50% of the surgical group and 22% of medical group had mRS scores of 2 to 3, and only the surgical group had participants with an mRS score of 2. Clearly, a substantial proportion of patients with mRS scores of 3 to 4 at 3 months had functional improvement during the year, and the study had the greatest ability to detect therapeutic benefit at 1 year. The key issue is not the natural history of patients with a severe stroke at baseline. Many such patients die within 2 to 4 weeks from the stroke itself and do poorly overall, and stroke mortality by itself is a poor clinical end point for stroke trials. Rather, an effective therapy can positively alter the natural history of patients with severe strokes, when compared with standard therapy, and this benefit may be best demonstrated over a longer time window. Extending a study from 3 to 6 months represents little cost in terms of a single in-person or phone visit per patient by a study coordinator to determine functional outcome. Extending an entire trial for ≥3 months to incorporate a later end point does cost a little more but may provide greater power to detect a treatment benefit for patients with severe stroke, demonstrates the durability of the benefit that is important from a public health standpoint, and provides much better data for cost-effectiveness analyses than an earlier end point.
In summary, trials of patients with a severe acute stroke, such as those with intracerebral hemorrhage, ischemic stroke caused by internal carotid artery occlusion, or with major mass effect and incipient herniation, may benefit by measuring functional outcome at a time point >3 months.
All in all, we support collecting outcome data at 3 months to assess the speed of recovery (which could have important health-saving implications), but we also advocate for using phone calls at later time points (6 months and perhaps ≤12 months). Not only will this allow the assessment for the full extent of recovery, durability of benefit, delayed recovery, and hence the full effect of the intervention but also the progress of recovery of function on an individual level. The added cost is minimal, but the gathered data could be of substantial importance to our field and patients.
Dr Lees chaired the European Stroke Organization (ESO) Outcomes working group, chairs the Virtual International Stroke Trials Archive (VISTA), and is a member of the Stroke Thrombolysis Trialists Collaboration (STTC) and the ThRombEctomy and tPA (TREAT) collaboration. Dr Broderick receives research monies to the Department of Neurology and Rehabilitation Medicine from Genentech for his role on Steering Committee on PRISMS (A Study of the Efficacy and Safety of Activase [Alteplase] in Patients With Mild Stroke) trial and grant support from the National Institutes of Health and the National Institute of Neurological Disorders and Stroke (UC U01NS052220, MUSC U01NS054630, and U01NS077304). The other authors report no conflicts.
The opinions expressed in this article are not necessarily those of the editors or of the American Heart Association.
Presented in part at the International Stroke Conference of the American Heart Association, Los Angeles, CA, February 17–19, 2016.
- © 2016 American Heart Association, Inc.
- Hong KS,
- Saver JL
- Vahedi K,
- Vicaut E,
- Mateo J,
- Kurtz A,
- Orabi M,
- Guichard JP,
- et al
- Lees KR,
- Bath PM,
- Schellinger PD,
- Kerr DM,
- Fulton R,
- Hacke W,
- et al
- Emberson J,
- Lees KR,
- Lyden P,
- Blackwell L,
- Albers G,
- et al
- MacIsaac RL,
- Khatri P,
- Bendszus M,
- Bracard S,
- Broderick J,
- Campbell B,
- et al
- Palesch YY,
- Yeatts SD,
- Tomsick TA,
- Foster LD,
- Demchuk AM,
- Khatri P,
- et al
- 9.↵MISTIE II results. American Heart Association website: https://my.americanheart.org/idc/groups/ahamah-public/@wcm/@sop/@scon/documents/downloadable/ucm_436205.pdf. Accessed November 24, 2015.
- 10.↵MISTIE II: 365-day results demonstrate improved outcome and cost benefit. American Heart Association website: http://my.americanheart.org/idc/groups/ahamah-public/@wcm/@sop/@scon/documents/downloadable/ucm_449049.pdf. Accessed November 24, 2015.
- Kwiatkowski T,
- Libman R,
- Frankel M,
- Tilley B,
- Morganstern L,
- et al
- Ganesalingam J,
- Pizzo E,
- Morris S,
- Sunderland T,
- Ames D,
- Lobotesis K