Response to Letter Regarding Article Entitled “A Simple, Assumption-Free, and Clinically Interpretable Approach for Analysis of Modified Rankin Outcomes”
We appreciate Dr Bath drawing attention to his group's work offering empirical evidence of the greater power in ordinal over dichotomous outcomes, the opportunity for adjustment for baseline conditions, and an approach using the numbers needed to treat (NNT) as a clinical index of magnitude of treatment effect.1
Taking these points in the reverse order, we compliment the Optimising the Analysis of Stroke Trials (OAST) group for raising the NNT as a clinical index of the magnitude of treatment effect.2 Interestingly, their article was concurrently reviewed with ours (published December 2011 and February 2012, respectively), and the overlap with our approaches is striking. The OAST proposal develops 2 primary approaches, with the second approach forming all matched pairs of patients with 1 patient from each treatment group and calculating proportions of patients who were better (Θ+ve) and worse (Θ−ve). We note that both of our approaches are identical to this point. Table 2 of our publication shows for the NINDS (National Institute of Neurological Disorders and Stroke) tissue plasminogen activator (tPA) trial there are 46 016 (48%) pairs with better modified Rankin Scale score outcomes for tPA, 30 178 (31%) pairs with better outcomes for placebo, and 19 596 (20%) with tied outcomes; hence, Θ+ve=0.48 and Θ−ve=0.31.2 There are only 2 subsequent substantial differences.
For the clinical index creation, the OAST group suggests calculation of the NNT as 1/(Θ+ve−Θ−ve), eg, tPA trial 5.88 [1/(0.48–0.31)], with a supporting statement similar to, “We must treat 5.88 patients to have 1 patient have a superior outcome with tPA.” We propose to calculate the proportion of these pairs with a better outcome for tPA as Θ+ve/(Θ+ve−Θ−ve), eg, tPA trial 0.61 [0.48/(0.48+0.31)] and a supporting statement, “When treatment made a difference, patients treated with tPA did better 61% of the time.” Both approaches express a degree of excess or information favoring a treatment. Which index is more easily interpreted is a matter of opinion, and it may be useful to express both.
For test of significance creation, the OAST takes an asymptotic parametric approach to provide confidence intervals for the NNT. We do note that the OAST article may be incorrectly stating the standard error for NNT, which appears to be that for the absolute risk difference (Θ+ve−Θ−ve) rather than the NNT. In addition, the standard error formula for the “unmatched” case may not fully capture the correlated nature of the pair-wise combinations created to calculate the proportions and appears at odds with reported simulation results used for confirmation. Our approach is to test with permutation tests.3 As noted, assuming our confusion regarding the OAST approach is resolved, both approaches are reasonable.
In summary, the “spirit” of both of these simultaneously developed approaches has much in common, and the differences seem only to be a matter of personal taste.
Dr Bath also raises the point of adjustment for baseline covariates,4 which is directly addressed by their “matched” analysis (the method not described), which also can be implemented in our approach (and similarly in their unmatched approach) by forming strata based on these covariates before summing the pairs.2 Because this concern can be addressed by all approaches, it should be a nonissue.
Finally, Dr Bath notes that the OAST has previously demonstrated that the power from the Mann–Whitney U test is superior to dichotomous testing.5 There are 2 issues raised by this point.
The statistical literature is resplendent in describing the loss in power by dichotomizing continuous or ordinal data (Selvin in 1991 provided a formula for the sample size increase associated with dichotomization of normal data6); hence, it would be remarkable if the Mann–Whitney U test was not more powerful. The gain in power relative to dichotomous analysis was not our motivation; rather, it was to find a clinically interpretable index for the ordinal modified Rankin Scale. Our proposed statistical test flowed from this index.
Dr Bath seems to be suggesting that we are proposing a Mann–Whitney U, which is not the case. Rather, the clinically interpretable parameter we suggested gave rise to a test related to the Mann–Whitney U, but it differs in its management of tied pairs of patients.
In summary, we compliment the OAST for their excellent work encouraging the use of ordinal scales for the analysis of modified Rankin Scale data, which will result in smaller studies. However, in the United States, these efforts have been discouraged by federal agencies, not on the basis of power but on the basis of clinical interpretability, which is the motivation for our report.
George Howard, DrPH
University of Alabama at Birmingham
Department of Biostatistics
Stroke welcomes Letters to the Editor and will publish them, if suitable, as space permits. Letters must reference a Stroke published-ahead-of-print article or an article printed within the past 3 weeks. The maximum length is 750 words including no more than 5 references and 3 authors. Please submit letters typed double-spaced. Letters may be shortened or edited. Include a completed copyright transfer agreement form (available online at http://stroke.ahajournals.org and http://submit-stroke.ahajournals.org).
- © 2012 American Heart Association, Inc.
- Bath PMW
- Howard G,
- Waller JL,
- Voeks JH,
- Howard VJ,
- Jauch EC,
- Lees KR,
- et al
- Optimising the Analysis of Stroke Trials (OAST) Collaboration,,
- Gray LJ,
- Bath PM,
- Collier T
- Optimising Analysis of Stroke Trials (OAST) Collaboration,,
- Bath PM,
- Gray LJ,
- Collier T,
- Pocock S,
- Carpenter J
- Selvin S