Deriving Number-Needed-to-Treat and Number-Needed-to-Harm From the Saint I Trial Results
To the Editor:
In their recent publication regarding interpretation of the Stroke-Acute Ischemic NXY Treatment (SAINT I) trial,1 Koziol and Feng use the definition, but not the derivation method, for number-needed-to-treat (NNT) and number-needed-to-harm (NNH) that was previously described.2 As a result they arrive at biologically untenable estimates of NNT and NNH.
A challenge in deriving NNT and NNH values for ordinal scale responses is that parallel group trial results do not specify the within-patient correlation—the degree to which the rank order of patient outcomes is similar under control versus active therapy.2 Koziol and Feng implicitly assume that the within-patient correlation is 0. Although this is an approach suggested in the literature,3 for most disease states this assumption is biologically invalid and will result in random person effects being inappropriately counted as treatment effects. This artifact occurs in the analysis of Koziol and Feng, resulting in the derivation of unrealistically low values for the NNT (2.17 to 2.48) and NNH (2.39 to 2.75). These NNT and NNH values indicate that for every 100 patients treated with NXY-059, about 43 improve as a result of therapy and about 39 are harmed as a result of therapy. These estimates are highly implausible. The SAINT results provided evidence of only modest benefit and no evidence of harm of therapy.
Stroke is a condition in which baseline disease severity and other prognostic factors strongly influence the ordering of patient outcomes, under both control and intervention conditions.4,5 As a result, the within-patient correlation is far from 0. The expert joint outcome table specification method offers a means of estimating the within-patient correlation and providing biologically probable estimates of NNT and NNH.2 When applied to the SAINT I trial results, this approach yielded an NNT value of 9.8 (95% CI, 8.7 to 10.9) and an NNH value of infinity.6
These findings accord with Koziol and Feng’s fundamental suggestion that the treatment effect observed in SAINT I was modest. For every 100 patients treated with NXY-059, about 10 benefit by improving 1 or more grades on the modified Rankin Scale and none are harmed by worsening 1 or more grades on the modified Rankin Scale. This magnitude of treatment benefit is substantially less than that offered by tissue plasminogen activator, but the absence of harm is favorable. As Koziol and Feng suggest, patients, physicians, and payors must decide whether this degree of treatment benefit is clinically, as well as statistically, significant.
J.L.S. was a site subinvestigator in the SAINT II trial.
Koziol JA, Feng AC. On the analysis and interpretation of outcome measures in stroke clinical trials: lessons from the SAINT I study of NXY-059 for acute ischemic stroke. Stroke. 2006; 37: 2644–2647.
Guyatt GH, Juniper EF, Walter SD, Griffith LE, Goldstein RS. Interpreting treatment effects in randomised trials. BMJ. 1998; 316: 690–693.
Hacke W, Donnan G, Fieschi C, Kaste M, von Kummer R, Broderick JP, Brott T, Frankel M, Grotta JC, Haley EC Jr, Kwiatkowski T, Levine SR, Lewandowski C, Lu M, Lyden P, Marler JR, Patel S, Tilley BC, Albers G, Bluhmki E, Wilhelm M, Hamilton S; ATLANTIS Trials Investigators; ECASS Trials Investigators; NINDS rt-PA Study Group Investigators. Association of outcome with early stroke treatment: pooled analysis of ATLANTIS, ECASS, and NINDS rt-PA stroke trials. Lancet. 2004; 363: 768–774.
Saver JL. An evaluation of the statistical and outcome methods employed in the SAINT I trial. International Stroke Conference. Kissimmee, FL, February 17, 2006.