Value of Central Event Adjudication
To the Editor:
The Perindopril Protection Against Recurrent Stroke Study (PROGRESS) Collaborative Group notes that the End Point Adjudication Committee (EPAC) process had no discernible impact on that trial’s statistical conclusions.1 However, we caution that this result is not necessarily representative and that an EPAC is a wise investment for most studies. This claim is supported by the following points.
First, the rejection rate of 9.9% seen in PROGRESS is low by our experience. Traditionally, one asks investigators to “cast a wide net” and send in any possible event, even if they believe it unlikely to be a true event, to ensure that no true events escape detection (ie, no false-negatives). This process leads to more spurious events, but that is rectified by the EPAC process to remove them (ie, no false-positives). This process ensures the best overall data. As mentioned by the authors, this rate of rejection can also be affected by the end points chosen. A “softer” end point such as unstable angina typically has a much larger rejection rate than a “harder” end point such as central nervous system hemorrhage. Furthermore, if local investigators submitting potential stroke end points are neurologists, the rate of rejection is lower than in cardiology-based multicenter trials in which typically 20% to 30% of stroke end points are rejected by central neurology adjudicators.
Second, failing to perform the adjudication process will result in more noise in the end point data. Even if this noise is balanced by treatment arm, this could significantly impact the study by reducing the statistical power. (For example, Arm A has 75 true events and is contaminated by 25 false-positives versus Arm B has 50 true events and is also contaminated by 25 false-positives. A statistical test using the investigator-assigned results would likely not result in a statistically significant result, but the adjudicated results would.)
Another way to look at this issue is at the rejection rate and whether it is differential by treatment arm. This appears not to have been an issue for PROGRESS, but it is virtually impossible to know this prospectively. This could be caused by actual mechanistic process of the treatment or perceived bias if the trial is not well blinded. (For example, Arm A has 120 potential events and a rejection rate of 20% and Arm B has 100 potential events but a rejection rate of 30%. A statistical test using the investigator-assigned results would likely not result in a statistically significant result, but the adjudicated results would.) Any open-label study must have the EPAC built in to mitigate investigator biases.
Clinical trial data sets are often analyzed to determine predictors of the outcome events in one or both treatment arms. A substantial fraction of nonevents seriously compromises assessment of independent predictors of events and absolute event rates.
Lastly, the US Food and Drug Administration has recently recognized the importance of the use of EPAC in the recent guidance for new diabetes mellitus Type 2 studies: “Sponsors should establish an independent cardiovascular endpoints committee to prospectively adjudicate, in a blinded fashion, cardiovascular events during all phase 2 and phase 3 trials.”2 Assuming consistent definitions are used across studies, using an EPAC has the benefit of allowing meta-analyses to be done within a class of medications without worrying that the investigator skill level or minimum threshold of reporting an event is different between the studies. In our experience, overall costs of the EPAC process are generally <1% of the total cost of the study, a wise investment to ensure a compelling primary end point whose use will not be debated after the fact.
- ↵Ninomiya T, Donnan G, Anderson N, Bladin C, Chambers B, Gordon G, Sharpe N, Chalmers J, Woodward M, Neal B; PROGRESS Collaborative Group. Effects of the end point adjudication process on the results of the Perindopril Protection Against Recurrent Stroke Study (PROGRESS). Stroke. 2009; 40: 2111–2115.
- ↵www.fda.gov/downloads/Drugs/GuidanceComplianceRegulatoryInformation/Guidances/ucm071627.pdf. Accessed on Sept 8, 2009.