Reliability of Hematoma Volume Measurement at Local Sites in a Multicenter Acute Intracerebral Hemorrhage Clinical Trial
Background and Purpose—The reliability of hematoma volume (HV) measurement using the ABC/2 method in multicenter clinical trials is unknown. We determined the accuracy of ABC/2 method as an on-site test in comparison with the gold standard central HV-assessment and semiautomatic HV-assessment.
Method—We analyzed data from an acute intracerebral hemorrhage multicenter clinical trial. HV was measured by site investigators to determine enrollment eligibility (<60 cm3) using the ABC/2 method (on-site HV), and independently by the core-imaging laboratory using computer-based analysis (Medical Image Processing, Analysis, and Visualization [MIPAV] HV). HV was also measured by ABC/2 method (central HV) at the core-imaging laboratory to assess the difference in measurements between on-site (multiple raters with variable experiences) and central (single experienced rater) HVs.
Results—Fifty-six subjects were analyzed (mean age 62±15 years; 45% women). On-site HV values showed a significantly lower correlation with the MIPAV HV (r=0.63) than central HV and MIPAV HV (r=0.93) values. The correlation between on-site HV and central HV values was modest (r=0.51). A total of 73% of the central HVs were within 25% of the corresponding MIPAV HVs, whereas only 46% of the on-site HVs were within 25% of the corresponding MIPAV HVs (P<0.001). One protocol violation occurred as a result of inaccuracy of on-site HV measurement.
Conclusion—On-site HV measurements showed high variability, but the impact on the eligibility determination was small. Centralized remeasurements of HVs with feedback to the sites may increase the reliability of the on-site HV measurements.
Clinical Trial Registration—URL: http://www.clinicaltrials.gov/ct2/show/NCT00415610 or http://www.atach-2.com. Unique identifier: NCT00415610.
Because of prognostic significance, hematoma volume (HV)1 is frequently used as 1 of the criteria for enrollment or stratification in clinical trials involving intracerebral hemorrhage (ICH).2 The ABC/2 method is used for calculating HV, which is based on ellipsoid volume formula.3,4 Previous studies have reported good correlation between the ABC/2 method and computer-assisted methods.3,5,6 Kothari et al3 and Huttner et al6 reported high correlation between HV values derived from ABC/2 and computer-assisted methods.3,6 Sheth et al5 found that ABC/3 and ABC/2 methods underestimated median HV by 9 (3–28) cm3 and 4 (0.8–12) cm3, respectively, compared with computer-assisted method.5 However, the reliability of on-site HV measurement using ABC/2 by numerous investigators in multicenter ICH trials has not been evaluated. We analyzed data from a multicenter trial to (1) examine the accuracy of HV measurement using ABC/2 method by multiple investigators and (2) assess the impact of utilizing the ABC/2 method in eligibility determination.
We analyzed data from the multicenter, open-labeled Antihypertensive Treatment in Acute Cerebral Hemorrhage (ATACH) trial (ClinicalTrial.gov # NCT00415610). ATACH enrolled 60 subjects with supratentorial ICH and HV <60 cm32 to determine the tolerability and safety of blood pressure reduction in acute ICH.
On-Site HV Measurement
To determine eligibility, on-site HV was measured by the site investigators using the ABC/2 method on the initial computed tomographic (CT) scan: A, maximum hematoma diameter; B, maximum diameter perpendicular to A; and C, number of slices in vertical plane with hematoma multiplied by slice thickness.3,4 The study protocol, which included a description of the ABC/2 method, was available to investigators before trial initiation, but no dedicated training was provided. The ATACH trial did not have specific requirements for CT scan acquisition. Only 2 institutions, which contributed a total of 7 patients, had slice thickness of 4.5 mm; the rest of the CT scans had slice thickness of 5 mm.
HV Measurement at Core Imaging Laboratory
CT scans were sent to the core-imaging laboratory in DICOM format. Blinded imaging analysis was performed using the Medical Image Processing, Analysis, and Visualization (MIPAV) software (gold standard). This is an imaging analysis software package available online without charge from the National Institutes of Health.7 The hematoma margins are manually traced on each CT slice. MIPAV software automatically calculated the surface area per slice and the total HV (see Figure I in the online-only Data Supplement).
Central HV Measurement
HV was remeasured using the ABC/2 method independently by one of the authors (H.M.H.), using HV calculator, which calculates HV once the 3 dimensions are entered.8 Central ABC/2 was performed to (1) differentiate the error attributable to the mathematical limitations of ABC/2 formula and the inaccuracy of on-site ABC/2 attributable to multiple investigators and time sensitive ascertainment; and (2) identify hematoma characteristics that may disproportionately contribute to inaccuracy of on-site ABC/2, including irregularity, heterogeneity, segmentation, and presence of satellite lesions.6
Pair-wise correlations among the 3 methods were obtained by linear regression. Paired t test was used to pairwise compare on-site HV, central HV, and MIPAV HV. Percent error of on-site HV ([on-site – MIPAV]/MIPAV × 100) and central HV ([central – MIPAV]/MIPAV × 100) were calculated. The agreement with MIPAV (defined as being within 25% margin of error of the MIPAV-calculated HV) for each of the on-site ABC/2 and central ABC/2 methods was assessed using the McNemar test. All analyses were performed with JMP software (SAS Corp, Cary, NC).
Of the 60 subjects enrolled, 56 were included (mean age 62±15 years; 45% women), whereas 4 were excluded (hard copies [n=2] or missing [n=2] initial CT scans). The mean ± standard deviation (SD) MIPAV HV (13.9±14.2 cm3) was smaller than on-site HV (14.5±13.5 cm3) and central HV (15±17.2 cm3). Each of the 3 pair wise differences was statistically significant (P<0.0001), although the absolute differences (0.6±0.7 cm3 between MIPAV versus onsite HV; 1.1±3 cm3 between MIPAV versus central HV; and 0.5±3.7 cm3 between onsite versus central HV) did not appear clinically significant.
The was a strong correlation between central HV and MIPAV HV (r=0.93), moderate between on-site HV and MIPAV HV (r=0.63) and between on-site HV and central HV (r=0.51; Figure). Percent error of the on-site HV was larger than that of central HV (54%±338% versus 3%±33%). The central ABC/2 was able to calculate HV accurately in 73% of the patients (with 25% margin of error), whereas on-site ABC/2 was accurate in only 43% (McNemar χ2 P=0.0006) compared with MIPAV HV (Figure II in the online-only Data Supplement). Location was the only morphologic characteristic significantly associated with agreement between on-site HV and central HV (Table I in the online-only Data Supplement).
On-site HV measurement (54 cm3) led to inclusion of 1 subject, who, based on MIPAV HV (62 cm3), should have been excluded. Another subject was enrolled with an on-site HV of 52 cm3 and MIPAV measured HV of 56 cm3. However, central HV was 69 cm3 and could have led to unnecessary exclusion.
Both on-site ABC/2 and central ABC/2 method overestimated HV, which is in agreement with previous studies.5 The strong agreement between central ABC/2 and MIPAV (r=0.93) was significantly reduced when the ABC/2 method was used by multiple trial investigators with variable familiarity with the ABC/2 method within the short recruitment time window. The inter-rater reliability in previous studies has ranged from 0.63 to 0.99.3,9 The mean difference between central ABC/2 and MIPAV (1 cm3) was smaller than that observed by Hutter et al (4 cm3) but closer to that observed by Kothari et al (1.5 cm3). This may be a result of the fact that the mean HV in the ATACH trial (14 cm3) was smaller than Hutter et al (37 cm3) and closer to Kothari et al (26 cm3), supporting the concept that the accuracy of ABC/2 method is lower in large ICHs. One study estimated the error of ABC/2 method to be 10% for HV < 20 cm3, and 37% for HV > 40 cm3.9 Location (lobar versus deep) was the only morphological characteristic that affected the accuracy of on-site HV measurement, perhaps because of the small sample size or the homogeneity of the sample with specific inclusion/exclusion criteria.
A critical analysis of our data suggested 2 explanations for the differences between on-site and central ABC/2 method: (1) wrong decimal point placement, probably an error in unit conversion from mm to cm, (2) using too few or too many 2s, some on-site HV being exactly half or double the corresponding central HV. This is a common misinterpretation of the ABC/2 formula, in which the denominator of the formula is confused with the denominator used to calculate the C-dimension from 5-mm CT slices.10 Despite these limitations, on-site ABC/2 was reliable in determining eligibility in the ATACH trial and only led to 1 protocol violation.
In the ongoing phase III ATACH II trial, we addressed the issue of ABC/2 reliability by (1) incorporating ABC/2 method training in the educational material of the trial and requiring investigators to pass a test before site initiation, (2) requiring the entry of the 3 hematoma dimensions, not just the total HV, before randomization, ensuring accurate data entry, and (3) providing HV calculator as a mobile application.8
Sources of Funding
Antihypertensive Treatment in Acute Cerebral Hemorrhage (ATACH) was funded by National Institutes of Health grant RO-1-NS44976-01A2 (medication provided by ESP Pharma). Dr Qureshi has received funding from American Heart Association Established Investigator Award 0840053N, National Institute of Health grant U01-NS062091-01A2, and the Minnesota Medical Foundation, Minneapolis, MN.
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.112.667220/-/DC1.
- Received June 26, 2012.
- Revision received August 28, 2012.
- Accepted September 25, 2012.
- © 2012 American Heart Association, Inc.
- Broderick JP,
- Brott TG,
- Duldner JE,
- Tomsick T,
- Huster G
- Kothari RU,
- Brott T,
- Broderick JP,
- Barsan WG,
- Sauerbeck LR,
- Zuccarello M,
- et al
- Kwak R,
- Kadoya S,
- Suzuki T
- Huttner HB,
- Steiner T,
- Hartmann M,
- Köhrmann M,
- Juettler E,
- Mueller S,
- et al
- McAuliffe M,
- Lalonde E,
- McGarry D,
- Gandler W,
- Csaky K,
- Trus B
- 8.↵ATACH-II official website http://www.atach-2.com/ Accessed April 11, 2012.
- Wang CW,
- Juan CJ,
- Liu YJ,
- Hsu HH,
- Liu HS,
- Chen CY,
- et al
- Newman GC