Towards a Consensus-Based Classification of Childhood Arterial Ischemic Stroke
Background and Purpose—The implementation of uniform nomenclature and classification in adult arterial ischemic stroke (AIS) has been critical for defining outcomes and recurrence risks according to etiology and in developing risk-stratified treatments. In contrast, current classification and nomenclature in childhood AIS are often overlapping or contradictory. Our purpose was to develop a comprehensive consensus-based classification system for childhood AIS.
Methods—Using a modified-Delphi method, members of the International Pediatric Stroke Study (IPSS) developed the Childhood AIS Standardized Classification And Diagnostic Evaluation (CASCADE) criteria. Two groups of pediatric stroke specialists from the IPSS classified 7 test cases using 2 methods each: (1) classification typical of the individual clinician's current clinical practice; and (2) classification based on the CASCADE criteria. Group 1 underwent in-person training in the utilization of the CASCADE criteria. Group 2 classified the same cases via an online survey, including definitions but without training. Inter-rater reliability (IRR) was assessed via multi-rater unweighted κ-statistic.
Results—In Group 1 (with training), IRR was improved using CASCADE criteria (κ=0.78, 95% CI=[0.49, 0.94]), compared with typical clinical practice (κ=0.40, 95% CI=[0.11, 0.60]). In Group 2 (without training), IRR was lower than among trained raters (κ=0.61, 95% CI=[0.29, 0.77]), but higher than current practice (κ=0.23, 95% CI=[0.03, 0.36]).
Conclusions—A new, consensus-based classification system for childhood AIS, the CASCADE criteria, can be used to classify cases with good IRR. These preliminary findings suggest that the CASCADE criteria may be particularity useful in the setting of prospective multicenter studies in childhood-onset AIS, where standardized training of investigators is feasible.
The adoption of uniform nomenclature and a validated classification system in arterial ischemic stroke (AIS) in adults has been critical to the study of neurological outcomes, risk-stratified therapies and recurrence risk.1,2 Multiple classification systems exist for adult AIS that have been validated and used for the purposes of multicenter collaborative efforts. As an example, the Trial of ORG 10172 in Acute Stroke Treatment (TOAST) classification system has been utilized in more than 180 peer- reviewed publications in adult stroke, facilitating the study of differences in adult stroke subtypes with respect to etiology, risk factors (including genetic polymorphisms), treatment, and outcomes.3–5 Use of the TOAST criteria has, in turn, led to the development and successful conduct of clinical trials evaluating risk-stratified approaches to treatment in adult AIS.
Over the past decade, collaborative groups have made considerable progress in understanding the epidemiology, etiology, pathophysiology, outcomes, and prognostic factors in childhood AIS.6–9 Nevertheless, current research efforts are hampered by the lack of standardized nomenclature and classification in the childhood form of this disease.10–12 Initial attempts at classification in childhood AIS using adult criteria found that the majority of children with AIS do not meet criteria for atherothrombotic, cardioembolic or small-vessel disease, as seen in adults.13,14 Subsequent efforts toward childhood-specific nomenclature and classification in AIS have been proposed (the Pediatric Stroke Classification and the Sebire Criteria), but no one system has been widely adopted.15,16
The Pediatric Stroke Classification was created by adding subtypes that were identified in the literature to selected subtypes from the TOAST criteria.15 Although the system has good IRR, with ICC=0.92, it has not been widely adopted in the childhood stroke literature, possibly due to the lack of consensus development, overlap with the Sebire criteria nomenclature, and inability to classify changes in arteriopathy over time. Although the Sebire criteria are consensus-based, the authors did not attempt to categorize strokes of nonvascular etiology, making it an incomplete system for classification.16 In addition, these 2 systems also have overlapping and, sometimes, contradictory nomenclature, especially in regard to arteriopathies, which are common in childhood AIS. The pediatric rheumatology literature provides additional definitions for inflammatory arteriopathies by adapting the adult Calabrese criteria for primary angiitis of the nervous system, which have even further potential overlap with the Sebire and Pediatric Stroke Classification definitions for arteriopathy.17,18
In order to devise a consensus-based classification system and uniform nomenclature for childhood AIS, the International Pediatric Stroke Study (IPSS) established a working group in childhood AIS classification in November 2007. The objectives of this report are to describe the methodology used to develop the Childhood AIS Standardized Classification and Diagnostic Evaluation (CASCADE) criteria, and to provide evidence of inter-rater reliability using this new, comprehensive classification system in childhood AIS.
The IPSS is an international research network that created a registry of patients with pediatric cerebrovascular disease, established in 2003 with the long-term goal of developing multinational clinical trials in pediatric ischemic stroke.7,19 Investigators of the IPSS created a working group to construct a consensus-based classification system for childhood AIS. The working group consisted of 14 volunteers from the IPSS: 10 pediatric neurologists, 1 adult neurologist, 2 pediatric hematologists, and 1 pediatric rheumatologist. The initial version of the CASCADE criteria represented a compilation of criteria from 3 published systems: (1) the Pediatric Stroke Classification,15 (2) the Sebire definitions for cerebral arteriopathy,16 and (3) Calabrese criteria for childhood primary central nervous system vasculitis.17,20,21 In subsequent versions, the working group established a unifying theme for the new classification system: an anatomically based system. In order to minimize subjectivity, the primary classification is largely based on objective anatomic evidence of any identifiable disease contributing to stroke causation, including disorders of the heart and cervical and intracranial vasculature.
The CASCADE criteria was revised using a modified-Delphi method (Figure 1).22,23 Each revision incorporated critiques solicited from members of the working group regarding the clinical relevance and conceptual validity of the overall system as well as each subtype. The cochairs (T.B. and R.I.) evaluated all responses and incorporated the critiques and suggestions into the next revision. Reviewers were asked to specify their assessments as: (1) agreement, (2) agreement with minor modifications, or (3) disagreement. In addition, there was a section provided for additional comments and suggested revisions. The proposal for the CASCADE criteria underwent 4 revisions. When disagreements arose, the working group was polled regarding possible solutions, and the majority opinion was accepted.
Based on this consensus-driven process, the 4th revision was presented to the IPSS members attending the IPSS investigators' meeting in February 2009. After a 20-minute training session, 7 childhood stroke vignettes were presented to the membership by the working group chairs, and the IPSS members in attendance were asked to classify the cases using 2 methods: (1) the approach typical of the clinician's current practice, via entry of free text (individual classification), and (2) the CASCADE criteria. Group 1 was comprised of the 6 members who volunteered to classify all 7 cases. Each vignette was based on real cases selected from stroke registries at the Children's Hospital of Philadelphia or Children's Hospital Colorado and contained a synopsis of clinical history, selected neurovascular imaging, echocardiogram results, and thrombophilia testing. T.B. and R.I. selected real-life cases from the Children's Hospital of Philadelphia or Children's Hospital Colorado to represent a broad range of stroke etiologies, including cardioembolic intracranial arteriopathy (2 cases), dissection, postinfectious, multiple causes (moyamoya disease and cardiac catheterization procedure) and idiopathic. Group 2 was comprised of members of IPSS who did not participate in Group 1 and responded to a call for volunteers. They reviewed the same cases via an e-mail survey. Group 2 raters classified the cases in the same manner as Group 1, via individual classification and the CASCADE criteria but were not given a training session.
Description of the CASCADE Classification System
The primary CASCADE criteria15–17,24–26 are presented in Figure 2 and the Table. The IPSS defines childhood AIS as follows: (1) neurological deficit of acute onset; (2) radiographic image(s) (magnetic resonance imaging or computed tomography), showing cerebral parenchymal infarct(s) conforming to known arterial territory(ies) and corresponding to clinical manifestations; and (3) occurring in children 29 days to 18 years of age. The CASCADE criteria are not designed to classify perinatal stroke, defined as stroke occurring before 29 days of life, which has unique risk factors.27
The primary CASCADE criteria is based on anatomic site of disease (Table), which provides the capacity to classify any patient with childhood AIS into a single acute primary category at the time of initial diagnosis (within 1 month of presentation). Standardized definitions for vascular lesions are provided in online-only Supplemental Table 1a (http://stroke.ahajournals.org).
The classification at presentation is based on the location and nature of any identifiable anatomic abnormality involving the heart, great vessels in the neck, or the intracranial vessels that could explain AIS in an individual patient (Figure 2). It is based on the investigator's assessment of the anatomic site of disease after interpreting the results of the following clinical data: echocardiogram, vascular imaging (magnetic resonance angiography, computed tomography angiography, and/or conventional angiogram), and pattern of anatomic distribution of infarct as demonstrated by magnetic resonance imaging or computed tomography and clinical history.
There are wide variations in vascular imaging capabilities, algorithms, and interpretation across medical centers—and these can change over time. We designed this system to be applicable regardless of the specific imaging modality or techniques used in the evaluation of an individual patient. For the purposes of this classification system, computed tomography angiography and magnetic resonance angiography are considered an adequate evaluation, while conventional angiogram, when performed, is considered the gold standard.
In cases with multiple anatomic locations of disease potentially contributing to stroke mechanism (eg, dissection in a patient with congenital heart disease), the stroke is classified as multifactorial (and the multiple categories will be recorded). Finally, strokes in patients with normal cardiac and vascular imaging (that might be viewed as “unknown”) or strokes with an atypical anatomic etiology (ie, AIS during an endovascular procedure) are classified as “other.”
The inter-rater reliability was measured among raters for each of the major 7 classification categories represented by the cases. Agreement for individualized classification was defined as those cases that utilized the same or similar nomenclature within their response and was inclusive of multiple spellings and/or tenses. If agreement was uncertain, T.B. and M.T. reviewed the responses, with agreement being awarded as a default in uncertain situations. We generated a κ-statistic for each of these categories, utilizing the Fleiss unweighted κ-statistic for multiple raters.28 In addition, 95% confidence intervals were determined using the Wald method. Employing the parameters proposed by Landis and Koch,29 we considered a value above 0.8 to represent near perfect agreement; a value between 0.61 to 0.80, substantial agreement; a value 0.41 to 0.60, moderate agreement; a value 0.21 to 0.40, fair agreement; and a value <0.20, slight agreement.
In Group 1 (with training), 6 raters provided complete data on all test cases (ie, raters who classified all 7 subjects for both the CASCADE criteria and individual system). Using these data from the 6 raters, we calculated an unweighted κ-statistic as a measure of inter-rater reliability. In the 7 test cases presented to Group 1, interobserver agreement (average among subjects of the percentage of raters that give the majority score for each subject) and κ-statistics are better when using CASCADE criteria (90%, range=4/6 to 6/6 and κ=0.78, 95% CI=[0.49, 0.94]), as compared with using individual classification (64%, range=1/6 to 6/6 and κ=0.40, 95% CI=[0.11, 0.60]). The pairwise rater kappas for the CASCADE criteria (ie, each rater versus all others) ranged from 0.61 to 1.0, indicating moderate to excellent agreement across all raters.
In Group 2, 17 untrained raters evaluated the 7 test cases. Interobserver agreement and κ-statistics are better when using CASCADE criteria (82%, range=10/17 to 17/17 and κ=0.61, 95% CI=[0.29, 0.77]), as compared with using individual classification (52%, range=6/17 to 15/17 and κ=0.23, 95% CI=[0.03, 0.36]). The pairwise kappas for the CASCADE criteria ranged from 0.04 to 0.86.
The CASCADE criteria offer a novel and unique approach to classification of stroke subtype in childhood AIS compared with previous efforts. The criteria are derived from a consensus-based process involving pediatric AIS experts and are comprehensive with respect to childhood-onset AIS etiologies and, therefore, seem more likely to be widely adopted than previously proposed schemes. In addition, the CASCADE criteria unifies previously published classification systems with the aim of eliminating terms with overlapping and/or nonspecific definitions, such as focal cerebral arteriopathy, primary angiitis of the central nervous system of childhood, transient cerebral arteriopathy, and steno-occlusive arteriopathy.6,15–17 As arteriopathy is associated with increased recurrence risk and adverse outcomes,8,30 standardizing the nomenclature employed in describing vascular disorders will be essential to developing risk-stratified therapies in childhood AIS.
While the initial κ-statistic of the CASCADE criteria is encouraging (κ=0.61 to 0.78), it is based on a limited number of raters in an unpowered analysis of few cases. Future efforts to establish inter-rater reliability and validity are warranted, using larger cohorts, prospectively collected cases and consistently trained raters. The decreased κ found in Group 2 highlights the need for training to maximize the reliability of the CASCADE criteria. In addition, the discrepancy between groups may have resulted from Group 1 sharing their answers with each other (although they were asked not to do so) or a discrepancy between the imaging qualities presented to Group 1 and Group 2. The importance of imaging quality and the need for training (possibly Web-based) before classification should be addressed by future studies of reliability. Finally, construct validity needs testing, possibly by determining the predictive utility of the CASCADE criteria on recurrence risk and clinical outcomes.
A limitation of the system is its reliance on consensus expert opinion when there is a lack of evidence-based data. In addition, the primary classification is anatomically based but has subcategories that are not purely anatomic. This compromise was reached within the modified Delphi method in order to include established and accepted clinical definitions (such as dissection or Takayasu arteritis) within the 7 major anatomic categories. Similar to the initial TOAST classification, the CASCADE criteria provide a starting point for consensus-based classification in childhood AIS and has the potential for ongoing modification as new information about childhood-onset AIS is uncovered.
Additionally, there are many important risk factors that are not related to structural disease of the heart or blood vessels, such as inflammation, thrombophilias, genetic syndromes, hemoglobinopathies, and infections. Future modifications of the CASCADE criteria will need to further unify and elaborate classification of these factors in a secondary classification system, using the same modified-Delphi methods, as well as further test our ability to revise classification beyond the acute period of childhood AIS. This secondary classification is under development (online-only Supplemental Table 1b) and will be further defined and validated in future studies. In addition, a temporal dimension to the criteria (online-only Supplemental Table 1c) is also under development and will classify the natural history of the patient's vascular disease as stable, reversible, or progressive, based on follow-up imaging.
Despite these limitations, this represents the first consensus-based classification system developed by a large consortium of childhood stroke investigators that is inclusive of all childhood AIS. These findings suggest that the CASCADE criteria may be particularity useful in the setting of prospective multicenter studies in childhood-onset AIS, where standardized training of investigators is feasible.
Sources of Funding
T.J.B. was supported by a grant from the National Institutes of Health, National Heart, Lung, and Blood Institute (1K23HL096895-01A1). M.J.M.-J. was supported by the Centers for Disease Control and Prevention (UR6/CCU820552). N.A.G. was supported by the National Institutes of Health, National Heart Lung and Blood Institute (1K23HL084055-01A1;). M.M.D. was supported by the National Institutes of Health, North and Central Texas Clinical and Translational Science Initiative (KL2RR024983) and First American Real Estate Information Services, Inc. R.I. was supported by the National Institutes of Health, National Institute of Neurological Disease and Stroke (R01NS050488).
We would like to acknowledge all of the IPSS members who participated as raters in the cases: Ruth Nass, Veronica Gonzalez Alvarez, Steven Pavlakis, Paola Pergami, Neil Friedman, Mubeen Rafay, Marta Isabel Hernandez Chavez, Mahendra Moharir, Lisa Abraham, Ilona Kopyta, Clarisa Maxit, Virginia Wong, Lisa M. Nassif, Vesna Brankovic-Sreckovic, and Anneli Kolk.
Ralph L. Sacco, MD, was the Guest Editor for this paper.
The online-only Data Supplement is available at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.111.624585/-/DC1.
- Received April 27, 2011.
- Accepted October 3, 2011.
- © 2012 American Heart Association, Inc.
- Kolominsky-Rabas PL,
- Weber M,
- Gefeller O,
- Neundoerfer B,
- Heuschmann PU
- Markus HS,
- Khan U,
- Birns J,
- Evans A,
- Kalra L,
- Rudd AG,
- et al
- Diener HC,
- Sacco R,
- Yusuf S
- Amlie-Lefond C,
- Bernard TJ,
- Sebire G,
- Friedman NR,
- Heyer GL,
- Lerner NB,
- et al
- Golomb MR,
- Fullerton HJ,
- Nowak-Gottl U,
- DeVeber G
- Dowling MM
- Williams LS,
- Garg BP,
- Cohen M,
- Fleck JD,
- Biller J
- deVeber G
- Dalkey NC
- Ozen S,
- Ruperto N,
- Dillon MJ,
- Bagga A,
- Barron K,
- Davin JC,
- et al
- Ay H,
- Benner T,
- Arsava EM,
- Furie KL,
- Singhal AB,
- Jensen MB,
- et al
- Raju TN,
- Nelson KB,
- Ferriero D,
- Lynch JK
- Fullerton HJ,
- Wu YW,
- Sidney S,
- Johnston SC